WO2020134411A1 - Merchandise category recognition method, apparatus, and electronic device - Google Patents

Abstract

A merchandise category recognition method, apparatus, and electronic device, the method comprising: acquiring a target image of a merchandise region contained in an unmanned vending machine (S110); performing merchandise feature recognition on the target image to obtain a target feature vector of target merchandise contained in the target image (S120); determining from a preset reference feature vector library a first reference feature vector that matches the target feature vector (S130); and determining a merchandise category corresponding to the first reference feature vector to be the merchandise category of the target merchandise (S140). The described method may shorten the time spent on updating a new category of merchandise, thereby improving the overall efficiency of merchandise category recognition.

Description

Commodity category recognition method, device and electronic equipment

This application requires the priority of the Chinese patent application submitted to the China Patent Office on December 29, 2018, with the application number 201811638179.8 and the invention titled "A commodity class identification method, device and electronic equipment", the entire contents of which are incorporated by reference In this application.

Technical field

The present application relates to the field of data recognition technology, in particular to a method, device and electronic equipment for commodity category recognition.

Background technique

With the development of science and technology, the application range of unmanned vending machines is getting wider and wider. Among them, the unmanned vending machine can be an unmanned vending rack or an unmanned vending cabinet. For example, unmanned vending machines can be applied to supermarkets, offices, campuses, cafeterias, shopping malls, etc. Since the unmanned vending machine is not managed by staff, in order to ensure that the products in the unmanned vending machine can be normally and orderly sold, it is necessary to identify the category of the goods in the unmanned vending machine; for example, after adding the goods to the unmanned vending machine , You need to identify the category of the added product.

In the related art, the specific process of product category recognition is to use a sample image containing the product in the unmanned vending machine and the product category of the product in the unmanned vending machine to train a machine learning model, such as a convolutional neural network model, so When it is necessary to identify the product category, the trained machine learning model is used to identify the product category of the product in the unmanned vending machine.

Since the machine learning model used in the related art is trained based on the fixed product category in the unmanned vending machine, then, once a new category product needs to be added, in order to be able to identify the product category of the new category product, it is necessary to retrain the machine learning Model, which undoubtedly leads to a longer time for the new process of the new category of goods, which in turn affects the overall efficiency of commodity category recognition.

Summary of the invention

The purpose of the embodiments of the present application is to provide a method, device and electronic device for commodity category recognition, so as to shorten the time consumed by the new category commodity updating process, and thereby improve the overall efficiency of commodity category recognition. The specific technical solutions are as follows:

In a first aspect, an embodiment of the present application provides a method for identifying commodity categories, the method including:

Get the target image containing the product area in the unmanned vending machine;

Performing commodity feature recognition on the target image to obtain a target feature vector of the target commodity contained in the target image;

From a preset reference feature vector library, determine a first reference feature vector that matches the target feature vector, where the reference feature vector library includes multiple reference feature vectors, and each reference feature vector corresponds to a commodity category;

The commodity category corresponding to the first reference feature vector is determined as the commodity category of the target commodity.

Optionally, the method further includes:

Obtain images of new and new products to be uploaded;

Performing commodity feature recognition on the image of the new category commodity to obtain a feature vector of the new category commodity;

Add the feature vectors of the new category of commodities to the reference feature vector library.

Optionally, the reference feature vector library includes multiple feature vector sub-libraries, and the product categories corresponding to any two reference feature vectors in the same feature vector sub-library satisfy a predetermined difference condition, and the two product categories satisfy the predetermined difference The condition is that the similarity between the appearances of the products under the two commodity categories is less than the first similarity;

The step of determining the first reference feature vector matching the target feature vector from the preset reference feature vector library includes:

A first feature vector sub-library is determined from a preset reference feature vector library, where the first feature vector sub-library contains: a reference feature vector corresponding to a commodity category that meets a predetermined matching condition, and the commodity category meets a predetermined The matching condition is: the similarity between the appearance of the commodity under the commodity category and the target commodity is greater than the second similarity;

From the first feature vector sub-library, determine a first reference feature vector that matches the target feature vector.

Optionally, the step of determining the first feature vector sub-library from the preset reference feature vector library includes:

For each feature vector sub-library in the preset reference feature vector library, the reference appearance data corresponding to each reference feature vector in the feature vector sub-library is determined. The reference appearance data corresponding to each reference feature vector is: the reference feature vector Appearance data of the products under the corresponding product categories;

Obtaining appearance data of the target product from the target image;

For each feature vector sub-library, calculate the similarity between the reference appearance data corresponding to each reference feature vector and the appearance data of the target product in the feature vector sub-library, if the calculated similarity includes greater than The similarity of the second similarity determines the feature vector sub-library as the first feature vector sub-library.

Optionally, a plurality of feature vector sub-libraries included in the reference feature vector library correspond to a plurality of unmanned vending machines;

The step of determining the first feature vector sub-library from the preset reference feature vector library includes:

In the preset reference feature vector library, the reference feature vector sub-library corresponding to the unmanned vending machine where the target product is located is determined as the first feature vector sub-library.

Optionally, the step of performing commodity feature recognition on the target image to obtain the target feature vector of the target commodity contained in the target image includes:

Based on the pre-trained convolutional neural network, perform commodity feature recognition on the target image to obtain the target feature vector of the target commodity contained in the target image;

Wherein, the convolutional neural network is trained based on a plurality of sample images containing commodities and the commodity category of the commodities contained in each sample image.

Optionally, the step of performing commodity feature recognition on the target image based on the pre-trained convolutional neural network to obtain the target feature vector of the target commodity contained in the target image includes:

Extract the target area where the target commodity contained in the target image is located;

Based on the pre-trained convolutional neural network, commodity feature recognition is performed on the target area to obtain the target feature vector of the target commodity contained in the target image.

Optionally, the step of determining the first reference feature vector matching the target feature vector from the preset reference feature vector library includes:

Calculating the similarity between each reference feature vector and the target feature vector in the preset reference feature vector library separately;

The reference feature vector with the largest similarity is determined as the first reference feature vector matching the target feature vector.

In a second aspect, an embodiment of the present application provides a product category identification device, the device including:

The image acquisition module is used to acquire the target image containing the product area in the unmanned vending machine;

A feature recognition module, configured to perform commodity feature recognition on the target image to obtain a target feature vector of the target commodity contained in the target image;

The feature vector determination module is used to determine a first reference feature vector matching the target feature vector from a preset reference feature vector library, wherein the reference feature vector library includes multiple reference feature vectors, each reference Feature vector corresponds to a commodity category;

The commodity category determination module is configured to determine the commodity category corresponding to the first reference feature vector as the commodity category of the target commodity.

Optionally, the device further includes: an update module;

The update module is used to obtain an image of a new category product to be uploaded; perform product feature recognition on the image of the new category product to obtain a feature vector of the new category product; and convert the feature vector of the new category product , Added to the reference feature vector library.

Optionally, the reference feature vector library includes multiple feature vector sub-libraries, and the product categories corresponding to any two reference feature vectors in the same feature vector sub-library satisfy a predetermined difference condition, and the two product categories satisfy the predetermined difference The condition is that the similarity between the appearances of the products under the two commodity categories is less than the first similarity;

The feature vector determination module includes:

The feature vector sub-library determination module is used to determine a first feature vector sub-library from a preset reference feature vector library, where the first feature vector sub-library contains: a reference to which the corresponding product category meets a predetermined matching condition Feature vector, the predetermined matching condition that the commodity category meets is: the similarity between the appearance of the commodity under the commodity category and the target commodity is greater than the second similarity;

From the first feature vector sub-library, determine a first reference feature vector that matches the target feature vector.

Optionally, the feature vector sub-library determination module is specifically used for:

For each feature vector sub-library in the preset reference feature vector library, the reference appearance data corresponding to each reference feature vector in the feature vector sub-library is determined. The reference appearance data corresponding to each reference feature vector is: the reference feature vector Appearance data of the products under the corresponding product categories;

Obtaining appearance data of the target product from the target image;

For each feature vector sub-library, calculate the similarity between the reference appearance data corresponding to each reference feature vector and the appearance data of the target product in the feature vector sub-library, if the calculated similarity includes greater than The similarity of the second similarity determines the feature vector sub-library as the first feature vector sub-library.

Optionally, a plurality of feature vector sub-libraries included in the reference feature vector library correspond to a plurality of unmanned vending machines;

The feature vector sub-library determination module is specifically used for:

In the preset reference feature vector library, the reference feature vector sub-library corresponding to the unmanned vending machine where the target product is located is determined as the first feature vector sub-library.

Optionally, the feature recognition module includes:

The feature recognition unit is used to perform commodity feature recognition on the target image based on the pre-trained convolutional neural network to obtain the target feature vector of the target commodity contained in the target image;

Wherein, the convolutional neural network is trained based on a plurality of sample images containing commodities and the commodity category of the commodities contained in each sample image.

Optionally, the feature recognition unit is specifically used for:

Extract the target area where the target commodity contained in the target image is located;

Based on the pre-trained convolutional neural network, commodity feature recognition is performed on the target area to obtain a target feature vector of the target commodity contained in the target image.

Optionally, the feature vector determination module is specifically used for:

Calculating the similarity between each reference feature vector and the target feature vector in the reference feature vector library;

The reference feature vector with the largest similarity is determined as the first reference feature vector matching the target feature vector.

In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;

Memory, used to store computer programs;

The processor, when used to execute the program stored on the memory, implements the commodity category identification method described in the first aspect.

According to a fourth aspect, an embodiment of the present application provides a computer-readable storage medium in which a computer program is stored, and when the computer program is executed by a processor, the commodity category recognition described in the first aspect is realized method.

The technical solution provided by the embodiment of the present application extracts the target feature vector of the target commodity when identifying the commodity category of the target commodity, and determines the first reference feature vector matching the target feature vector from the preset reference feature vector library, The product category corresponding to the first reference feature vector is determined as the product category of the target product. Since this solution implements commodity category recognition based on a preset reference feature vector library, when a new category product is added to an unmanned vending machine, the feature vector of the new category product can be used to update the reference feature vector library without retraining the machine Learning the model, therefore, this solution can shorten the time spent on the new process of new category of goods, which can improve the overall efficiency of commodity category recognition.

BRIEF DESCRIPTION

In order to more clearly explain the embodiments of the present application and the technical solutions of the prior art, the following briefly introduces the drawings required in the embodiments and the prior art. Obviously, the drawings in the following description are only For some embodiments of the application, for those of ordinary skill in the art, without paying any creative labor, other drawings may be obtained based on these drawings.

FIG. 1 is a flowchart of a method for identifying commodity categories provided by an embodiment of the present application;

2 is a schematic diagram of a product category identification device provided by an embodiment of the present application;

FIG. 3 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without making creative efforts fall within the protection scope of the present application.

In order to shorten the time consumed in the process of updating a new category of commodities, and thereby improve the overall efficiency of commodity category recognition, the embodiments of the present application provide a commodity category recognition method, device, and electronic equipment.

It should be noted that the execution subject of a method for identifying a commodity category provided in the embodiments of the present application may be a device for identifying a commodity category, and the apparatus for identifying a commodity category may run in an electronic device, where the electronic device may be The human vending machine may also be a background server or the like that is communicatively connected to the unmanned vending machine. The embodiment of the present application does not limit the electronic device.

In order to clarify the description of the solution, first, the application scenarios of the technical solution provided by the embodiments of the present application are described. In practical applications, there may be multiple unmanned vending machines, and each unmanned vending machine may be provided with several layers of shelves. For any unmanned vending machine, it can be placed on each floor of the unmanned vending machine An image acquisition device is installed on the top of the shelf, and the image collected by the image acquisition device may contain all the products in the shelf.

As can be seen from the above description, by installing multiple image acquisition devices in multiple unmanned vending machines, you can obtain the product images of all the products in multiple unmanned vending machines, and then identify each product image to obtain the content of each product image. Feature vectors of commodities, and establish a reference feature vector library based on the obtained feature vectors of commodities. The reference feature vector library may include feature vectors of various categories of merchandise in each vending machine. Among them, each set of feature vectors in the reference feature vector library corresponds to a commodity category.

In the first aspect, an article category identification method provided in an embodiment of the present application is introduced.

As shown in FIG. 1, a method for identifying a commodity category provided by an embodiment of the present application may include the following steps:

S110. Acquire a target image containing a product area in the unmanned vending machine.

As can be seen from the above description, an unmanned vending machine can be provided with an image collection device, and the image collection device can collect images containing the product area. After the image acquisition device acquires the image containing the product area, the electronic device as the execution subject can acquire the image containing the product area. For convenience of description, the image containing the product area may be referred to as the target image.

It should be noted that the electronic device as the execution subject can obtain the target image including the product area in the unmanned vending machine in the following two ways:

The first way: the electronic device as the execution subject can detect in real time whether the image acquisition device acquires the target image of the commodity area containing the unmanned vending machine, and if it detects that the image acquisition device acquires the target of the commodity area containing the unmanned vending machine For images, the electronic device can acquire the target image containing the product area of the unmanned vending machine from the image acquisition device.

The second way: After the image acquisition device collects the target image of the product area containing the unmanned vending machine, it can send the target image of the product area containing the unmanned vending machine to the electronic device as the execution subject, so as to execute The electronic device of the main body can acquire the target image of the merchandise area including the unsold shelves.

S120: Perform product feature recognition on the target image to obtain a target feature vector of the target product included in the target image.

After the electronic device as the execution subject obtains the target image, it can perform commodity feature recognition on the target image to obtain the target feature vector of the target commodity contained in the target image. Among them, there are many ways to identify the target image for commodity features and obtain the target feature vector. Exemplarily, in one embodiment, the step of performing commodity feature recognition on the target image to obtain the target feature vector of the target commodity contained in the target image may include:

Based on the pre-trained convolutional neural network, the target image is subjected to commodity feature recognition to obtain the target feature vector of the target commodity contained in the target image;

Among them, the convolutional neural network is obtained by training based on a plurality of sample images containing commodities and commodity categories of the commodities contained in each sample image.

Here, the plurality of sample images containing commodities may be: images collected by image collecting devices provided in each unmanned vending machine.

In this embodiment, the step of performing commodity feature recognition on the target image based on the pre-trained convolutional neural network to obtain the target feature vector of the target commodity contained in the target image may include:

Extract the target area where the target product contained in the target image is located;

Based on the pre-trained convolutional neural network, commodity feature recognition is performed on the target area to obtain the target feature vector of the target commodity contained in the target image.

It can be understood that, in the target image collected by the image collection device, there may be multiple target regions. When product feature recognition is required, product feature recognition can be performed on the target area where the target product is located in the target image.

In practical applications, the target area included in the target image can be set in advance. Alternatively, the target area included in the target image may also be automatically determined by the image acquisition device, etc. The embodiment of the present application does not limit the specific implementation manner of extracting the target area where the target product included in the target image is located.

In another embodiment, the target image may be subjected to product feature recognition based on a feature representation method such as a color histogram or a gradient histogram of the target image to obtain a target feature vector of the target product included in the target image.

It should be noted that the specific implementation manner of performing the product feature recognition on the target image to obtain the target feature vector of the target product contained in the target image as described above is merely an example, and should not constitute a limitation on the present application.

S130: Determine a first reference feature vector that matches the target feature vector from a preset reference feature vector library, where the reference feature vector library includes multiple reference feature vectors, and each reference feature vector corresponds to a commodity category.

After the target feature vector is obtained, the first reference feature vector matching the target feature vector can be determined from a preset reference feature vector library. It can be understood that, since the first reference feature vector matches the target feature vector, the product category corresponding to the first reference feature vector also matches the product category corresponding to the target feature vector.

In one embodiment, the step of determining the first reference feature vector matching the target feature vector from the preset reference feature vector library may include:

Calculate the similarity between each reference feature vector in the reference feature vector library and the target feature vector;

The reference feature vector with the largest similarity to the target feature vector is determined as the first reference feature vector.

Those skilled in the art can understand that there are many ways to calculate the similarity between the reference feature vector and the target feature vector, which will not be repeated here.

For a reference feature vector, if the similarity between the reference feature vector and the target feature vector is high, it means that the reference feature vector and the target feature vector match more closely. Therefore, the reference feature vector with the highest similarity to the target feature vector can be determined Is the first reference feature vector.

S140: Determine the commodity category corresponding to the first reference feature vector as the commodity category of the target commodity.

Since the first reference feature vector is the reference feature vector matching the target feature vector, the product category corresponding to the first reference feature vector is the product category corresponding to the target feature vector; and because the target feature vector is the feature vector of the target product Therefore, the product category corresponding to the target feature vector can be determined as the product category of the target product.

It can be seen that, through the technical solution provided by the embodiment of the present invention, when identifying the commodity category of the target commodity, the target feature vector of the target commodity is extracted, and the first reference matching the target feature vector is determined from the preset reference feature vector library The feature vector determines the commodity category corresponding to the first reference feature vector as the commodity category of the target commodity. Since this solution implements commodity category recognition based on a preset reference feature vector library, when a new category product is added to an unmanned vending machine, the feature vector of the new category product can be used to update the reference feature vector library without retraining the machine Learning the model, therefore, this solution can shorten the time spent on the new process of new category of goods, which can improve the overall efficiency of commodity category recognition.

In an implementation manner, the method for identifying commodity categories provided in the embodiments of the present application may further include:

Obtain images of new and new products to be uploaded;

Perform product feature recognition on the image of the new category product to obtain the feature vector of the new category product;

The feature vector of the new category of goods is added to the reference feature vector library.

It can be understood that after adding the feature vectors of the new category of commodities to the reference feature vector library, the product category recognition can be implemented for the new category of commodities according to the steps shown in S110-S140.

Optionally, in one embodiment, in order to further improve the accuracy of commodity category recognition, the reference feature vector library may include multiple feature vector sub-libraries, and any two reference feature vectors in the same feature vector sub-library correspond to The product category meets a predetermined difference condition, and the predetermined difference condition satisfied by the two product categories is: the similarity between appearances of the products under the two product categories is less than the first similarity;

Correspondingly, the step of determining the first reference feature vector matching the target feature vector from the preset reference feature vector library may include the following two steps:

Step 1: Determine a first feature vector sub-library from a preset reference feature vector library, where the first feature vector sub-library contains: a reference feature vector corresponding to a product category that meets a predetermined matching condition, the product category meets The predetermined matching condition is: the similarity between the appearance of the commodity corresponding to the commodity category and the target commodity is greater than the second similarity;

Step 2: Determine the first reference feature vector matching the target feature vector from the first feature vector sub-library.

It can be understood that different types of products may have similar appearance packaging. For products with similar appearance packaging, the similarity of corresponding feature vectors is also high. Therefore, if the feature vectors of products with similar appearance packaging are stored Into the same reference feature vector library, when actually identifying the product category, it may affect the recognition accuracy of the product category. Therefore, in order to further improve the accuracy of product category recognition, the feature vectors of commodities whose appearance packaging similarity is less than the preset similarity can be stored in the same reference feature vector sub-library, and the appearance packaging similarity is greater than Feature vectors of products with preset similarities are stored in different reference feature vector sub-libraries.

For example, suppose there are 10 different types of products, of which 8 different types of products have similar appearance packaging similarity to each other less than the preset similarity, and 2 other products have similar appearance packaging similarity to the preset Similarity. In this case, these two products with high similarity in appearance and packaging can be obtained through a confusion analysis method, where the confusion analysis method can be human eye observation judgment, merchant presetting, or algorithm analysis, etc. There is no specific limitation on the confusion analysis method. Then, the feature vectors of the above eight products are put into the same feature vector sub-library, and the feature vectors of the above two kinds of products are put into different feature vector sub-libraries. Here, the size of the preset similarity may be set according to actual conditions, and the size of the preset similarity is not specifically limited in the embodiment of the present application.

In practical applications, the feature vector sub-library can be associated with the unmanned vending machine, that is, each unmanned vending machine can correspond to a feature vector sub-library, and the feature vector sub-library contains the unmanned vending machine. Feature vectors of all commodities in the cargo plane. In this way, when identifying the product category of the product in the unmanned vending machine, it is possible to directly use the feature vector sub-library corresponding to the unmanned vending machine to perform product category recognition, thereby increasing the speed of product category recognition.

In step 1, from the preset reference feature vector library, there may be multiple specific implementation manners for determining the first feature vector sub-library. Exemplarily, in an embodiment, the step of determining the first feature vector sub-library from the preset reference feature vector library may include:

For each feature vector sub-library in the preset reference feature vector library, determine the reference appearance data corresponding to each reference feature vector in the feature vector sub-library, and the reference appearance data corresponding to each reference feature vector is: the reference Appearance data of products under the product category corresponding to the feature vector;

Obtain the appearance data of the target product from the target image;

For each feature vector sub-library, calculate the similarity between the reference appearance data corresponding to each reference feature vector in the feature vector sub-library and the appearance data of the target product, if the calculated similarity includes greater than the second similarity Degree of similarity, the feature vector sub-library is determined as the first feature vector sub-library.

The size of the second similarity can be set according to actual conditions, which is not specifically limited in the embodiments of the present application. It can be understood that the determined first feature vector sub-library contains the same reference feature vector as the target feature vector.

It can be understood that, since the reference feature vectors included in the feature vector sub-library are all extracted from the product image containing the product area of the unmanned vending machine, the corresponding reference feature vector can be obtained from the product image Reference appearance data. Among them, each feature vector sub-library may correspond to a commodity image library.

In another embodiment, multiple feature vector sub-libraries included in the reference feature vector library may correspond to multiple unmanned vending machines;

Correspondingly, in step 1, the step of determining the first feature vector sub-library from the preset reference feature vector library may include:

The reference feature vector sub-library corresponding to the unmanned vending machine where the target product is located in the preset reference feature vector library is determined as the first feature vector sub-library.

It is understandable that the target product is stored on the unmanned vending machine, so for each unmanned vending machine, when you want to identify the product category of any target product on the unmanned vending machine, you can The feature vector sub-library corresponding to the unmanned vending machine serves as the first feature vector sub-library.

With regard to step 2, the specific implementation manner of the first reference feature vector matching the target feature vector is determined from the first feature vector sub-library, which is determined to match the target feature vector from the preset reference feature vector library in S130 The first implementation manner of the first reference feature vector of is the same or similar, and will not be repeated here.

In a second aspect, an embodiment of the present application provides a product category identification device. As shown in FIG. 2, the device includes:

The image acquisition module 210 is used to acquire a target image containing the product area in the unmanned vending machine;

The feature recognition module 220 is used to perform commodity feature recognition on the target image to obtain a target feature vector of the target commodity contained in the target image;

The feature vector determination module 230 is configured to determine a first reference feature vector matching the target feature vector from a preset reference feature vector library, wherein the reference feature vector library includes multiple reference feature vectors, each The reference feature vector corresponds to a commodity category;

The commodity category determination module 240 is configured to determine the commodity category corresponding to the first reference feature vector as the commodity category of the target commodity.

Optionally, the device further includes: an update module;

The update module is used to obtain an image of a new category product to be uploaded; perform product feature recognition on the image of the new category product to obtain a feature vector of the new category product; and convert the feature vector of the new category product , Added to the reference feature vector library.

It can be seen that, through the technical solution provided by the embodiment of the present invention, when identifying the commodity category of the target commodity, the target feature vector of the target commodity is extracted, and the first reference matching the target feature vector is determined from the preset reference feature vector library The feature vector determines the commodity category corresponding to the first reference feature vector as the commodity category of the target commodity. Since this solution implements commodity category recognition based on a preset reference feature vector library, when a new category product is added to an unmanned vending machine, the feature vector of the new category product can be used to update the reference feature vector library without retraining the machine Learning the model, therefore, this solution can shorten the time spent on the new process of new category of goods, which can improve the overall efficiency of commodity category recognition.

Optionally, the reference feature vector library includes multiple feature vector sub-libraries, and the product categories corresponding to any two reference feature vectors in the same feature vector sub-library satisfy a predetermined difference condition, and the two product categories satisfy the predetermined difference The condition is that the similarity between the appearances of the products under the two commodity categories is less than the first similarity;

The feature vector determination module includes:

The feature vector sub-library determination module is used to determine a first feature vector sub-library from a preset reference feature vector library, where the first feature vector sub-library contains: a reference to which the corresponding product category meets a predetermined matching condition Feature vector, the predetermined matching condition met by the product category is: the similarity between the appearance of the product under the product category and the target product is greater than the second similarity;

From the first feature vector sub-library, determine a first reference feature vector that matches the target feature vector.

Optionally, the feature vector sub-library determination module is specifically used for:

For each feature vector sub-library in the preset reference feature vector library, the reference appearance data corresponding to each reference feature vector in the feature vector sub-library is determined. The reference appearance data corresponding to each reference feature vector is: the reference feature vector Appearance data of the products under the corresponding product categories;

Obtaining appearance data of the target product from the target image;

For each feature vector sub-library, calculate the similarity between the reference appearance data corresponding to each reference feature vector and the appearance data of the target product in the feature vector sub-library, if the calculated similarity includes greater than the second The similarity of the similarity determines the feature vector sub-library as the first feature vector sub-library.

Optionally, a plurality of feature vector sub-libraries included in the reference feature vector library correspond to a plurality of unmanned vending machines;

The feature vector sub-library determination module is specifically used for:

In the preset reference feature vector library, the reference feature vector sub-library corresponding to the unmanned vending machine where the target product is located is determined as the first feature vector sub-library.

Optionally, the feature recognition module includes:

The feature recognition unit is used to perform commodity feature recognition on the target image based on the pre-trained convolutional neural network to obtain the target feature vector of the target commodity contained in the target image;

Wherein, the convolutional neural network is trained based on a plurality of sample images containing commodities and feature vectors of commodities contained in each sample image.

Optionally, the feature recognition unit is specifically used for:

Extract the target area where the target commodity contained in the target image is located;

Based on the pre-trained convolutional neural network, commodity feature recognition is performed on the target area to obtain the target feature vector of the target commodity contained in the target image.

Optionally, the feature vector determination module is specifically used for:

Calculating the similarity between each reference feature vector and the target feature vector in the preset reference feature vector library separately;

The reference feature vector with the largest similarity is determined as the first reference feature vector matching the target feature vector.

In a third aspect, an embodiment of the present application further provides an electronic device, as shown in FIG. 3, including a processor 301, a communication interface 302, a memory 303, and a communication bus 304, where the processor 301, the communication interface 302, and the memory 303 Complete communication with each other through the communication bus 304,

Memory 303, used to store computer programs;

The processor 301 is configured to implement the commodity category identification method described in the first aspect when executing the program stored in the memory 303.

It can be seen that, through the technical solution provided by the embodiment of the present invention, when identifying the commodity category of the target commodity, the target feature vector of the target commodity is extracted, and the first reference matching the target feature vector is determined from the preset reference feature vector library The feature vector determines the commodity category corresponding to the first reference feature vector as the commodity category of the target commodity. Since this solution implements commodity category recognition based on a preset reference feature vector library, when a new category product is added to an unmanned vending machine, the feature vector of the new category product can be used to update the reference feature vector library without retraining the machine Learning the model, therefore, this solution can shorten the time spent on the new process of new category of goods, which can improve the overall efficiency of commodity category recognition.

The communication bus mentioned in the above electronic equipment may be a peripheral component interconnection standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard structure (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, and a control bus. For ease of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the electronic device and other devices.

The memory may include random access memory (Random Access Memory, RAM), or non-volatile memory (Non-Volatile Memory, NVM), for example, at least one disk memory. Optionally, the memory may also be at least one storage device located away from the foregoing processor.

The aforementioned processor may be a general-purpose processor, including a central processor (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; it may also be a digital signal processor (Digital Signal Processing, DSP), dedicated integration Circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

According to a fourth aspect, an embodiment of the present application provides a computer-readable storage medium in which a computer program is stored, and when the computer program is executed by a processor, the commodity category recognition described in the first aspect is realized method.

It can be seen that, through the technical solution provided by the embodiment of the present invention, when identifying the commodity category of the target commodity, the target feature vector of the target commodity is extracted, and the first reference matching the target feature vector is determined from the preset reference feature vector library The feature vector determines the commodity category corresponding to the first reference feature vector as the commodity category of the target commodity. Since this solution implements commodity category recognition based on a preset reference feature vector library, when a new category product is added to an unmanned vending machine, the feature vector of the new category product can be used to update the reference feature vector library without retraining the machine Learning the model, therefore, this solution can shorten the time spent on the new process of new category of goods, which can improve the overall efficiency of commodity category recognition.

In yet another embodiment provided by the present application, there is also provided a computer program product containing instructions, which when run on a computer, causes the computer to execute the commodity category identification method described in the first aspect above.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations There is any such actual relationship or order. Moreover, the terms "include", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also those not explicitly listed Or other elements that are inherent to this process, method, article, or equipment. Without more restrictions, the element defined by the sentence "include one..." does not exclude that there are other identical elements in the process, method, article or equipment that includes the element.

Each embodiment in this specification is described in a related manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the device, the electronic device, and the storage medium, since they are basically similar to the method embodiments, the description is relatively simple. For the related parts, refer to the description of the method embodiments.

The above are only the preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application are included in the scope of protection of this application.

Claims (18)

1. A commodity category identification method, characterized in that the method includes:

   Get the target image containing the product area in the unmanned vending machine;

   Performing commodity feature recognition on the target image to obtain a target feature vector of the target commodity contained in the target image;

   From a preset reference feature vector library, determine a first reference feature vector that matches the target feature vector, where the reference feature vector library includes multiple reference feature vectors, and each reference feature vector corresponds to a commodity category;

   The commodity category corresponding to the first reference feature vector is determined as the commodity category of the target commodity.
2. The method according to claim 1, wherein the method further comprises:

   Obtain images of new and new products to be uploaded;

   Performing commodity feature recognition on the image of the new category commodity to obtain a feature vector of the new category commodity;

   Add the feature vectors of the new category of commodities to the reference feature vector library.
3. The method according to claim 1, wherein the reference feature vector library includes a plurality of feature vector sub-libraries, and the product categories corresponding to any two reference feature vectors in the same feature vector sub-library satisfy a predetermined difference condition, The predetermined difference condition met by the two commodity categories is: the similarity in appearance of the products corresponding to the two commodity categories is less than the first similarity;

   The step of determining the first reference feature vector matching the target feature vector from the preset reference feature vector library includes:

   A first feature vector sub-library is determined from a preset reference feature vector library, where the first feature vector sub-library contains: a reference feature vector corresponding to a commodity category that meets a predetermined matching condition, and the commodity category meets a predetermined The matching condition is: the appearance similarity between the product corresponding to the product category and the target product is greater than the second similarity;

   From the first feature vector sub-library, determine a first reference feature vector that matches the target feature vector.
4. The method according to claim 3, wherein the step of determining a first feature vector sub-library from a preset reference feature vector library includes:

   For each feature vector sub-library in the preset reference feature vector library, the reference appearance data corresponding to each reference feature vector in the feature vector sub-library is determined. The reference appearance data corresponding to each reference feature vector is: the reference feature vector Appearance data of the products under the corresponding product categories;

   Obtaining appearance data of the target product from the target image;

   For each feature vector sub-library, calculate the similarity between the reference appearance data corresponding to each reference feature vector and the appearance data of the target product in the feature vector sub-library, if the calculated similarity includes greater than The similarity of the second similarity determines the feature vector sub-library as the first feature vector sub-library.
5. The method according to claim 3, wherein a plurality of feature vector sub-libraries included in the reference feature vector library correspond to a plurality of unmanned vending machines;

   The step of determining the first feature vector sub-library from the preset reference feature vector library includes:

   In the preset reference feature vector library, the reference feature vector sub-library corresponding to the unmanned vending machine where the target product is located is determined as the first feature vector sub-library.
6. The method according to any one of claims 1 to 5, wherein the step of performing commodity feature recognition on the target image to obtain a target feature vector of the target commodity contained in the target image includes:

   Based on the pre-trained convolutional neural network, perform commodity feature recognition on the target image to obtain the target feature vector of the target commodity contained in the target image;

   Wherein, the convolutional neural network is trained based on a plurality of sample images containing commodities and the commodity category of the commodities contained in each sample image.
7. The method according to claim 6, wherein the step of performing commodity feature recognition on the target image based on the pre-trained convolutional neural network to obtain the target feature vector of the target commodity contained in the target image ,include:

   Extract the target area where the target commodity contained in the target image is located;

   Based on the pre-trained convolutional neural network, commodity feature recognition is performed on the target area to obtain the target feature vector of the target commodity contained in the target image.
8. The method according to claim 1, wherein the step of determining a first reference feature vector matching the target feature vector from a preset reference feature vector library includes:

   Calculating the similarity between each reference feature vector and the target feature vector in the preset reference feature vector library separately;

   The reference feature vector with the largest similarity is determined as the first reference feature vector matching the target feature vector.
9. A product category identification device, characterized in that the device includes:

   The image acquisition module is used to acquire the target image containing the product area in the unmanned vending machine;

   A feature recognition module, configured to perform commodity feature recognition on the target image to obtain a target feature vector of the target commodity contained in the target image;

   The feature vector determination module is used to determine a first reference feature vector matching the target feature vector from a preset reference feature vector library, wherein the reference feature vector library includes multiple reference feature vectors, each reference Feature vector corresponds to a commodity category;

   The commodity category determination module is configured to determine the commodity category corresponding to the first reference feature vector as the commodity category of the target commodity.
10. The apparatus according to claim 9, wherein the apparatus further comprises: an update module;

    The update module is used to obtain an image of a new category product to be uploaded; perform product feature recognition on the image of the new category product to obtain a feature vector of the new category product; and convert the feature vector of the new category product , Added to the reference feature vector library.
11. The device according to claim 9, wherein the reference feature vector library includes a plurality of feature vector sub-libraries, and the product categories corresponding to any two reference feature vectors in the same feature vector sub-library satisfy a predetermined difference condition, The predetermined difference condition met by the two commodity categories is: the similarity in appearance of the products corresponding to the two commodity categories is less than the first similarity;

    The feature vector determination module includes:

    The feature vector sub-library determination module is used to determine a first feature vector sub-library from a preset reference feature vector library, where the first feature vector sub-library contains: a reference to which the corresponding product category meets a predetermined matching condition Feature vector, the predetermined matching condition met by the product category is: the similarity in appearance of the product corresponding to the product category and the target product is greater than the second similarity;

    From the first feature vector sub-library, determine a first reference feature vector that matches the target feature vector.
12. The apparatus according to claim 11, wherein the feature vector sub-library determination module is specifically configured to:

    For each feature vector sub-library in the preset reference feature vector library, the reference appearance data corresponding to each reference feature vector in the feature vector sub-library is determined. The reference appearance data corresponding to each reference feature vector is: the reference feature vector Appearance data of the products under the corresponding product categories;

    Obtaining appearance data of the target product from the target image;

    For each feature vector sub-library, calculate the similarity between the reference appearance data corresponding to each reference feature vector and the appearance data of the target product in the feature vector sub-library, if the calculated similarity includes greater than The similarity of the second similarity determines the feature vector sub-library as the first feature vector sub-library.
13. The apparatus according to claim 11, wherein the reference feature vector library includes a plurality of feature vector sub-libraries and one-to-one correspondence with a plurality of unmanned vending machines;

    The feature vector sub-library determination module is specifically used for:

    In the preset reference feature vector library, the reference feature vector sub-library corresponding to the unmanned vending machine where the target product is located is determined as the first feature vector sub-library.
14. The device according to any one of claims 9 to 13, wherein the feature recognition module includes:

    The feature recognition unit is used to perform commodity feature recognition on the target image based on the pre-trained convolutional neural network to obtain the target feature vector of the target commodity contained in the target image;

    Wherein, the convolutional neural network is trained based on a plurality of sample images containing commodities and the commodity category of the commodities contained in each sample image.
15. The apparatus according to claim 14, wherein the feature recognition unit is specifically used for:

    Extract the target area where the target commodity contained in the target image is located;

    Based on the pre-trained convolutional neural network, commodity feature recognition is performed on the target area to obtain the target feature vector of the target commodity contained in the target image.
16. The apparatus according to claim 9, wherein the feature vector determination module is specifically used for:

    Calculating the similarity between each reference feature vector and the target feature vector in the preset reference feature vector library separately;

    The reference feature vector with the largest similarity is determined as the first reference feature vector matching the target feature vector.
17. An electronic device characterized by comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;

    Memory, used to store computer programs;

    The processor, when used to execute the program stored on the memory, implements the method steps of any one of claims 1-8.
18. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method steps of any one of claims 1-8 are realized.

Images