CN112784083A

CN112784083A - Method and device for acquiring category prediction model and feature extraction model

Info

Publication number: CN112784083A
Application number: CN201911064895.4A
Authority: CN
Inventors: 张严浩; 郑赟; 潘攀; 徐盈辉; 金榕
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2019-11-04
Filing date: 2019-11-04
Publication date: 2021-05-11

Abstract

The application discloses a method for acquiring a category prediction model, which comprises the following steps: acquiring a target object; acquiring a candidate similar object corresponding to the target object; determining candidate objects of interest to the user from the candidate similar objects; obtaining candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs; and training to obtain a target category prediction model by taking the target object and the candidate category information to which the target object belongs as sample data, wherein the target category prediction model is used for predicting the category information to which the object to be retrieved belongs according to the object to be retrieved. According to the method, the candidate category information of the target object is obtained according to the candidate similar object and the candidate concerned object corresponding to the target object, the target object and the candidate category information of the target object are used for training to obtain the target category prediction model, and the obtaining speed and the accuracy of the category prediction model can be improved.

Description

Method and device for acquiring category prediction model and feature extraction model

Technical Field

The application relates to an image retrieval technology, in particular to a method and a device for acquiring a category prediction model, an electronic device and a storage device. The application also relates to a method and a device for acquiring the feature extraction model, electronic equipment and storage equipment. The application also relates to an object retrieval method, an object retrieval device, electronic equipment and storage equipment.

Background

With the continuous development of computer technology, the image retrieval method is applied to various fields of people's life, and great convenience is brought to people's life. The image retrieval method is a method for obtaining similar images similar to the images to be retrieved by extracting the characteristic information of the images to be retrieved after the images to be retrieved are obtained. For example, in an e-commerce site, a user may use an image including product information as a search image and quickly select a product of interest to the user from similar images provided by the e-commerce site and corresponding to the search image.

The current image retrieval method is generally to train a category prediction model for extracting category information to which an object contained in an image belongs and train a feature extraction model for extracting image feature information by using sample data with supervision information; then, obtaining the feature information of the image to be retrieved by using the category prediction model and the feature extraction model; and then, acquiring a similar image corresponding to the image to be retrieved according to the characteristic information. The sample data used for training the category prediction model and the feature extraction model is from a common sample data set or a sample data set collected by a user, that is, the user collects original data by himself and then labels the collected original data by investing a large amount of manpower and material resources, so that sample data with supervision information is generated.

However, when the image retrieval method is actually applied, each field generally has its unique characteristics, for example, in the field of face recognition, attention is generally paid to how to accurately extract face feature information in an image, and a similar image is retrieved according to the extracted face feature information; in the e-commerce field, attention is generally paid to how to accurately extract feature information related to a commodity in an image, and a similar image is retrieved according to the extracted feature information related to the commodity. However, since common sample data is generally not targeted, when image retrieval is performed in various fields by using a category prediction model and a feature extraction model obtained by training according to the common sample data, the retrieval speed and accuracy are not good enough; after the sample data is acquired manually, although the category prediction model and the feature extraction model acquired by training the sample data can be used for image retrieval according to the characteristics of various fields in a targeted manner, when the sample data is acquired by the method and used for training to acquire the category prediction model and the feature extraction model, personnel are required to have specific professional field knowledge, and the difficulty in acquiring the sample data is increased invisibly; meanwhile, large-scale original data are manually marked to obtain targeted sample data, so that the workload is large, manpower and material resources are wasted, and further, the acquisition speed of a category prediction model and a feature extraction model is low, and the category prediction model and the feature extraction model cannot be rapidly put into use; moreover, if the error rate of the manually provided labeling information is high, the obtained results of the category prediction model and the feature extraction model are inaccurate.

Therefore, the current image retrieval method has the problem that a category prediction model and a feature extraction model with high accuracy cannot be conveniently and quickly obtained.

Disclosure of Invention

The embodiment of the application provides an acquisition method of a category prediction model, and aims to solve the problem that the existing image retrieval method cannot conveniently and quickly acquire the category prediction model with pertinence and high accuracy.

The embodiment of the application provides a method for acquiring a category prediction model, which comprises the following steps:

acquiring a target object; acquiring a candidate similar object corresponding to the target object; determining candidate objects of interest to the user from the candidate similar objects; obtaining candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs; and training to obtain a target category prediction model by taking the target object and the candidate category information to which the target object belongs as sample data, wherein the target category prediction model is used for predicting the category information to which the object to be retrieved belongs according to the object to be retrieved.

Optionally, the obtaining of the candidate similar object corresponding to the target object includes: acquiring a first candidate similar object corresponding to the target object; acquiring behavior data of a user for the first candidate similar object; and if behavior data of the user for the first candidate similar object is acquired, taking the first candidate similar object as the candidate similar object.

Optionally, the method includes: if the behavior data of the user for the first candidate similar object is not acquired, acquiring category information to be selected determined by the user, acquiring a second candidate similar object corresponding to the target object according to the category information to be selected, and taking the second candidate similar object as the candidate similar object.

Optionally, the determining a candidate attention object focused by the user from the candidate similar objects includes: acquiring behavior data of a user aiming at the candidate similar objects; and determining candidate attention objects concerned by the user according to the behavior data.

Optionally, the obtaining the candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs includes: and acquiring primary category information to which the candidate attention object belongs, and taking the primary category information to which the candidate attention object belongs as simple candidate category information to which the target object belongs.

Optionally, the obtaining the candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs includes: acquiring the primary category information to which the first candidate similar object belongs, taking the primary category information to which the first candidate similar object belongs as the negative candidate category information to which the target object belongs, and taking the primary category information to which the candidate attention object concerned by the user determined in the candidate similar objects belongs as the difficult candidate category information to which the target object belongs.

Optionally, the obtaining the candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs includes: acquiring leaf category information to which the candidate attention object belongs, wherein the leaf category information is category information at the last level of a category tree corresponding to the candidate attention object; and according to the leaf category information to which the candidate attention object belongs, acquiring virtual category information to which the target object belongs, and taking the virtual category information to which the target object belongs as the candidate category information to which the target object belongs, wherein the quantity of the virtual category information is more than that of the primary category information and is not more than that of the leaf category information.

Optionally, the obtaining, according to the leaf category information to which the candidate attention object belongs, the virtual category information to which the target object belongs includes: taking the leaf category information to which the candidate attention object belongs as a node, and acquiring a leaf category information matrix; performing graph embedding processing on the leaf category information matrix to obtain a leaf category vector corresponding to the leaf category information matrix; and acquiring the virtual category information of the target object according to the leaf category vector.

Optionally, the obtaining, according to the leaf category vector, the virtual category information to which the target object belongs includes: and carrying out cluster analysis on the leaf category vectors to obtain the virtual category information of the target object.

Optionally, the training, with the target object and the candidate category information to which the target object belongs as sample data, to obtain a target category prediction model includes: and training to obtain the target category prediction model by taking the target object and the simple candidate category information to which the target object belongs as sample data.

Optionally, the training, with the target object and the candidate category information to which the target object belongs as sample data, to obtain a target category prediction model includes: and training to obtain the target category prediction model by taking the target object, the negative candidate category information to which the target object belongs and the difficult candidate category information to which the target object belongs as sample data.

Optionally, the training, with the target object and the candidate category information to which the target object belongs as sample data, to obtain a target category prediction model includes: and training to obtain the target category prediction model by taking the target object and the virtual category information to which the target object belongs as sample data.

Optionally, the training, with the target object and the candidate category information to which the target object belongs as sample data, to obtain a target category prediction model includes: training to obtain the target category prediction model by using at least two sample data: the target object and the simple candidate category information to which the target object belongs, the negative candidate category information to which the target object and the target object belong, and the hard candidate category information to which the target object belongs, and the virtual category information to which the target object and the target object belong.

The embodiment of the present application further provides a method for acquiring a feature extraction model, including: acquiring a target object; acquiring a candidate similar object corresponding to the target object; determining candidate objects of interest to the user from the candidate similar objects; acquiring sample data according to the target object and the candidate attention object; and training by using the sample data to obtain a target feature extraction model, wherein the target feature extraction model is used for obtaining feature information of the object to be retrieved according to the object to be retrieved.

Optionally, the obtaining sample data according to the target object and the candidate attention object includes: acquiring virtual characteristic information corresponding to the target object according to the candidate attention object; and acquiring sample data according to the target object and the virtual characteristic information corresponding to the target object.

Optionally, the obtaining, according to the candidate attention object, virtual feature information corresponding to the target object includes: taking the candidate attention object as a node to obtain a candidate attention object matrix; performing graph embedding processing on the candidate attention object matrix to obtain a virtual feature vector corresponding to the candidate attention object; and acquiring virtual characteristic information corresponding to the target object according to the virtual characteristic vector of the candidate attention object.

Optionally, the obtaining, according to the virtual feature vector of the candidate attention object, virtual feature information corresponding to the target object includes: and performing cluster analysis on the virtual feature vectors of the candidate attention objects to acquire virtual feature information corresponding to the target object.

Optionally, the obtaining sample data according to the target object and the candidate attention object includes: and taking the candidate attention object as positive candidate labeling information corresponding to the target object, taking other objects in the candidate similar objects as negative candidate labeling information corresponding to the target object, and acquiring sample data according to the target object, the positive candidate labeling information and the negative candidate labeling information, wherein the other objects are objects except the candidate attention object in the candidate similar objects.

Optionally, the obtaining sample data according to the target object and the candidate attention object includes: acquiring positive sample sorting data corresponding to the target object, wherein the positive sample sorting data is obtained by sorting objects in the candidate similar objects in a descending order according to the similarity with the target object; and acquiring sample data according to the target object and the positive sample sorting data corresponding to the target object.

Optionally, the method includes: training to obtain a target feature extraction model by using at least one of the following sample data: the target object is a target object, the target object is a target object corresponding to the target object, and the target object is a target object corresponding to the target object.

An embodiment of the present application further provides an object retrieval method, including: acquiring an object to be retrieved; inputting the object to be retrieved into a target category prediction model to acquire category information to which the object to be retrieved belongs, wherein the target category prediction model is obtained by using the acquisition method of the category prediction model; inputting the object to be retrieved into a target feature extraction model corresponding to the category information, and acquiring feature information of the object to be retrieved, wherein the target feature extraction model is obtained by using the feature extraction model acquisition method; and acquiring a similar object corresponding to the object to be retrieved according to the characteristic information.

Optionally, the method further includes: if the screening instruction information for the similar objects is acquired, then: displaying attribute information of the similar objects; acquiring target attribute information of the similar object determined by a user; displaying similar objects corresponding to the target attribute information; and if the screening instruction information aiming at the similar object is not acquired, directly displaying the similar object.

The embodiment of the present application further provides a method for obtaining a commodity category prediction model, including: acquiring an original retrieval image containing a commodity object; acquiring a candidate similar image corresponding to the original retrieval image; determining a candidate attention image concerned by the user from the candidate similar images; obtaining candidate category information to which the commodity object belongs in the original retrieval image according to category information to which the commodity object contained in the candidate attention image belongs; and training to obtain a target commodity category prediction model by taking the original retrieval image and candidate category information to which the commodity object in the original retrieval image belongs as sample data, wherein the target commodity category prediction model is used for predicting the category information to which the commodity object in the image to be retrieved belongs according to the image to be retrieved.

The embodiment of the present application further provides a method for obtaining a commodity feature extraction model, including: acquiring an original retrieval image containing a commodity object; acquiring a candidate similar image corresponding to the original retrieval image; determining a candidate attention image concerned by the user from the candidate similar images; acquiring sample data according to the original retrieval image and the candidate attention image; and training by using the sample data to obtain a target commodity feature extraction model, wherein the target commodity feature extraction model is used for acquiring feature information of a commodity object in an image to be retrieved according to the image to be retrieved.

The embodiment of the present application further provides an image retrieval method, including: acquiring an image to be retrieved containing a commodity object; inputting the image to be retrieved into a target commodity category prediction model, and acquiring category information to which a commodity object in the image to be retrieved belongs, wherein the target commodity category prediction model is obtained by using the method for obtaining the commodity category prediction model; inputting the image to be retrieved into a target commodity feature extraction model corresponding to the category information, and acquiring feature information of a commodity object in the image to be retrieved, wherein the target commodity feature extraction model is obtained by using the method for acquiring the commodity feature extraction model; and acquiring a similar image corresponding to the image to be retrieved according to the characteristic information.

Optionally, the method further includes: if the screening instruction information for the commodity objects contained in the similar images is acquired, then: displaying the attribute information of the commodity object contained in the similar image; acquiring target attribute information of the commodity object contained in the similar image determined by the user; displaying similar images corresponding to the target attribute information; if the screening instruction information aiming at the commodity objects contained in the similar images is not acquired, directly displaying the similar images;

and if the accurate retrieval instruction information input by the user is not acquired, directly displaying the similar image.

Optionally, the attribute information includes stock quantity units of the commodity images included in the similar image.

The embodiment of the present application further provides an obtaining apparatus of a category prediction model, including: an object acquisition unit configured to acquire a target object; a similar object obtaining unit configured to obtain a candidate similar object corresponding to the target object; an attention object acquisition unit, wherein the user determines a candidate attention object concerned by the user from the candidate similar objects; the category information acquisition unit is used for acquiring candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs; and the model training unit is used for training to obtain a target category prediction model by taking the target object and the candidate category information to which the target object belongs as sample data, wherein the target category prediction model is used for predicting the category information to which the object to be retrieved belongs according to the object to be retrieved.

An embodiment of the present application further provides an electronic device, including:

a processor;

a memory for storing a program of an acquisition method of a category prediction model, the apparatus performing the following steps after being powered on and running the program of the acquisition method of the category prediction model by the processor:

The embodiment of the present application further provides a storage device, in which a program of an acquisition method of a category prediction model is stored, where the program is run by a processor and executes the following steps:

The embodiment of the present application further provides an obtaining apparatus for a feature extraction apparatus, including: an object acquisition unit configured to acquire a target object; a similar object obtaining unit configured to obtain a candidate similar object corresponding to the target object; an attention object acquisition unit for determining a candidate attention object focused by a user from the candidate similar objects; the sample data acquisition unit is used for acquiring sample data according to the target object and the candidate attention object; and the model training unit is used for training by using the sample data to obtain a target feature extraction model, wherein the target feature extraction model is used for acquiring feature information of the object to be retrieved according to the object to be retrieved.

a processor;

a memory for storing a program of an acquisition method of a feature extraction model, the apparatus performing the following steps after being powered on and running the program of the acquisition method of the feature extraction model by the processor:

acquiring a target object; acquiring a candidate similar object corresponding to the target object; determining candidate objects of interest to the user from the candidate similar objects; acquiring sample data according to the target object and the candidate attention object; and training by using the sample data to obtain a target feature extraction model, wherein the target feature extraction model is used for obtaining feature information of the object to be retrieved according to the object to be retrieved.

An embodiment of the present application further provides a storage device, in which a program of an acquisition method of a feature extraction model is stored, where the program is run by a processor and executes the following steps:

An embodiment of the present application further provides an object retrieving apparatus, including: the object acquisition unit is used for acquiring an object to be retrieved; a category information obtaining unit, configured to input the object to be retrieved into a target category prediction model, and obtain category information to which the object to be retrieved belongs, where the target category prediction model is obtained by using the method for obtaining the category prediction model; a feature information obtaining unit, configured to input the object to be retrieved into a target feature extraction model corresponding to the category information, and obtain feature information of the object to be retrieved, where the target feature extraction model is a model obtained by using an obtaining method of the feature extraction model; and the similar object acquisition unit is used for acquiring a similar object corresponding to the object to be retrieved according to the characteristic information.

a processor;

a memory for storing a program of an image retrieval method, the apparatus performing the following steps after being powered on and running the program of the image retrieval method by the processor:

acquiring an object to be retrieved; inputting the object to be retrieved into a target category prediction model to acquire category information to which the object to be retrieved belongs, wherein the target category prediction model is obtained by using the acquisition method of the category prediction model; inputting the object to be retrieved into a target feature extraction model corresponding to the category information, and acquiring feature information of the object to be retrieved, wherein the target feature extraction model is obtained by using the feature extraction model acquisition method; and acquiring a similar object corresponding to the object to be retrieved according to the characteristic information.

An embodiment of the present application further provides a storage device, in which a program of an image retrieval method is stored, where the program is executed by a processor, and executes the following steps:

The embodiment of the present application further provides an obtaining apparatus of a commodity category prediction model, including:

an original retrieval image acquisition unit for acquiring an original retrieval image containing a commodity object; a similar image acquisition unit for acquiring a candidate similar image corresponding to the original retrieval image; a focused image acquisition unit for determining a candidate focused image focused by the user from the candidate similar images; a candidate category information obtaining unit, configured to obtain candidate category information to which a commodity object in the original retrieval image belongs according to category information to which the commodity object included in the candidate attention image belongs; and the model obtaining unit is used for training to obtain a target commodity category prediction model by taking the original retrieval image and candidate category information to which the commodity object in the original retrieval image belongs as sample data, wherein the target commodity category prediction model is used for predicting the category information to which the commodity object in the image to be retrieved belongs according to the image to be retrieved.

a processor;

a memory for storing a program of an acquisition method of a commodity category prediction model, the apparatus being powered on and executing the program of the acquisition method of the commodity category prediction model by the processor to perform the steps of:

acquiring an original retrieval image containing a commodity object; acquiring a candidate similar image corresponding to the original retrieval image; determining a candidate attention image concerned by the user from the candidate similar images; obtaining candidate category information to which the commodity object belongs in the original retrieval image according to category information to which the commodity object contained in the candidate attention image belongs; and training to obtain a target commodity category prediction model by taking the original retrieval image and candidate category information to which the commodity object in the original retrieval image belongs as sample data, wherein the target commodity category prediction model is used for predicting the category information to which the commodity object in the image to be retrieved belongs according to the image to be retrieved.

An embodiment of the present application further provides a storage device, in which a program of an acquisition method of a commodity category prediction model is stored, where the program is executed by a processor to perform the following steps:

The embodiment of the present application further provides an obtaining apparatus for a commodity feature extraction apparatus, including: an original retrieval image acquisition unit for acquiring an original retrieval image containing a commodity object; a similar image acquisition unit for acquiring a candidate similar image corresponding to the original retrieval image; a focused image acquisition unit for determining a candidate focused image focused by the user from the candidate similar images; the sample data acquisition unit is used for acquiring sample data according to the original retrieval image and the candidate attention image; and the model obtaining unit is used for training by using the sample data to obtain a target commodity feature extraction model, wherein the target commodity feature extraction model is used for obtaining feature information of a commodity object in the image to be retrieved according to the image to be retrieved.

a processor;

a memory for storing a program of an acquisition method of a commodity feature extraction model, the apparatus performing the following steps after being powered on and running the program of the acquisition method of the commodity feature extraction model by the processor:

acquiring an original retrieval image containing a commodity object; acquiring a candidate similar image corresponding to the original retrieval image; determining a candidate attention image concerned by the user from the candidate similar images; acquiring sample data according to the original retrieval image and the candidate attention image; and training by using the sample data to obtain a target commodity feature extraction model, wherein the target commodity feature extraction model is used for acquiring feature information of a commodity object in an image to be retrieved according to the image to be retrieved.

An embodiment of the present application further provides a storage device, in which a program of an acquisition method of a product feature extraction model is stored, where the program is run by a processor and executes the following steps:

An embodiment of the present application further provides an image retrieval apparatus, including:

the retrieval-waiting image acquisition unit is used for acquiring a retrieval-waiting image containing a commodity object; a category information obtaining unit, configured to input the image to be retrieved into a target commodity category prediction model, and obtain category information to which a commodity object in the image to be retrieved belongs, where the target commodity category prediction model is obtained by using the method for obtaining the commodity category prediction model; a feature information obtaining unit, configured to input the image to be retrieved into a target commodity feature extraction model corresponding to the category information, and obtain feature information of a commodity object in the image to be retrieved, where the target commodity feature extraction model is a model obtained by using an obtaining method of the commodity feature extraction model; and the similar image acquisition unit is used for acquiring a similar image corresponding to the image to be retrieved according to the characteristic information.

a processor;

acquiring an image to be retrieved containing a commodity object; inputting the image to be retrieved into a target commodity category prediction model, and acquiring category information to which a commodity object in the image to be retrieved belongs, wherein the target commodity category prediction model is obtained by using the method for obtaining the commodity category prediction model; inputting the image to be retrieved into a target commodity feature extraction model corresponding to the category information, and acquiring feature information of a commodity object in the image to be retrieved, wherein the target commodity feature extraction model is obtained by using the method for acquiring the commodity feature extraction model; and acquiring a similar image corresponding to the image to be retrieved according to the characteristic information.

Compared with the prior art, the method has the following advantages:

the embodiment of the application provides a method for acquiring a category prediction model, which comprises the following steps: acquiring a target object; acquiring a candidate similar object corresponding to the target object; determining candidate objects of interest to the user from the candidate similar objects; obtaining candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs; and training to obtain a target category prediction model by taking the target object and the candidate category information to which the target object belongs as sample data, wherein the target category prediction model is used for predicting the category information to which the object to be retrieved belongs according to the object to be retrieved. Compared with the mode of acquiring sample data for training a category prediction model by using a common sample data set or manually labeling a target object, the method provided by the embodiment of the application acquires the candidate similar object corresponding to the target image, and conveniently and quickly acquires the information of the affiliated candidate category by determining the candidate concerned object concerned by the user in the candidate similar object, so that the finally acquired sample data is more pertinent; and then, training by taking the target object and the acquired candidate category information as sample data to obtain a target category prediction model, so that the acquisition speed and accuracy of the category prediction model can be improved.

The embodiment of the application provides a method for acquiring a feature extraction model, which comprises the following steps: acquiring a target object; acquiring a candidate similar object corresponding to the target object; determining candidate objects of interest to the user from the candidate similar objects; acquiring sample data according to the target object and the candidate attention object; and training by using the sample data to obtain a target feature extraction model, wherein the target feature extraction model is used for obtaining feature information of the object to be retrieved according to the object to be retrieved. Compared with the mode that a common sample data set or a target object is manually marked to obtain sample data for training the feature extraction model, the method provided by the embodiment of the application obtains the candidate similar object corresponding to the target image, then obtains the sample data according to the obtained candidate concerned object by determining the candidate concerned object concerned by the user in the candidate similar object, and trains the obtained target feature extraction model by using the obtained sample data, so that the obtaining speed and the accuracy of the category prediction model can be improved.

An embodiment of the present application provides an object retrieval method, including: acquiring an object to be retrieved; inputting the object to be retrieved into a target category prediction model to acquire category information to which the object to be retrieved belongs, wherein the target category prediction model is obtained by using the acquisition method of the category prediction model; inputting the object to be retrieved into a target feature extraction model corresponding to the category information, and acquiring feature information of the object to be retrieved, wherein the target feature extraction model is obtained by using the feature extraction model acquisition method; and acquiring a similar object corresponding to the object to be retrieved according to the characteristic information. By the method, more accurate similar objects corresponding to the objects to be retrieved can be obtained.

Drawings

Fig. 1 is a schematic diagram of an application scenario provided in a first embodiment of the present application.

Fig. 2 is a flowchart of a method for acquiring a category prediction model according to a first embodiment of the present application.

Fig. 3 is a schematic diagram of a process for acquiring first sample data according to a first embodiment of the present application.

Fig. 4 is a schematic diagram of an acquisition process of second sample data according to the first embodiment of the present application.

Fig. 5 is a schematic diagram of an acquisition process of third sample data according to the first embodiment of the present application.

Fig. 6 is a flowchart of a method for acquiring a feature extraction model according to a second embodiment of the present application.

Fig. 7 is a schematic diagram of a process for acquiring first sample data according to a second embodiment of the present application.

Fig. 8 is a schematic diagram of a second sample data obtaining process according to a second embodiment of the present application.

Fig. 9 is a schematic diagram of an acquisition process of third sample data according to the second embodiment of the present application.

Fig. 10 is a flowchart of an object retrieval method according to a third embodiment of the present application.

Fig. 11 is a flowchart of a method for obtaining a commodity category prediction model according to a fourth embodiment of the present application.

Fig. 12 is a flowchart of a method for acquiring a commodity feature extraction model according to a fifth embodiment of the present application.

Fig. 13 is a flowchart of an image retrieval method according to a sixth embodiment of the present application.

Fig. 14 is a schematic diagram of an apparatus for obtaining a category prediction model according to a seventh embodiment of the present application.

Fig. 15 is a schematic view of an electronic device according to an eighth embodiment of the present application.

Fig. 16 is a schematic diagram of an apparatus for acquiring a feature extraction model according to a tenth embodiment of the present application.

Fig. 17 is a schematic diagram of an image retrieval apparatus according to a thirteenth embodiment of the present application.

Fig. 18 is a schematic diagram of an apparatus for obtaining a prediction model of a commodity category according to a sixteenth embodiment of the present application.

Fig. 19 is a schematic diagram of an apparatus for acquiring a product feature extraction model according to a nineteenth embodiment of the present application.

Fig. 20 is a schematic diagram of an image retrieval apparatus according to a twenty-second embodiment of the present application.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of implementation in many different ways than those herein set forth and of similar import by those skilled in the art without departing from the spirit of this application and is therefore not limited to the specific implementations disclosed below.

In order to enable those skilled in the art to better understand the solution of the present application, the following detailed description is given to the specific application scenario of the embodiment of the present application based on the method for acquiring a category prediction model, the method for acquiring a feature extraction model, and the method for retrieving an object provided by the present application, where the target category prediction model acquired by the method for acquiring a category prediction model provided by the present application and the target feature extraction model acquired by the method for acquiring a feature extraction model provided by the present application are main technical means for object retrieval, so that the three methods provided by the present application are combined to describe the application scenario in detail.

Fig. 1 is a schematic diagram of an application scenario provided in the first embodiment of the present application. When the method for acquiring a category prediction model, the method for acquiring a feature extraction model and the object retrieval method provided by the embodiment of the application are implemented, a computing device usually trains in advance to acquire a target category prediction model capable of quickly and accurately acquiring category information of an object to be retrieved and a target feature extraction model capable of acquiring feature information of the object to be retrieved; then, in order to search for a similar object corresponding to the object to be retrieved, when the user issues an instruction for retrieving the similar object corresponding to the object to be retrieved to the computing equipment, the computing equipment responds to the instruction of the user, inputs the obtained object to be retrieved into the target category prediction model, and obtains category information to which the object to be retrieved belongs; then, inputting the object to be retrieved into a target feature extraction model corresponding to the category information, and acquiring feature information of the object to be retrieved; then, acquiring a similar object corresponding to the object to be retrieved according to the characteristic information of the object to be retrieved; the computing device then presents the similar objects to a user interface for viewing by the user.

It should be noted that, in this embodiment, the computing device for extracting, training, obtaining the target category prediction model and the target feature extraction model generally refers to a server-side computing device, and certainly, with the continuous development of technology, may also be other computing devices, and is not limited herein; in addition, when the object retrieval method provided by the embodiment of the present application is specifically implemented, the method can be applied to a scenario in which a client computing device interacts with a server computing device, and certainly can also be implemented in a client alone or in a server alone, where the client computing device generally refers to a mobile terminal device, such as a mobile phone, a tablet computer, and the like; and the server computing device is generally referred to as a server.

For example, in an e-commerce platform, a platform server computing device usually trains in advance to obtain a target commodity category prediction model capable of quickly and accurately obtaining category information of a commodity object and a target commodity feature extraction model capable of obtaining feature information of the commodity object, and when a user uploads an image to be retrieved containing the commodity object through a client computing device and issues a search instruction, the client computing device sends a request message for searching for a similar image of the image to be retrieved to the server computing device; after the server-side computing device acquires the request message, acquiring an image to be retrieved from the request message, inputting the image to be retrieved into a pre-acquired target commodity category prediction model, and acquiring category information of a commodity object contained in the image to be retrieved; then, inputting the image to be retrieved into a target commodity feature extraction model corresponding to the category information, and acquiring a feature vector of a commodity object contained in the image to be retrieved; then, according to the feature vector, similar feature vector information similar to the feature vector is obtained; then, acquiring similar commodity information corresponding to the similar feature vector information; and then, displaying the similar commodity image corresponding to the similar commodity information on a user interface for a user to check.

It should be noted that the above application scenario is only one specific embodiment of the method for acquiring a category prediction model, the method for acquiring a feature extraction model, and the object retrieval method provided in the present application, and the application scenario is provided to facilitate understanding of the above methods provided in the embodiments of the present application, and is not intended to limit the above methods provided in the embodiments of the present application.

Hereinafter, the method for acquiring the category prediction model, the method for acquiring the feature extraction model, and the object retrieval method provided by the present application will be described in detail with different embodiments.

Aiming at the problem that a category prediction model with high accuracy cannot be conveniently and quickly obtained when the conventional common sample data is used or manually marked to obtain the sample data and used for training the category prediction model, the first embodiment of the application provides an obtaining method of the category prediction model, and the method automatically obtains different types of sample data for training the category prediction model by carrying out multi-level data mining on user behavior data, such as user click data, and is explained with reference to fig. 2 to 5.

In step S201, a target object is acquired.

The target object refers to a carrier containing information to be retrieved, wherein the carrier may be an image or a video frame in a video resource, and certainly, may also be an audio resource. For example, the target object may be an image containing commodity information.

The target object is obtained from a data file stored in a computing device providing retrieval service according to a corresponding retrieval field identifier, wherein the data file may be a user behavior log, that is, a log file recording operation behaviors of a user on the target object and a candidate similar object corresponding to the target object when the user performs retrieval; the candidate similar object refers to a candidate object similar to the target object, which is presented on a user interface for a user to view after the computing device performs corresponding retrieval according to the object to be retrieved. For example, in an e-commerce platform, a computing device will typically generate a log record of a user's search object and the user's operation behavior on a candidate similar object corresponding to the search object, and store the log record in a user behavior log file. Therefore, the log file can generally acquire the history search target of the user and the operation behavior of the user on the candidate similar object corresponding to the history search target. Of course, the data file storing the target object may also be a data table; in this embodiment, for convenience of description, an example of acquiring a target object in an image format from a user behavior log stored in an e-commerce platform is taken as an example to describe an acquisition method of the category prediction model described in this embodiment.

Step S202, obtaining candidate similar objects corresponding to the target object.

The candidate similar object is a candidate object which is similar to the target object and is acquired by the computing equipment. When the target object is an image to be retrieved containing a commodity object, the candidate similar object is a candidate image containing the same or similar commodity object.

For example, in an e-commerce platform, for a single image to be retrieved that includes a "satchel," the platform will typically provide a plurality of similar images that are the same as or similar to the "satchel" for the user to view the corresponding merchandise through the similar images.

Generally, after a computing device searches according to a target object, the searched candidate similar objects corresponding to the target object are provided for a user to view, however, the following situations generally exist: 1. the candidate similar objects comprise objects concerned by the user, namely, when the primary category of the candidate similar objects provided by the computing device is the same as that of the target object, the relevance of the objects of the candidate similar objects to the target object is larger; 2. the candidate similar objects do not include the object concerned by the user, that is, when the primary category of the candidate similar object provided by the computing device is different from the category of the target object, the correlation between the object of the candidate similar object and the target object is usually small, and at this time, the user is usually required to manually switch the retrieval category, and the computing device retrieves and provides the secondary candidate similar object according to the category selected by the user.

For the first case, the obtaining of the candidate similar object corresponding to the target object includes: acquiring a first candidate similar object corresponding to the target object; acquiring behavior data of a user for the first candidate similar object; and if behavior data of the user for the first candidate similar object is acquired, taking the first candidate similar object as the candidate similar object. That is, after the computing device performs a search according to the target object, acquires a first candidate similar object corresponding to the target object, and provides the first candidate similar object for the user to view, if the first candidate similar object includes an object focused by the user, the computing device will usually record behavior data of the user operating on the first candidate similar object, so that if the behavior data of the user on the first candidate similar object can be acquired in a user behavior log file stored in the computing device, the first candidate similar object can be directly regarded as a final candidate similar object.

Fig. 3 is a schematic diagram illustrating a process of acquiring first sample data according to a first embodiment of the present application. As can be seen from fig. 3, when the computing device searches for the target object, the provided first candidate similar objects include objects focused by the user, that is, objects similar to the target object exist, so that the first candidate similar object may be directly used as the candidate similar object, and sample data may be acquired according to the acquired candidate similar object.

For the second case, the obtaining the candidate similar object corresponding to the target object includes: if the behavior data of the user for the first candidate similar object is not acquired, acquiring category information to be selected determined by the user, acquiring a second candidate similar object corresponding to the target object according to the category information to be selected, and taking the second candidate similar object as the candidate similar object. That is, after the computing device performs a first search according to the target object, acquires a first candidate similar object corresponding to the target object, and provides the first candidate similar object for the user to view, if the first candidate similar object does not include an object focused by the user, the computing device usually does not record behavior data of the user for operating on the first candidate similar object, so that if the behavior data of the user for the first candidate similar object is not acquired in the user behavior log file stored in the computing device, it is indicated that the first candidate similar object is not related to the target object, and the user is required to provide information related to the target object as auxiliary information, so that the computing device can perform a second search according to the auxiliary information. In this embodiment, the auxiliary information is information of a category to be selected of a target object determined by a user.

Fig. 4 is a schematic diagram illustrating a process of acquiring second sample data according to a first embodiment of the present application. As can be seen from fig. 4, when the computing device searches for the target object, the first candidate similar object provided by the computing device is likely to include an object that is not focused by the user, that is, the first candidate similar object does not include an object that is similar to the target object, and the maximum probability in this case is caused by inaccuracy of the category of the target object predicted by the computing device, so that the user is required to manually switch the category to be selected, and the computing device performs secondary search according to the category to be selected determined by the user, and provides a second candidate similar object obtained after the secondary search.

Step S203, determining candidate objects of interest concerned by the user from the candidate similar objects.

The candidate attention object is an object which is in the candidate similar objects and is concerned by the user, and the object is generally high in similarity or correlation with the target object.

Wherein the determining of the candidate attention object concerned by the user from the candidate similar objects comprises: acquiring behavior data of a user aiming at the candidate similar objects; and determining candidate attention objects concerned by the user according to the behavior data.

The behavior data may be click data of a user or viewing time of the user for a certain object in the candidate similar objects, where the click data of the user refers to that the user performs a click operation for the certain object in the candidate similar objects; the user's viewing time for an object of the candidate similar objects may be a dwell time of the object in the user interface recorded by the computing device.

Since the computing device records all the behavior data of the user in the corresponding data file, after the target object and the candidate similar object corresponding to the target object are acquired from the data file through the above steps S201 and S202, the candidate attention object concerned by the user may be further acquired. As shown in fig. 3-5, the candidate attention object concerned by the user can be determined according to the click data of the user in the candidate similar object.

In addition, the candidate similar objects and the candidate attention objects in the present embodiment are not limited to only include one object, and for the same target object, the candidate similar objects acquired by different users and the candidate attention objects focused by different users in the candidate similar objects are different.

For example, for the target object s _ img1, the candidate similar object acquired by different users may be a part of image of 1000 images d _ img 1-d _ img1000, and at the same time, the users may also focus on multiple candidate objects of interest among the candidate similar objects. For example, the candidate similar image corresponding to the user1 is d _ img 1-d _ img10, and the candidate attention objects thereof are d _ img8 and d _ img 10; the candidate similar images corresponding to the user2 are d _ img 5-d _ img15, and the candidate attention objects are d _ img5, d _ img8 and d _ img 10; and so on. Therefore, multiple candidate attention objects may be obtained for the same target object, and the attention degrees of different candidate attention objects by the users are different, and here, it can be considered a priori that the candidate attention objects with higher attention degrees by the users have relatively higher similarity to the target object, so that data mining can be performed on the candidate attention objects of the same target object for multiple users, and fine-grained candidate category information most relevant to the target object is mined.

Step S204, obtaining the candidate category information of the target object according to the category information of the candidate attention object.

First, when classifying a target object, a category tree corresponding to the target object generally exists, and in the category tree, there are first-level category information at a root node, leaf category information at a last level, and category information at other levels between the first-level category information and the leaf category information. For example, for a commodity object, there will usually be a more general first class category, such as a package; correspondingly, there is also information on the leaf category with the finest granularity, such as a lady's hand bag.

In practice, since the category information most relevant to the target object and having a moderate coverage cannot be effectively acquired as the labeling information of the target object, the category prediction model obtained by the current sample data training has a case of wrong category prediction. In view of this problem, the method for acquiring a category prediction model according to this embodiment performs multi-level data mining on the data after acquiring a target object, a candidate similar object corresponding to the target object, and a candidate object of interest corresponding to the target object, so as to extract category information most relevant to the target object from the data as candidate category information, and use the candidate category information as label information of the target object, thereby acquiring sample data for training the category prediction model, which will be described in detail below.

The obtaining of the candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs includes: and acquiring primary category information to which the candidate attention object belongs, and taking the primary category information to which the candidate attention object belongs as simple candidate category information to which the target object belongs, wherein the simple candidate category information is category information which can be simply acquired and is most relevant to the category information to which the target object actually belongs.

As shown in fig. 3, for a target object, a candidate object of interest to which a user focuses on can be obtained from candidate similar objects corresponding to the target object through one-time search, and therefore, the first-level category information to which the candidate object of interest belongs can be used as simple candidate category information of the target object, and the simple candidate category information can be used as labeling information of the target object to obtain first sample data for training a target category prediction model. For example, [ target object, package ] may be taken as the first sample data of the present embodiment.

In addition, after a second candidate similar object corresponding to the target object is obtained according to the category information to be selected determined by the user and the second candidate similar object is taken as the candidate similar object, the obtaining of the candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs includes: acquiring primary category information to which the first candidate similar object belongs, taking the primary category information to which the first candidate similar object belongs as negative candidate category information to which the target object belongs, and taking the primary category information to which a candidate attention object concerned by a user determined in the candidate similar object belongs as difficult candidate category information to which the target object belongs, wherein the negative candidate category information is category information which is not in the same category as the category information to which the target object belongs, and the difficult candidate category information is category information which is easy to be confused and difficult to predict.

As shown in fig. 4, when searching according to a target object, it is necessary for a user to switch category information before obtaining a candidate object of interest from candidate similar objects corresponding to the target object, and therefore, it may be considered that the category prediction difficulty of the target object is high, and when labeling the target object, first-order category information to which a first-order candidate similar object belongs may be used as negative candidate category information to which the target object belongs, first-order category information to which a candidate object of interest determined in second-order candidate similar objects belongs may be used as difficult candidate category information to which the target object belongs, and the negative candidate category information and the difficult candidate category information may be used as labeling information of the target object, so as to obtain second sample data for training a target category prediction model. For example, [ target object, bottle drink category, makeup category ] may be used as the second sample data of the present embodiment.

The obtaining of the candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs includes: acquiring leaf category information to which the candidate attention object belongs, wherein the leaf category information is category information at the last level of a category tree corresponding to the candidate attention object; and according to the leaf category information to which the candidate attention object belongs, acquiring virtual category information to which the target object belongs, and taking the virtual category information to which the target object belongs as the candidate category information to which the target object belongs, wherein the quantity of the virtual category information is more than that of the primary category information and is not more than that of the leaf category information.

The obtaining the virtual category information to which the target object belongs according to the leaf category information to which the candidate attention object belongs includes: taking the leaf category information to which the candidate attention object belongs as a node, and acquiring a leaf category information matrix; performing graph embedding processing on the leaf category information matrix to obtain a leaf category vector corresponding to the leaf category information matrix; and acquiring the virtual category information of the target object according to the leaf category vector. Wherein, the obtaining the virtual category information to which the target object belongs according to the leaf category vector comprises: and carrying out cluster analysis on the leaf category vectors to obtain the virtual category information of the target object.

The method includes the steps of mining on the basis of behavior data of a plurality of candidate attention objects aiming at the same target object, and forming a leaf category information matrix by taking leaf category information of the candidate attention objects as nodes, wherein in the leaf category information matrix, matrix elements corresponding to row and column nodes are the number of the candidate attention objects which correspond to the two types of leaf category information and are attended by users together. Then, performing Graph Embedding (Graph Embedding) processing on the leaf category information matrix through a deep random walk technology or other Graph Embedding algorithms to obtain a leaf category vector corresponding to the leaf category information matrix, and then performing cluster analysis on the leaf category vector to obtain a plurality of pieces of virtual category information which are most related to the target object and most concerned by the user, wherein the virtual category information is a plurality of virtual categories with the highest attention degree to the user, and therefore the number of the virtual category information is not high, for example, the number of the pieces of first-level category information is 10, and when the number of the leaf category information is 10000, the number of the pieces of virtual category information is only 30. In addition, when a leaf category vector is constructed using a graph embedding algorithm, Side Information (Side Information) can also be introduced to enable a better representation.

Step S205, taking the target object and the candidate category information to which the target object belongs as sample data, and training to obtain a target category prediction model, wherein the target category prediction model is used for predicting the category information to which the object to be retrieved belongs according to the object to be retrieved.

The training of the target object and the candidate category information to which the target object belongs as sample data to obtain the target category prediction model comprises the following steps: and training to obtain the target category prediction model by taking the target object and the simple candidate category information to which the target object belongs as sample data.

The obtained first sample data is input into a target category prediction model, supervised training is carried out on the target category prediction model, and the difference value between the category prediction result output by the target category prediction model and the labeling information in the sample data is continuously calculated through a loss function to adjust the parameters of the target category prediction model so that the target category prediction model is converged.

The training of the target object and the candidate category information to which the target object belongs as sample data to obtain the target category prediction model comprises the following steps: and training to obtain the target category prediction model by taking the target object, the negative candidate category information to which the target object belongs and the difficult candidate category information to which the target object belongs as sample data.

Inputting the obtained second sample data into the target category prediction model, performing supervised training on the target category prediction model, and continuously calculating the difference value between the category prediction result output by the target category prediction model and the labeling information in the sample data through a loss function to adjust the parameters of the target category prediction model so as to make the target category prediction model converge.

The training of the target object and the candidate category information to which the target object belongs as sample data to obtain the target category prediction model comprises the following steps: and training to obtain the target category prediction model by taking the target object and the virtual category information to which the target object belongs as sample data.

Inputting the obtained third sample data into the target category prediction model, performing supervised training on the target category prediction model, and continuously calculating a difference value between a category prediction result output by the target category prediction model and the labeling information in the sample data through a loss function to adjust parameters of the target category prediction model so as to make the target category prediction model converge.

In addition, in order to further improve the accuracy of the prediction result of the target category prediction model, the training to obtain the target category prediction model by using the target object and the candidate category information to which the target object belongs as sample data includes: training to obtain the target category prediction model by using at least two sample data: the target object and the simple candidate category information to which the target object belongs, the negative candidate category information to which the target object and the target object belong, and the hard candidate category information to which the target object belongs, and the virtual category information to which the target object and the target object belong.

The present embodiment provides a preferred implementation manner, that is, the target category prediction model is jointly trained by using the three sample data to obtain a target category prediction model containing multiple branches. In this manner, the overall loss function of the target category prediction model can be expressed as: zeta_Cat＝ζ_virtual+αζ_hart-aware+βζ_pairTherein, ζ_CatLoss function, ζ, for a predictive model representing the entire target category_virtualLoss function, ζ, for a branch predicted for a category using third sample data in a target category prediction model_hart-awareLoss function, ζ, for a branch predicted for a category using second sample data in a target category prediction model_pairAlpha and beta are respectively hyper-parameters for the loss function of the branch which uses the first sample data to carry out category prediction in the target category prediction model. When the target category prediction model acquires an input target object, the category information of the target object is predicted simultaneously through three branches in the target category prediction model, namely, first prediction category information corresponding to the target object is acquired through a branch used for performing category prediction according to first sample data, second prediction category information corresponding to the target object is acquired through a branch used for performing category prediction according to second sample data, third prediction category information corresponding to the target object is acquired through a branch used for performing category prediction according to third sample data, and then comprehensive evaluation is performed through weight values corresponding to the three branches to acquire the target category information corresponding to the target object.

By carrying out multi-level data mining on the data file which corresponds to the target object and contains the user behavior data, three sample data for training the target category prediction model can be conveniently, quickly and pertinently obtained, and then the category prediction model is trained through the three sample data, so that the obtaining speed and the accuracy of the category prediction model can be improved.

In summary, the method for obtaining a category prediction model according to the first embodiment of the present application includes: acquiring a target object; acquiring a candidate similar object corresponding to the target object; determining candidate objects of interest to the user from the candidate similar objects; obtaining candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs; and training to obtain a target category prediction model by taking the target object and the candidate category information to which the target object belongs as sample data, wherein the target category prediction model is used for predicting the category information to which the object to be retrieved belongs according to the object to be retrieved. Compared with the mode of acquiring sample data for training a category prediction model by using a common sample data set or manually labeling a target object, the method provided by the embodiment of the application acquires the candidate similar object corresponding to the target image, and conveniently and quickly acquires the information of the affiliated candidate category by determining the candidate concerned object concerned by the user in the candidate similar object, so that the finally acquired sample data is more pertinent; and then, training by taking the target object and the acquired candidate category information as sample data to obtain a target category prediction model, so that the acquisition speed and accuracy of the category prediction model can be improved.

In the above description, in order to further improve the accuracy of object retrieval corresponding to the above-mentioned category prediction model, the present application also provides a feature extraction model acquisition method, please refer to fig. 6, which is a flowchart of a feature extraction model acquisition method provided in the second embodiment of the present application, wherein some steps have been described in detail in the above-mentioned first embodiment, so that the description herein is relatively simple, and relevant points refer to some descriptions in the method provided in the first embodiment of the present application, and the processing procedure described below is only schematic.

Fig. 6 is a flowchart of a method for obtaining a feature extraction model according to a second embodiment of the present application, which is described below with reference to fig. 6 to 9.

In step S601, a target object is acquired.

Step S602, obtaining a candidate similar object corresponding to the target object.

The determining a candidate object of interest of the user from the candidate similar objects comprises: acquiring behavior data of a user aiming at the candidate similar objects; and determining candidate attention objects concerned by the user according to the behavior data.

Step S603, determining a candidate attention object focused by the user from the candidate similar objects.

Step S604, sample data is obtained according to the target object and the candidate attention object.

The obtaining sample data according to the target object and the candidate attention object comprises: acquiring virtual characteristic information corresponding to the target object according to the candidate attention object; and acquiring sample data according to the target object and the virtual characteristic information corresponding to the target object.

The acquiring, according to the candidate attention object, virtual feature information corresponding to the target object includes: taking the candidate attention object as a node to obtain a candidate attention object matrix; performing graph embedding processing on the candidate attention object matrix to obtain a virtual feature vector corresponding to the candidate attention object; and acquiring virtual characteristic information corresponding to the target object according to the virtual characteristic vector of the candidate attention object.

The obtaining of the virtual feature information corresponding to the target object according to the virtual feature vector of the candidate attention object includes: and performing cluster analysis on the virtual feature vectors of the candidate attention objects to acquire virtual feature information corresponding to the target object.

Fig. 7 is a schematic diagram of a process for acquiring first sample data according to a second embodiment of the present application. In a second embodiment of the present application, a process of mining behavior data of a user for multiple candidate attention objects of a same target object to obtain multiple pieces of virtual feature information corresponding to the target object is performed, where the process is different from the process of acquiring a leaf category information matrix according to a leaf category information to which the candidate attention object belongs as a node in the first embodiment, and further acquiring virtual category information corresponding to the target object according to the leaf category information matrix, and the process is as follows: and taking a plurality of candidate attention objects corresponding to the target object as nodes to obtain a candidate attention object matrix, wherein in the candidate attention object matrix, the numerical value between the row-column nodes is the similarity of the two candidate attention objects. Then, carrying out Graph Embedding (Graph Embedding) processing on the candidate attention object matrix through a depth random walk technology or other Graph Embedding algorithms to obtain a virtual feature vector corresponding to the candidate attention object; then, carrying out cluster analysis on the virtual feature vectors to obtain a plurality of pieces of virtual feature information which are most relevant to the target object and most concerned by the user; then, the obtained virtual feature information is used as the labeling information of the target object, so that the first sample data of the second embodiment of the present application can be obtained.

It should be noted that, in order to improve the capability of obtaining the finally obtained target feature extraction model to extract visual features, it is necessary to increase the number of virtual feature information as much as possible, so as to divide the candidate attention object and the target object into a feature space of a cluster with finer granularity, and according to the virtual feature information that reflects the cluster index embedded by the user's inter-click.

In addition, the obtaining sample data according to the target object and the candidate attention object includes: and taking the candidate attention object as positive candidate labeling information corresponding to the target object, taking other objects in the candidate similar objects as negative candidate labeling information corresponding to the target object, and acquiring sample data according to the target object, the positive candidate labeling information and the negative candidate labeling information, wherein the other objects are objects except the candidate attention object in the candidate similar objects.

Fig. 8 is a schematic diagram illustrating a second sample data obtaining process according to a second embodiment of the present application. That is, generally, in the candidate similar objects corresponding to the target object, the user only pays attention to the object most similar to the target object, that is, the candidate object of interest, and it can be considered a priori that the candidate object of interest has the highest similarity to the target object, that is, the same visual feature information contained in both of them is the most, and therefore, the candidate object of interest can be used as the positive candidate label information of the target object; meanwhile, the similarity between the object which is not concerned by the user and the target object in the candidate similar objects corresponding to the target object can be considered a priori, namely the similarity between the object and the target object is lower, namely the same visual characteristic information contained in the object and the target object is less, so that the object can be used as the negative candidate marking information of the target object; further, a triple sample data may be formed, which is in the form of [ target object, positive candidate label information, negative candidate label information ], that is, the second sample data in the second embodiment of the present application.

When second sample data is obtained from the candidate similar objects corresponding to the target object, in order to make the labeling information in the finally obtained sample data more accurate, a strong similarity calculation model can be used to filter out objects which are obviously not related to the target object in the candidate similar objects, and then the second sample data is obtained from the filtered candidate similar objects, namely a more simplified target feature extraction model is obtained based on the teacher-student network architecture.

In addition, the obtaining sample data according to the target object and the candidate attention object includes: acquiring positive sample sorting data corresponding to the target object, wherein the positive sample sorting data is obtained by sorting objects in the candidate similar objects in a descending order according to the similarity with the target object; and acquiring sample data according to the target object and the positive sample sorting data corresponding to the target object.

Fig. 9 is a schematic diagram illustrating a third sample data obtaining process according to a second embodiment of the present application. According to the above description, the acquisition process of the second sample data is relatively simple, however, in the second sample data, we are negative candidate labeling information that directly takes an object that is not focused by the user in the candidate similar objects as the target object. However, in the candidate similar objects, there is false negative candidate label information which is not focused by the user but has a high similarity with the target object, and the object can be actually used as the positive candidate label information of the target object.

Step S905, training by using the sample data to obtain a target feature extraction model, wherein the target feature extraction model is used for obtaining feature information of the object to be retrieved according to the object to be retrieved.

Training to obtain a target feature extraction model by using the sample data, wherein the training comprises the following steps: training to obtain a target feature extraction model by using at least one of the following sample data: the target object is a target object, the target object is a target object corresponding to the target object, and the target object is a target object corresponding to the target object.

That is, after three sample data used for training the target feature extraction model are acquired through the above steps, the target feature extraction model may be obtained by training using any one or more of the acquired sample data.

In summary, a method for obtaining a feature extraction model according to a second embodiment of the present application includes: acquiring a target object; acquiring a candidate similar object corresponding to the target object; determining candidate objects of interest to the user from the candidate similar objects; acquiring sample data according to the target object and the candidate attention object; and training by using the sample data to obtain a target feature extraction model, wherein the target feature extraction model is used for obtaining feature information of the object to be retrieved according to the object to be retrieved. Compared with the mode that a common sample data set or a target object is manually marked to obtain sample data for training the feature extraction model, the method provided by the embodiment of the application obtains the candidate similar object corresponding to the target image, then obtains the sample data according to the obtained candidate concerned object by determining the candidate concerned object concerned by the user in the candidate similar object, and trains the obtained target feature extraction model by using the obtained sample data, so that the obtaining speed and the accuracy of the category prediction model can be improved.

In the above description, a method for acquiring a category prediction model and a method for acquiring a feature extraction model are provided, and in correspondence with the above method for acquiring a category prediction model and the above method for acquiring a feature extraction model, the present application also provides an object retrieval method, please refer to fig. 10, which is a flowchart of an object retrieval method provided in a third embodiment of the present application, wherein some steps have been described in detail in the above first embodiment and the second embodiment, so that the description here is simple, and relevant points refer to a part of the method for acquiring a category prediction model provided in the first embodiment of the present application and a method for acquiring a feature extraction model provided in the second embodiment of the present application, and the processing procedures described below are only schematic.

Fig. 10 is a flowchart of an object retrieval method according to a third embodiment of the present application, and the following description is made with reference to fig. 10.

Step S1001, an object to be retrieved is obtained.

Step S1002, inputting the object to be retrieved into a target category prediction model, and obtaining category information to which the object to be retrieved belongs, where the target category prediction model is obtained by using the method for obtaining the category prediction model provided in the first embodiment.

Step S1003, inputting the object to be retrieved into a target feature extraction model corresponding to the category information, and acquiring feature information of the object to be retrieved, where the target feature extraction model is obtained by using the method for acquiring a feature extraction model provided in the second embodiment.

Step S1004, obtaining a similar object corresponding to the object to be retrieved according to the feature information.

In the third embodiment of the present application, after the step S1004 is performed to obtain the similar object corresponding to the object to be retrieved, in order to obtain the similar object which is more related to the object to be retrieved and more meets the user requirement, the method further includes: if the screening instruction information for the similar objects is acquired, then: displaying attribute information of the similar objects; acquiring target attribute information of the similar object determined by a user; displaying similar objects corresponding to the target attribute information; and if the screening instruction information aiming at the similar object is not acquired, directly displaying the similar object.

The screening instruction information for the similar objects refers to instruction information for screening the similar objects acquired by the computing device. The filtering instruction information may be instruction information that is controlled by a user, or may be instruction information that is preset in the computing device.

The attribute information of the belonging target refers to attribute information which is selected by a user from attribute information provided by the computing equipment and is concerned by the user.

That is, after the computing device acquires the similar object corresponding to the object to be retrieved, the computing device first determines whether the screening instruction information for the similar object can be acquired, and if such instruction information exists, the computing device first acquires the attribute information of the similar object; then, displaying the attribute information of the similar objects on a user interface for a user to select; then, the computing equipment acquires target attribute information of the similar object determined by the user; and then, the computing equipment acquires the similar objects corresponding to the target attribute information from the acquired similar objects and displays the similar objects corresponding to the target attribute information. In addition, if the computing equipment does not acquire the screening instruction information for the similar objects, the acquired similar objects are directly displayed on a user interface. In addition, it should be noted that, the above is only a specific implementation provided for the third embodiment of the present application, and during the specific implementation, similar objects that are more suitable for the needs of the user may also be provided to the user through other manners according to actual needs, and details are not described here.

In the above description, a method for obtaining a category prediction model is provided, and in correspondence with the above method for obtaining a category prediction model, the present application also provides a method for obtaining a commodity category prediction model, please refer to fig. 11, which is a flowchart of a method for obtaining a commodity category prediction model provided in a fourth embodiment of the present application, wherein some steps have been described in detail in the above first embodiment, so that the description herein is relatively simple, and for the relevant points, reference may be made to some descriptions in the method for obtaining a category prediction model provided in the first embodiment of the present application, and the processing procedures described below are only schematic.

Fig. 11 is a flowchart of a method for obtaining a commodity category prediction model according to a fourth embodiment of the present application, and the method is described below with reference to fig. 11.

In step S1101, an original search image including a commodity object is acquired.

Step S1102, acquiring a candidate similar image corresponding to the original retrieval image.

Step S1103 determines a candidate attention image to which the user pays attention from the candidate similar images.

Step S1104, obtaining candidate category information to which the commodity object in the original retrieval image belongs according to the category information to which the commodity object included in the candidate attention image belongs.

Step S1105, using the original retrieval image and the candidate category information to which the commodity object in the original retrieval image belongs as sample data, training to obtain a target commodity category prediction model, wherein the target commodity category prediction model is used for predicting the category information to which the commodity object in the image to be retrieved belongs according to the image to be retrieved.

In the above description, a method for obtaining a feature extraction model is provided, and in correspondence with the above method for obtaining a feature extraction model, the present application also provides a method for obtaining a commodity feature extraction model, please refer to fig. 12, which is a flowchart of a method for obtaining a commodity feature extraction model according to a fifth embodiment of the present application, wherein some steps have been described in detail in the above second embodiment, so that the description herein is relatively simple, and for the relevant points, reference may be made to some descriptions in the method for obtaining a feature extraction model according to the second embodiment of the present application, and the processing procedures described below are only schematic.

Fig. 12 is a flowchart of a method for acquiring a product feature extraction model according to a fifth embodiment of the present application, which is described below with reference to fig. 12.

Step S1201, an original search image including a commodity object is acquired.

Step S1202, a candidate similar image corresponding to the original retrieval image is acquired.

In step S1203, a candidate attention image to which the user pays attention is determined from the candidate similar images.

Step S1204, obtaining sample data according to the original retrieval image and the candidate attention image.

And step S1205, training by using the sample data to obtain a target commodity feature extraction model, wherein the target commodity feature extraction model is used for obtaining feature information of a commodity object in an image to be retrieved according to the image to be retrieved.

In the above description, an object retrieval method is provided, and in correspondence with the above object retrieval method, the present application also provides an image retrieval method, please refer to fig. 13, which is a flowchart of an image retrieval method provided in a sixth embodiment of the present application, wherein some steps have been described in detail in the above third embodiment, so that the description herein is relatively simple, and for the relevant points, reference may be made to some descriptions in an image retrieval method provided in the third embodiment of the present application, and the processing procedures described below are only schematic.

Step S1301, an image to be retrieved including the commodity object is acquired.

Step S1302, inputting the image to be retrieved into a target commodity category prediction model, and obtaining the category information to which the commodity object in the image to be retrieved belongs, wherein the target commodity category prediction model is obtained by using the method for obtaining the commodity category prediction model provided in the fourth embodiment.

Step S1303, inputting the image to be retrieved into a target commodity feature extraction model corresponding to the category information, and obtaining feature information of a commodity object in the image to be retrieved, where the target commodity feature extraction model is obtained by using the method for obtaining a commodity feature extraction model provided in the fifth embodiment.

Step S1304, obtaining a similar image corresponding to the image to be retrieved according to the feature information.

In the sixth embodiment of the present application, after the step S1304 is performed to obtain the similar image corresponding to the image to be retrieved, in order to obtain the similar image which is more related to the commodity object included in the image to be retrieved and more meets the user requirement, the method further includes: if the screening instruction information for the similar images is acquired, then: displaying the attribute information of the commodity object contained in the similar image; acquiring target attribute information of the commodity object contained in the similar image determined by the user; displaying similar images corresponding to the target attribute information; and if the screening instruction information aiming at the similar images is not acquired, directly displaying the similar images. The attribute information includes stock quantity units of the product images included in the similar image.

The screening instruction information for the commodity objects contained in the similar images refers to instruction information for screening the commodity objects contained in the similar images acquired by the computing device. The filtering instruction information may be instruction information that is controlled by a user, or may be instruction information that is preset in the computing device.

The Stock Keeping Unit (SKU) is a Unit for measuring Stock entrance and exit of the commodity object. For example, a SKU for a textile may be generally expressed as: specification, color and style.

That is, after the computing device acquires the similar image corresponding to the image to be retrieved, the computing device first determines whether the screening instruction information for the commodity object included in the similar image can be acquired, and if such instruction information exists, the computing device first acquires attribute information of the commodity object included in the similar image, such as specification, color and style of the commodity object, or model and storage space size of the commodity object; then, displaying the attribute information of the commodity object contained in the similar image on a user interface for a user to select; then, the computing equipment acquires target attribute information of the commodity object contained in the similar image determined by the user; and then, the computing equipment acquires the similar image corresponding to the target attribute information from the acquired similar image and displays the similar image corresponding to the target attribute information. In addition, if the computing device does not acquire the screening instruction information for the commodity objects contained in the similar images, the acquired similar images are directly displayed on a user interface. Of course, the above is only a specific implementation provided for the sixth embodiment of the present application, and during the specific implementation, similar images more conforming to the requirements of the user may also be provided to the user through other manners according to actual needs, and details are not described here.

In the above description, a method for obtaining a category prediction model is provided, and in correspondence with the above method for obtaining a category prediction model, the present application also provides a device for obtaining a category prediction model, please refer to fig. 14, which is a schematic diagram of a device for obtaining a category prediction model provided in a seventh embodiment of the present application, and since the device embodiment is substantially similar to the method embodiment, the description is simple, and relevant points can be referred to partial description of the method embodiment, and the device embodiment described below is only schematic. A seventh embodiment of the present application provides an apparatus for obtaining a category prediction model, including:

an object acquisition unit 1401 for acquiring a target object.

A similar object obtaining unit 1402, configured to obtain a candidate similar object corresponding to the target object.

An attention object acquisition unit 1403, from which the user determines a candidate attention object that the user pays attention to.

A category information obtaining unit 1404, configured to obtain candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs.

A model training unit 1405, configured to train to obtain a target category prediction model by using the target object and candidate category information to which the target object belongs as sample data, where the target category prediction model is configured to predict, according to an object to be retrieved, category information to which the object to be retrieved belongs.

Corresponding to the method for obtaining a category prediction model provided in the first embodiment, please refer to fig. 15, which is a schematic diagram of an electronic device provided in the eighth embodiment of the present application, and since the embodiment of the electronic device is substantially similar to the embodiment of the method, the description is relatively simple, and related points can be referred to part of the description of the embodiment of the method, and the embodiment of the electronic device described below is only schematic. An eighth embodiment of the present application provides an electronic device including:

a processor 1501;

a memory 1502 for storing a program of an acquisition method of a category prediction model, which executes the following steps after the apparatus is powered on and the program of the acquisition method of the category prediction model is executed by the processor:

acquiring a target object;

acquiring a candidate similar object corresponding to the target object;

determining candidate objects of interest to the user from the candidate similar objects;

obtaining candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs;

and training to obtain a target category prediction model by taking the target object and the candidate category information to which the target object belongs as sample data, wherein the target category prediction model is used for predicting the category information to which the object to be retrieved belongs according to the object to be retrieved.

In correspondence with the method for acquiring a category prediction model provided by the first embodiment, the present application also provides a storage device, since the storage device embodiment is substantially similar to the method embodiment, the description is relatively simple, and for relevant points, reference may be made to part of the description of the method embodiment, and the storage device embodiment described below is only illustrative. A storage device according to a ninth embodiment of the present application stores a program of a method for acquiring a category prediction model, where the program is executed by a processor to perform the following steps:

acquiring a target object;

acquiring a candidate similar object corresponding to the target object;

In the above description, a method for obtaining a feature extraction model is provided, and in correspondence with the above method for obtaining a feature extraction model, the present application also provides a device for obtaining a feature extraction model, please refer to fig. 16, which is a schematic diagram of a device for obtaining a feature extraction model provided in a tenth embodiment of the present application, and since the device embodiment is substantially similar to the method embodiment, the description is simple, and in relation to the above, reference may be made to part of the description of the method embodiment, and the device embodiment described below is only illustrative. An apparatus for obtaining a feature extraction model according to a tenth embodiment of the present application includes:

an object obtaining unit 1601 is configured to obtain a target object.

A similar object obtaining unit 1602, configured to obtain a candidate similar object corresponding to the target object.

An attention object obtaining unit 1603, configured to determine a candidate attention object focused by the user from the candidate similar objects.

A sample data obtaining unit 1604, configured to obtain sample data according to the target object and the candidate attention object.

A model training unit 1605, configured to obtain a target feature extraction model by using the sample data training, where the target feature extraction model is used to obtain feature information of an object to be retrieved according to the object to be retrieved.

Corresponding to the method for acquiring a feature extraction model provided in the second embodiment, the present application further provides an electronic device, which has a structure substantially similar to that of the electronic device described in the eighth embodiment of the present application, and since the electronic device is substantially similar to the method embodiment, the description is relatively simple, and related points can be referred to part of the description of the method embodiment, and the electronic device embodiment described below is only schematic. An electronic device provided in an eleventh embodiment of the present application includes:

a processor;

acquiring a target object;

acquiring a candidate similar object corresponding to the target object;

acquiring sample data according to the target object and the candidate attention object;

and training by using the sample data to obtain a target feature extraction model, wherein the target feature extraction model is used for obtaining feature information of the object to be retrieved according to the object to be retrieved.

Corresponding to the method for acquiring the feature extraction model provided by the second embodiment, the present application also provides a storage device, and since the embodiment of the storage device is substantially similar to the embodiment of the method, the description is relatively simple, and for relevant points, reference may be made to part of the description of the embodiment of the method, and the embodiment of the storage device described below is only illustrative. A storage device according to a twelfth embodiment of the present application stores a program of a feature extraction model acquisition method, the program being executed by a processor and executing steps of:

acquiring a target object;

acquiring a candidate similar object corresponding to the target object;

In the above description, an object retrieval method is provided, and in correspondence with the above object retrieval method, the present application also provides an object retrieval apparatus, please refer to fig. 17, which is a schematic diagram of an object retrieval apparatus provided in the thirteenth embodiment of the present application. A thirteenth embodiment of the present application provides an object retrieval apparatus including:

an object acquisition unit 1701 is used to acquire an object to be retrieved.

A category information obtaining unit 1702, configured to input the object to be retrieved into a target category prediction model, and obtain category information to which the object to be retrieved belongs, where the target category prediction model is obtained by using the method for obtaining a category prediction model provided in the first embodiment of the present application.

A feature information obtaining unit 1703, configured to input the object to be retrieved into a target feature extraction model corresponding to the category information, and obtain feature information of the object to be retrieved, where the target feature extraction model is obtained by using the method for obtaining a feature extraction model provided in the second embodiment of the present application.

A similar object obtaining unit 1704, configured to obtain a similar object corresponding to the object to be retrieved according to the feature information.

In correspondence with the object retrieval method provided by the third embodiment, the present application further provides an electronic device, the structure of which is substantially similar to that of the electronic device described in the eighth embodiment of the present application, and since the electronic device is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to part of the description of the method embodiment, and the electronic device embodiment described below is only illustrative. A fourteenth embodiment of the present application provides an electronic device including:

a processor;

acquiring an object to be retrieved;

inputting the object to be retrieved into a target category prediction model, and acquiring category information to which the object to be retrieved belongs, wherein the target category prediction model is obtained by using the method for obtaining the category prediction model provided by the first embodiment of the application;

inputting the object to be retrieved into a target feature extraction model corresponding to the category information, and acquiring feature information of the object to be retrieved, wherein the target feature extraction model is obtained by using the method for acquiring the feature extraction model provided by the second embodiment of the application;

and acquiring a similar object corresponding to the object to be retrieved according to the characteristic information.

Corresponding to the object retrieval method provided by the third embodiment, the present application also provides a storage device, since the storage device embodiment is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to part of the description of the method embodiment, and the storage device embodiment described below is only illustrative. A fifteenth embodiment of the present application provides a storage device, which stores a program of an object retrieval method, the program being executed by a processor to perform the steps of:

acquiring an object to be retrieved;

inputting the object to be retrieved into a target feature extraction model corresponding to the category information, and acquiring feature information of the object to be retrieved, wherein the target feature extraction model is obtained by using the method for acquiring the feature extraction model provided by the first embodiment of the application;

In the above description, a method for obtaining a product category prediction model is provided, and in correspondence with the method for obtaining the product category prediction model, the present application also provides a device for obtaining a product category prediction model, please refer to fig. 18, which is a schematic diagram of a device for obtaining a product category prediction model provided in the sixteenth embodiment of the present application. A sixteenth embodiment of the present application provides an apparatus for obtaining a commodity category prediction model, including:

an original search image acquiring unit 1801 is configured to acquire an original search image including a commodity object.

A similar image obtaining unit 1802, configured to obtain a candidate similar image corresponding to the original retrieval image.

An attention image acquiring unit 1803, configured to determine a candidate attention image focused by the user from the candidate similar images.

A candidate category information obtaining unit 1804, configured to obtain candidate category information to which a commodity object in the original retrieval image belongs according to category information to which the commodity object included in the candidate attention image belongs.

A model obtaining unit 1805, configured to train to obtain a target commodity category prediction model by using the original retrieval image and candidate category information to which the commodity object in the original retrieval image belongs as sample data, where the target commodity category prediction model is used to predict, according to the image to be retrieved, category information to which the commodity object in the image to be retrieved belongs.

Corresponding to the method for acquiring a commodity category prediction model provided by the fourth embodiment, the present application further provides an electronic device, the structure of which is substantially similar to that of the electronic device described in the eighth embodiment of the present application, and since the electronic device embodiment is substantially similar to the method embodiment, the description is relatively simple, and related points can be referred to partial description of the method embodiment, and the electronic device embodiment described below is only schematic. A seventeenth embodiment of the present application provides an electronic apparatus comprising:

a processor;

acquiring an original retrieval image containing a commodity object;

acquiring a candidate similar image corresponding to the original retrieval image;

determining a candidate attention image concerned by the user from the candidate similar images;

obtaining candidate category information to which the commodity object belongs in the original retrieval image according to category information to which the commodity object contained in the candidate attention image belongs;

and training to obtain a target commodity category prediction model by taking the original retrieval image and candidate category information to which the commodity object in the original retrieval image belongs as sample data, wherein the target commodity category prediction model is used for predicting the category information to which the commodity object in the image to be retrieved belongs according to the image to be retrieved.

Corresponding to the method for acquiring the commodity category prediction model provided by the fourth embodiment, the present application also provides a storage device, since the storage device embodiment is substantially similar to the method embodiment, the description is relatively simple, and the relevant points can be referred to the partial description of the method embodiment, and the storage device embodiment described below is only illustrative. A storage device according to an eighteenth embodiment of the present application stores a program of a method for acquiring a prediction model of an item of merchandise, where the program is executed by a processor to perform the steps of:

acquiring an original retrieval image containing a commodity object;

In the above description, a method for acquiring a product feature extraction model is provided, and in correspondence with the method for acquiring a product feature extraction model, the present application also provides an apparatus for acquiring a product feature extraction model, please refer to fig. 19, which is a schematic diagram of an apparatus for acquiring a product feature extraction model provided in a nineteenth embodiment of the present application. A nineteenth embodiment of the present application provides an apparatus for obtaining a product feature extraction model, including:

an original search image acquisition unit 1901 is configured to acquire an original search image including a commodity object.

A similar image obtaining unit 1902, configured to obtain a candidate similar image corresponding to the original retrieval image.

A focused image obtaining unit 1903, configured to determine a candidate focused image focused by the user from the candidate similar images.

A sample data obtaining unit 1904, configured to obtain sample data according to the original retrieval image and the candidate attention image.

A model obtaining unit 1905, configured to obtain a target commodity feature extraction model by using the sample data training, where the target commodity feature extraction model is used to obtain feature information of a commodity object in an image to be retrieved according to the image to be retrieved.

Corresponding to the method for acquiring the product feature extraction model provided in the fifth embodiment, the present application further provides an electronic device, which has a structure substantially similar to that of the electronic device described in the eighth embodiment of the present application, and since the electronic device embodiment is substantially similar to the method embodiment, the description is relatively simple, and related points can be referred to only part of the description of the method embodiment, and the electronic device embodiment described below is only schematic. A twentieth embodiment of the present application provides an electronic device including:

a processor;

acquiring an original retrieval image containing a commodity object;

acquiring sample data according to the original retrieval image and the candidate attention image;

and training by using the sample data to obtain a target commodity feature extraction model, wherein the target commodity feature extraction model is used for acquiring feature information of a commodity object in an image to be retrieved according to the image to be retrieved.

Corresponding to the method for acquiring the commodity feature extraction model provided by the fifth embodiment, the present application also provides a storage device, since the storage device embodiment is substantially similar to the method embodiment, the description is relatively simple, and the relevant points can be referred to the partial description of the method embodiment, and the storage device embodiment described below is only illustrative. A twenty-first embodiment of the present application provides a storage device, in which a program of an acquisition method of a product feature extraction model is stored, where the program is executed by a processor to perform the following steps:

acquiring an original retrieval image containing a commodity object;

In the above description, an image retrieval method is provided, and the present application also provides an image retrieval apparatus corresponding to the above object retrieval method, please refer to fig. 20, which is a schematic diagram of an image retrieval apparatus provided in the twenty-second embodiment of the present application, and since the apparatus embodiment is substantially similar to the method embodiment, the description is relatively simple, and for relevant points, reference may be made to partial description of the method embodiment, and the apparatus embodiment described below is only schematic. A twenty-second embodiment of the present application provides an image retrieval apparatus including:

a to-be-retrieved image acquiring unit 2001 for acquiring an image to be retrieved including the commodity object.

A category information obtaining unit 2002, configured to input the image to be retrieved into a target product category prediction model, and obtain category information to which a product object in the image to be retrieved belongs, where the target product category prediction model is obtained by using the method for obtaining a product category prediction model provided in the fourth embodiment.

A feature information obtaining unit 2003, configured to input the image to be retrieved into a target commodity feature extraction model corresponding to the category information, and obtain feature information of a commodity object in the image to be retrieved, where the target commodity feature extraction model is obtained by using the method for obtaining a commodity feature extraction model according to the fifth embodiment.

A similar image obtaining unit 2004, configured to obtain a similar image corresponding to the image to be retrieved according to the feature information.

In correspondence with the image retrieval method provided by the sixth embodiment, the present application further provides an electronic device, which has a structure substantially similar to that of the electronic device described in the eighth embodiment of the present application, and since the electronic device is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to part of the description of the method embodiment, and the electronic device embodiment described below is only illustrative. A twenty-third embodiment of the present application provides an electronic device including:

a processor;

acquiring an image to be retrieved containing a commodity object;

inputting the image to be retrieved into a target commodity category prediction model, and acquiring category information to which a commodity object in the image to be retrieved belongs, wherein the target commodity category prediction model is obtained by using the method for obtaining the commodity category prediction model according to claim 23;

inputting the image to be retrieved into a target commodity feature extraction model corresponding to the category information, and acquiring feature information of a commodity object in the image to be retrieved, wherein the target commodity feature extraction model is obtained by using the method for acquiring the commodity feature extraction model according to claim 24;

and acquiring a similar image corresponding to the image to be retrieved according to the characteristic information.

In correspondence with the image retrieval method provided by the sixth embodiment, the present application also provides a storage device, since the storage device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant points can be referred to part of the description of the method embodiment, and the storage device embodiment described below is only illustrative. A twenty-fourth embodiment of the present application provides a storage device storing a program of an image retrieval method, the program being executed by a processor to perform the steps of:

acquiring an image to be retrieved containing a commodity object;

inputting the image to be retrieved into a target commodity category prediction model, and acquiring category information to which a commodity object in the image to be retrieved belongs, wherein the target commodity category prediction model is obtained by using the method for obtaining the commodity category prediction model provided by the fourth embodiment;

inputting the image to be retrieved into a target commodity feature extraction model corresponding to the category information, and acquiring feature information of a commodity object in the image to be retrieved, wherein the target commodity feature extraction model is obtained by using the method for acquiring the commodity feature extraction model provided by the fifth embodiment;

Although the present application has been described with reference to the preferred embodiments, it is not intended to limit the present application, and those skilled in the art can make variations and modifications without departing from the spirit and scope of the present application, therefore, the scope of the present application should be determined by the claims that follow.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Claims

1. A method for obtaining a category prediction model is characterized by comprising the following steps:

acquiring a target object;

acquiring a candidate similar object corresponding to the target object;

2. The method according to claim 1, wherein the obtaining of the candidate similar object corresponding to the target object includes:

acquiring a first candidate similar object corresponding to the target object;

acquiring behavior data of a user for the first candidate similar object;

and if behavior data of the user for the first candidate similar object is acquired, taking the first candidate similar object as the candidate similar object.

3. The method for obtaining the category prediction model according to claim 2, comprising:

if the behavior data of the user for the first candidate similar object is not acquired, acquiring category information to be selected determined by the user, acquiring a second candidate similar object corresponding to the target object according to the category information to be selected, and taking the second candidate similar object as the candidate similar object.

4. The method according to claim 1, wherein the determining a candidate object of interest from the candidate similar objects includes:

acquiring behavior data of a user aiming at the candidate similar objects;

and determining candidate attention objects concerned by the user according to the behavior data.

5. The method according to claim 4, wherein the obtaining candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs includes:

and acquiring primary category information to which the candidate attention object belongs, and taking the primary category information to which the candidate attention object belongs as simple candidate category information to which the target object belongs.

6. The method according to claim 3, wherein the obtaining candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs includes:

acquiring the primary category information to which the first candidate similar object belongs, taking the primary category information to which the first candidate similar object belongs as the negative candidate category information to which the target object belongs, and taking the primary category information to which the candidate attention object concerned by the user determined in the candidate similar objects belongs as the difficult candidate category information to which the target object belongs.

7. The method according to claim 1, wherein the obtaining candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs includes:

acquiring leaf category information to which the candidate attention object belongs, wherein the leaf category information is category information at the last level of a category tree corresponding to the candidate attention object;

and according to the leaf category information to which the candidate attention object belongs, acquiring virtual category information to which the target object belongs, and taking the virtual category information to which the target object belongs as the candidate category information to which the target object belongs, wherein the quantity of the virtual category information is more than that of the primary category information and is not more than that of the leaf category information.

8. The method according to claim 7, wherein the obtaining, according to the leaf category information to which the candidate attention object belongs, the virtual category information to which the target object belongs includes:

taking the leaf category information to which the candidate attention object belongs as a node, and acquiring a leaf category information matrix;

performing graph embedding processing on the leaf category information matrix to obtain a leaf category vector corresponding to the leaf category information matrix;

and acquiring the virtual category information of the target object according to the leaf category vector.

9. The method for obtaining the category prediction model according to claim 8, wherein the obtaining, according to the leaf category vector, the virtual category information to which the target object belongs includes:

and carrying out cluster analysis on the leaf category vectors to obtain the virtual category information of the target object.

10. The method according to claim 5, wherein the training of obtaining the target category prediction model by using the target object and candidate category information to which the target object belongs as sample data includes:

and training to obtain the target category prediction model by taking the target object and the simple candidate category information to which the target object belongs as sample data.

11. The method according to claim 6, wherein the training for obtaining the target category prediction model by using the target object and candidate category information to which the target object belongs as sample data includes:

and training to obtain the target category prediction model by taking the target object, the negative candidate category information to which the target object belongs and the difficult candidate category information to which the target object belongs as sample data.

12. The method according to claim 7, wherein the training of obtaining the target category prediction model by using the target object and the candidate category information to which the target object belongs as sample data includes:

and training to obtain the target category prediction model by taking the target object and the virtual category information to which the target object belongs as sample data.

13. The method according to claim 1, wherein the training of obtaining the target category prediction model using the target object and candidate category information to which the target object belongs as sample data includes:

training to obtain the target category prediction model by using at least two sample data: the target object and the simple candidate category information to which the target object belongs, the negative candidate category information to which the target object and the target object belong, and the hard candidate category information to which the target object belongs, and the virtual category information to which the target object and the target object belong.

14. A method for obtaining a feature extraction model is characterized by comprising the following steps:

acquiring a target object;

acquiring a candidate similar object corresponding to the target object;

15. The method for acquiring the feature extraction model of claim 14, wherein the determining the candidate object of interest of the user from the candidate similar objects comprises:

acquiring behavior data of a user aiming at the candidate similar objects;

16. The method of claim 14, wherein the obtaining sample data according to the target object and the candidate object of interest comprises:

acquiring virtual characteristic information corresponding to the target object according to the candidate attention object;

and acquiring sample data according to the target object and the virtual characteristic information corresponding to the target object.

17. The method of claim 16, wherein the obtaining virtual feature information corresponding to the target object according to the candidate object of interest includes:

taking the candidate attention object as a node to obtain a candidate attention object matrix;

performing graph embedding processing on the candidate attention object matrix to obtain a virtual feature vector corresponding to the candidate attention object;

and acquiring virtual characteristic information corresponding to the target object according to the virtual characteristic vector of the candidate attention object.

18. The method for acquiring the feature extraction model according to claim 17, wherein the acquiring virtual feature information corresponding to the target object according to the virtual feature vector of the candidate attention object includes:

and performing cluster analysis on the virtual feature vectors of the candidate attention objects to acquire virtual feature information corresponding to the target object.

19. The method of claim 14, wherein the obtaining sample data according to the target object and the candidate object of interest comprises:

and taking the candidate attention object as positive candidate labeling information corresponding to the target object, taking other objects in the candidate similar objects as negative candidate labeling information corresponding to the target object, and acquiring sample data according to the target object, the positive candidate labeling information and the negative candidate labeling information, wherein the other objects are objects except the candidate attention object in the candidate similar objects.

20. The method of claim 14, wherein the obtaining sample data according to the target object and the candidate object of interest comprises:

acquiring positive sample sorting data corresponding to the target object, wherein the positive sample sorting data is obtained by sorting objects in the candidate similar objects in a descending order according to the similarity with the target object;

and acquiring sample data according to the target object and the positive sample sorting data corresponding to the target object.

21. The method of claim 14, wherein training the target feature extraction model using the sample data comprises:

training to obtain a target feature extraction model by using at least one of the following sample data: the target object is a target object, the target object is a target object corresponding to the target object, and the target object is a target object corresponding to the target object.

22. An object retrieval method, comprising:

acquiring an object to be retrieved;

inputting the object to be retrieved into a target category prediction model to obtain category information to which the object to be retrieved belongs, wherein the target category prediction model is obtained by using the method for obtaining the category prediction model according to any one of claims 1 to 13;

inputting the object to be retrieved into a target feature extraction model corresponding to the category information, and acquiring feature information of the object to be retrieved, wherein the target feature extraction model is obtained by using the method for acquiring the feature extraction model according to any one of claims 14 to 21;

23. The object retrieval method of claim 22, further comprising:

if the screening instruction information for the similar objects is acquired, then: displaying attribute information of the similar objects; acquiring target attribute information of the similar object determined by a user; displaying similar objects corresponding to the target attribute information;

and if the screening instruction information aiming at the similar object is not acquired, directly displaying the similar object.

24. A method for obtaining a commodity category prediction model is characterized by comprising the following steps:

acquiring an original retrieval image containing a commodity object;

25. A method for acquiring a commodity feature extraction model is characterized by comprising the following steps:

acquiring an original retrieval image containing a commodity object;

26. An image retrieval method, comprising:

acquiring an image to be retrieved containing a commodity object;

inputting the image to be retrieved into a target commodity category prediction model, and acquiring category information to which a commodity object in the image to be retrieved belongs, wherein the target commodity category prediction model is obtained by using the method for obtaining the commodity category prediction model according to claim 24;

inputting the image to be retrieved into a target commodity feature extraction model corresponding to the category information, and acquiring feature information of a commodity object in the image to be retrieved, wherein the target commodity feature extraction model is obtained by using the method for acquiring the commodity feature extraction model as claimed in claim 25;

27. The image retrieval method according to claim 26, further comprising:

if the screening instruction information for the commodity objects contained in the similar images is acquired, then: displaying the attribute information of the commodity object contained in the similar image; acquiring target attribute information of the commodity object contained in the similar image determined by the user; displaying similar images corresponding to the target attribute information;

and if the screening instruction information aiming at the commodity objects contained in the similar images is not acquired, directly displaying the similar images.

28. The image retrieval method according to claim 27, wherein the attribute information includes stock quantity units of commodity images included in the similar image.

29. An apparatus for acquiring a category prediction model, comprising:

an object acquisition unit configured to acquire a target object;

a similar object obtaining unit configured to obtain a candidate similar object corresponding to the target object;

an attention object acquisition unit, wherein the user determines a candidate attention object concerned by the user from the candidate similar objects;

the category information acquisition unit is used for acquiring candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs;

and the model training unit is used for training to obtain a target category prediction model by taking the target object and the candidate category information to which the target object belongs as sample data, wherein the target category prediction model is used for predicting the category information to which the object to be retrieved belongs according to the object to be retrieved.

30. An electronic device, comprising:

a processor;

acquiring a target object;

acquiring a candidate similar object corresponding to the target object;

31. A storage device storing a program of an acquisition method of a category prediction model, the program being executed by a processor and executing steps of:

acquiring a target object;

acquiring a candidate similar object corresponding to the target object;

32. An apparatus for obtaining a feature extraction model, comprising:

an object acquisition unit configured to acquire a target object;

an attention object acquisition unit for determining a candidate attention object focused by a user from the candidate similar objects;

the sample data acquisition unit is used for acquiring sample data according to the target object and the candidate attention object;

and the model training unit is used for training by using the sample data to obtain a target feature extraction model, wherein the target feature extraction model is used for acquiring feature information of the object to be retrieved according to the object to be retrieved.

33. An electronic device, comprising:

a processor;

acquiring a target object;

acquiring a candidate similar object corresponding to the target object;

34. A storage device characterized by storing a program of an acquisition method of a feature extraction model, the program being executed by a processor and executing the steps of:

acquiring a target object;

acquiring a candidate similar object corresponding to the target object;

35. An object retrieval apparatus, comprising:

the object acquisition unit is used for acquiring an object to be retrieved;

a category information obtaining unit, configured to input the object to be retrieved into a target category prediction model, and obtain category information to which the object to be retrieved belongs, where the target category prediction model is obtained by using the method for obtaining the category prediction model according to any one of claims 1 to 13;

a feature information obtaining unit, configured to input the object to be retrieved into a target feature extraction model corresponding to the category information, and obtain feature information of the object to be retrieved, where the target feature extraction model is a model obtained by using the feature extraction model obtaining method according to any one of claims 14 to 21;

and the similar object acquisition unit is used for acquiring a similar object corresponding to the object to be retrieved according to the characteristic information.

36. An electronic device, comprising:

a processor;

acquiring an object to be retrieved;

37. A storage device storing a program of an image retrieval method, the program being executed by a processor to execute the steps of:

acquiring an object to be retrieved;

38. An apparatus for obtaining a prediction model of a commodity category, comprising:

an original retrieval image acquisition unit for acquiring an original retrieval image containing a commodity object;

a similar image acquisition unit for acquiring a candidate similar image corresponding to the original retrieval image;

a focused image acquisition unit for determining a candidate focused image focused by the user from the candidate similar images;

a candidate category information obtaining unit, configured to obtain candidate category information to which a commodity object in the original retrieval image belongs according to category information to which the commodity object included in the candidate attention image belongs;

and the model obtaining unit is used for training to obtain a target commodity category prediction model by taking the original retrieval image and candidate category information to which the commodity object in the original retrieval image belongs as sample data, wherein the target commodity category prediction model is used for predicting the category information to which the commodity object in the image to be retrieved belongs according to the image to be retrieved.

39. An electronic device, comprising:

a processor;

acquiring an original retrieval image containing a commodity object;

40. A storage device storing a program for a method of acquiring a prediction model of an article category, the program being executed by a processor and executing the steps of:

acquiring an original retrieval image containing a commodity object;

41. An acquisition device for a commodity feature extraction model, comprising:

the sample data acquisition unit is used for acquiring sample data according to the original retrieval image and the candidate attention image;

and the model obtaining unit is used for training by using the sample data to obtain a target commodity feature extraction model, wherein the target commodity feature extraction model is used for obtaining feature information of a commodity object in the image to be retrieved according to the image to be retrieved.

42. An electronic device, comprising:

a processor;

acquiring an original retrieval image containing a commodity object;

43. A storage device storing a program of an acquisition method of a product feature extraction model, the program being executed by a processor and executing steps of:

acquiring an original retrieval image containing a commodity object;

44. An image retrieval apparatus, comprising:

the retrieval-waiting image acquisition unit is used for acquiring a retrieval-waiting image containing a commodity object;

a category information obtaining unit, configured to input the image to be retrieved into a target commodity category prediction model, and obtain category information to which a commodity object in the image to be retrieved belongs, where the target commodity category prediction model is obtained by using the method for obtaining the commodity category prediction model according to claim 24;

a feature information obtaining unit, configured to input the image to be retrieved into a target commodity feature extraction model corresponding to the category information, and obtain feature information of a commodity object in the image to be retrieved, where the target commodity feature extraction model is a model obtained by using the method for obtaining a commodity feature extraction model according to claim 25;

and the similar image acquisition unit is used for acquiring a similar image corresponding to the image to be retrieved according to the characteristic information.

45. An electronic device, comprising:

a processor;

acquiring an image to be retrieved containing a commodity object;

46. A storage device storing a program of an image retrieval method, the program being executed by a processor to execute the steps of:

acquiring an image to be retrieved containing a commodity object;