CN115905590A - An image recommendation method, system and terminal device based on deep hash retrieval - Google Patents

Abstract

The invention discloses an image recommendation method, system and terminal equipment based on deep hash retrieval, wherein the method comprises the following steps: crawling image data from the Internet through a crawler program, and preprocessing the image data to obtain preprocessed image data; inputting the preprocessed image data into a trained deep hash model to obtain a hash value, and determining an image candidate set based on the hash value; obtaining historical interactive data of a user, inputting the historical interactive data of the user and the image candidate set into a preset sequencing model to obtain an image recommendation list, and feeding back the image recommendation list to the user. According to the image recommendation method and device, the image candidate set is determined through the depth hash algorithm, the image recommendation list is determined based on the sorting model, the image retrieval speed can be greatly improved, and therefore the recommendation process of the image is accelerated.

Description

An image recommendation method, system and terminal device based on deep hash retrieval

technical field

The present invention relates to the technical field of image recommendation, in particular to an image recommendation method, system and terminal device based on deep hash retrieval.

Background technique

With the diversified development of data on social networking, multimedia information, e-commerce and other network platforms, especially image data, the number of digital images is growing at an alarming rate. Faced with this development trend, people's potential demand for information has gradually shifted from text to images. However, it is difficult for users to quickly find the images they are interested in from the huge and complicated image database. Therefore, image recommendation based on user interests has gradually become a popular research direction. At the same time, the explosive growth of image data on the Internet also brings new opportunities and challenges to the development of recommendation systems.

For a user-centered network platform, by capturing user preferences and making personalized recommendations, the recommendation system can quickly recommend content of interest to users, which will improve the efficiency of users' information acquisition and the economic benefits of the network platform. is of great significance. However, for the field of image recommendation, it becomes increasingly difficult to mine the user's interest in images from the sparse user image interaction data. The image itself has dense semantic information, and using the rich visual information of the image to mine the correlation between users and images has become a breakthrough in image recommendation. Therefore, the key challenge of image recommendation is how to mine the high-level semantic information of images, and how to model the relevant representations of images and users, so as to solve the contradiction between user needs and overloaded image data.

Image recommendation is a subdivision of the recommendation system. The early image recommendation basically regarded the image as an item, and used the description attributes of the image and user interaction records to make recommendations. The algorithm used the traditional Content-Based algorithm, ItemCF Collaborative filtering algorithm or matrix factorization algorithm, etc. With the continuous development of image feature extraction technology, image recommendation algorithms have begun to incorporate manually extracted image features, which is most effective for content-based recommendation systems. At the same time, image retrieval technology has also brought some new inspirations to image recommendation. In the traditional image retrieval process, first manually mark the text of the image, and then use keywords to retrieve the image. This kind of character matching based on image description The method of providing retrieval results is time-consuming and subjectively ambiguous, and the recommendation effect is not ideal. In the application scenario of large-scale images, given a query image, quickly find a certain number of images with similar content from a million-level image database through image features, this content-based image retrieval (content- based image retrieval) is a very popular research direction at present, but its retrieval speed is still limited by the image feature vector. It can be seen that the existing image recommendation methods are time-consuming and have low recommendation efficiency.

Therefore, the prior art still needs to be improved and improved.

Contents of the invention

The technical problem to be solved by the present invention is to provide an image recommendation method, system and terminal device based on deep hash retrieval in view of the above-mentioned defects of the prior art, aiming to solve the time-consuming and time-consuming image recommendation method in the prior art. The problem of low efficiency.

In order to solve the problems of the technologies described above, the technical scheme adopted in the present invention is as follows:

In the first aspect, the present invention provides an image recommendation method based on deep hash retrieval, wherein the method includes:

Crawling image data from the Internet through a crawler program, and preprocessing the image data to obtain preprocessed image data;

Input the preprocessed image data into the trained depth hash model to obtain a hash value, and determine an image candidate set based on the hash value;

Obtain user history interaction data, input the user history interaction data and the image candidate set into a preset ranking model, obtain an image recommendation list, and feed back the image recommendation list to the user.

In an implementation manner, the preprocessing the image data to obtain the preprocessed image data includes:

Perform format detection, picture cleaning and picture indexing on the image data.

In an implementation manner, the determining the image candidate set based on the hash value includes:

Obtain an index database, the index database includes image features corresponding to the image data;

The image candidate set is determined based on the hash value and the image features.

In an implementation manner, the determining the image candidate set based on the hash value and the image feature includes:

Based on the hash value, calculate the similarity between the image features of the image data, and acquire the image features whose similarity is higher than a preset threshold to obtain candidate image features;

Based on the candidate image features, candidate image data is obtained, and the candidate image data is used as the image candidate set.

In an implementation manner, the acquiring historical user interaction data includes:

Acquire historical query items generated by users through browsing, clicking, and bookmarking pictures within a historical time period, and use the historical query items as the user historical interaction data.

In an implementation manner, the inputting the user historical interaction data and the image candidate set into a preset sorting model to obtain an image recommendation list includes:

determining feedback behavior data corresponding to the historical query item according to the user historical interaction data;

Based on the sorting model, sort each piece of image data in the image candidate set according to the feedback behavior data, and output the image recommendation list.

In an implementation manner, the sorting of each piece of image data in the image candidate set based on the sorting model and the feedback behavior data, and outputting the image recommendation list include:

Based on the ranking model, the maximum posterior probability obtained by Bayesian analysis for each image data in the image candidate set;

Sorting each piece of image data in the image candidate set based on the maximum posterior probability, and outputting the image recommendation list.

In the second aspect, the embodiment of the present invention also provides an image recommendation system based on deep hash retrieval, wherein the system includes:

The image data processing module is used to crawl image data from the Internet through a crawler program, and preprocess the image data to obtain preprocessed image data;

An image candidate set determination module, configured to input the preprocessed image data into the trained deep hash model to obtain a hash value, and determine an image candidate set based on the hash value;

The image recommendation module is used to obtain user history interaction data, input the user history interaction data and the image candidate set into a preset sorting model, obtain an image recommendation list, and feed back the image recommendation list to the user.

In the third aspect, the embodiment of the present invention also provides a terminal device, wherein the terminal device includes a memory, a processor, and an image recommendation program based on deep hash retrieval that is stored in the memory and can run on the processor, and the processor executes In the image recommendation program based on deep hash retrieval, the steps of the image recommendation method based on deep hash retrieval in any one of the above schemes are realized.

In the fourth aspect, the embodiment of the present invention also provides a computer-readable storage medium, the computer-readable storage medium stores an image recommendation program based on deep hash retrieval, and when the image recommendation program based on deep hash retrieval is executed by a processor , to realize the steps of the image recommendation method based on deep hash retrieval in any one of the above schemes.

Beneficial effects: Compared with the prior art, the present invention provides an image recommendation method based on deep hash retrieval. First, the present invention crawls image data from the Internet through a crawler program, and preprocesses the image data. Get the preprocessed image data. Then, input the preprocessed image data into the trained deep hash model to obtain a hash value, and determine an image candidate set based on the hash value. Finally, the user history interaction data is obtained, the user history interaction data and the image candidate set are input into a preset sorting model, an image recommendation list is obtained, and the image recommendation list is fed back to the user. The invention determines the image candidate set through the deep hash algorithm, and determines the image recommendation list based on the sorting model, which can greatly improve the image retrieval speed, thereby accelerating the image recommendation process.

Description of drawings

FIG. 1 is a flowchart of a specific implementation of an image recommendation method based on deep hash retrieval provided by an embodiment of the present invention.

FIG. 2 is a schematic diagram of an overall framework of an image recommendation system based on deep hash retrieval provided by an embodiment of the present invention.

FIG. 3 is a schematic diagram of a deep hash model in an image recommendation method based on deep hash retrieval provided by an embodiment of the present invention.

FIG. 4 is a schematic diagram of a ranking model in an image recommendation method based on deep hash retrieval provided by an embodiment of the present invention.

FIG. 5 is a functional block diagram of an image recommendation system based on deep hash retrieval provided by an embodiment of the present invention.

Fig. 6 is a functional block diagram of an internal structure of a terminal device provided by an embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and effect of the present invention more clear and definite, the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

The embodiment of the present invention provides an image recommendation method based on deep hash retrieval. During specific implementation, this embodiment first crawls image data from the Internet through a crawler program, and preprocesses the image data to obtain the preprocessed image data. image data. Then, in this embodiment, the preprocessed image data is input into the trained deep hash model to obtain a hash value, and an image candidate set is determined based on the hash value. Finally, this embodiment acquires user history interaction data, inputs the user history interaction data and the image candidate set into a preset sorting model, obtains an image recommendation list, and feeds back the image recommendation list to the user. In this embodiment, the image candidate set is determined by the deep hash algorithm, and the image recommendation list is determined based on the ranking model, which can greatly improve the image retrieval speed, thereby speeding up the image recommendation process.

exemplary method

The image recommendation method based on deep hash retrieval in this embodiment can be applied to terminal devices, and the terminal devices are intelligent product terminals such as computers, smart TVs, and mobile phones. In this embodiment, as shown in Figure 1, the image recommendation method based on deep hash retrieval includes the following steps:

Step S100 , crawling image data from the Internet through a crawler program, and performing preprocessing on the image data to obtain preprocessed image data.

Specifically, in this embodiment, a crawler program is first written to crawl image data. This embodiment takes decoration style recommendation items as an example. Therefore, a picture website system can be built first. The picture website system can collect several image data, and these image data correspond to different decoration styles. For this reason, in this embodiment, a crawler program can be used to crawl image data, and then necessary preprocessing is performed on these image data. In this embodiment, as shown in FIG. 2 , the preprocessing includes performing one or more of format detection, image cleaning, and image indexing on the image data. After the image data is preprocessed, the preprocessed image data can be obtained.

Step S200, input the preprocessed image data into the trained deep hash model to obtain a hash value, and determine an image candidate set based on the hash value.

After the preprocessed image data is obtained, in this embodiment, the preprocessed image data can be input into the trained deep hash model, and the image candidate set can be obtained through the processing of the deep hash model. In this embodiment, the deep hash model chooses the deep convolutional neural network because the topology of the network can better match the input image, and it can perform feature extraction and image classification at the same time. In addition, CNN has the characteristics of sparse connection and weight sharing. The training parameters of the network can be greatly reduced during the training process, so the overall structure of the convolutional neural network is more adaptable and simpler. The deep hash model can be shown in Figure 3, the hash algorithm is embedded into the last layer of the CNN model (as shown in Figure 2), and then the classification task is set to perform error backpropagation to optimize the model, which can simultaneously Learn image features and hash functions, so the trained deep hash model can automatically extract image features based on image data and calculate hash values. After the hash value is obtained, this embodiment can determine the image candidate set. The image candidate set is the candidate set when performing image recommendation.

Specifically, in this embodiment, firstly, an index database is acquired, and the index database includes image features corresponding to the image data. Then, in this embodiment, based on the hash value, the similarity between the image features of the image data is calculated, and image features whose similarity is higher than a preset threshold are obtained to obtain candidate image features. The image data corresponding to these candidate image features have a high similarity. Therefore, in this embodiment, candidate image data is obtained based on the features of the candidate images, and the candidate image data is used as the image candidate set. This embodiment uses a locality-sensitive hash algorithm. In order to avoid hash collisions, the traditional hash algorithm distributes data in different coding positions as much as possible. The idea of this algorithm is that data with high similarity has a higher probability of being mapped to the same position, or the mapped positions are close to each other. It can keep the generated encoding space similar to the image feature space, and the calculation speed of the similarity measure of the generated binary encoding is fast, which can achieve a good retrieval speed performance.

Step S300, acquiring user historical interaction data, inputting the user historical interaction data and the image candidate set into a preset sorting model to obtain an image recommendation list, and feeding back the image recommendation list to the user.

Specifically, after obtaining the image candidate set, this embodiment can obtain the user's historical interaction data. In a specific application, the user generates historical query items by browsing, clicking, and collecting pictures within the historical time period, and these historical query items are interaction data. , so the historical query item can be used as the user historical interaction data. Next, in this embodiment, the user historical interaction data and the image candidate set are input into a preset ranking model, which is an improved BPR ranking model (Bayesian personalized ranking model). Different from other methods based on user rating matrix, the BPR ranking model of this embodiment is mainly to obtain implicit feedback (such as clicking, favorites, etc.) There is feedback behavior for item (that is, historical query item),? The sign means no feedback, and the feedback behavior data of each user is generated through the feedback matrix. The + sign in Figure 4 indicates that the user prefers i between i and j, and - is the opposite, ? The sign indicates no feedback or the same feedback. Therefore, in this embodiment, the feedback behavior data corresponding to the historical query item can be determined according to the user historical interaction data. Since the feedback behavior data can analyze which image data the user has given feedback to in the user historical interaction data, this embodiment can perform Bayesian analysis on each image data in the image candidate set at this time based on the BPR sorting model. The maximum posterior probability obtained from the analysis is used to reflect the probability that the image data is given feedback by the user (such as click, bookmark, etc.). Next, in this embodiment, each piece of image data in the image candidate set is sorted based on the maximum posterior probability, from high to low when sorting, and then the image recommendation list is output. And feed back the image recommendation list to the user. The user's interactive feedback based on the image recommendation list will also be recorded, and corresponding query items will be generated, which can be input into the deep hash model to optimize the parameters of the deep hash model so that the deep hash The model can output more accurate hashes. In addition, this embodiment also uses hybrid collaborative filtering that fuses image side information to increase the parameter constraints of the BPR model, which greatly enhances the low-dimensional implicit representation of users and images, and alleviates the data sparsity problem that only uses the User-Item matrix to train the model .

In summary, the recommended images obtained by traditional image hash retrieval rely on manually extracted features, and it is difficult to capture the user's visual preferences. The goal of the image recommendation method in this embodiment is to select images that meet the user's visual preferences or potential needs from a large amount of image data, and provide a set of recommended images sorted according to the scores. In this embodiment, the traditional image hashing algorithm is used to calculate the image similarity, which can effectively retrieve similar images, but it is difficult to measure the distance between different images. This embodiment uses the deep hash algorithm to calculate the image similarity. This method effectively combines the advanced semantic features extracted by the deep neural network and the hash-like coding operation. It can not only use the semantic information of the image for compression coding, but also take into account the computational efficiency. and resource consumption. The amount of data in the recommendation system of this embodiment is often very large, so the recommendation system usually adopts some recall strategies to recall candidate data from massive data to reduce the amount of calculation in the sorting stage. Aiming at the recall strategy of the image recommendation system, the present invention utilizes the similarity matrix calculated by the image depth hash result and user interaction records to recall candidate image data sets. In addition, this embodiment also proposes an improved BPR ranking model, which uses hybrid collaborative filtering that fuses image side information to increase BPR model parameter constraints, strengthens the low-dimensional implicit representation of users and images, and eases the problem of only using User-Item matrix training. The data sparsity problem of the model.

exemplary system

Based on the foregoing embodiments, the present invention also provides an image recommendation system based on deep hash retrieval, as shown in FIG. 5 , the system includes: an image data processing module 10, an image candidate set determination module 20 and an image recommendation module 30 . Specifically, the image data processing module 10 of this embodiment is used to crawl image data from the Internet through a crawler program, and preprocess the image data to obtain preprocessed image data. The image candidate set determination module 20 is configured to input the preprocessed image data into the trained deep hash model to obtain a hash value, and determine an image candidate set based on the hash value. The image recommendation module 30 is configured to obtain user history interaction data, input the user history interaction data and the image candidate set into a preset ranking model, obtain an image recommendation list, and feed back the image recommendation list to the user.

In one implementation, the image data processing module 10 of this embodiment includes:

The preprocessing unit is used to perform format detection, picture cleaning and picture indexing on the image data.

In an implementation manner, the image candidate set determining module 20 of this embodiment includes:

a database acquiring unit, configured to acquire an index database, the index database including image features corresponding to the image data;

A candidate set determining unit, configured to determine the image candidate set based on the hash value and the image feature.

In an implementation manner, the candidate set determination unit in this embodiment includes:

The candidate image feature acquisition subunit is used to calculate the similarity between image features of the image data based on the hash value, and acquire image features whose similarity is higher than a preset threshold to obtain candidate image features ;

An image candidate set determining unit, configured to obtain candidate image data based on the candidate image features, and use the candidate image data as the image candidate set.

In an implementation manner, the image recommendation module 30 of this embodiment includes:

The historical interaction data acquisition unit is configured to acquire historical query items generated by users through browsing, clicking, and bookmarking pictures within a historical time period, and use the historical query items as the user historical interaction data.

In an implementation manner, the image recommendation module 30 of this embodiment includes:

A feedback behavior data acquisition unit, configured to determine the feedback behavior data corresponding to the historical query item according to the user historical interaction data;

The recommendation list output unit is configured to sort each piece of image data in the image candidate set based on the ranking model and according to the feedback behavior data, and output the image recommendation list.

In an implementation manner, the recommendation list output unit of this embodiment includes:

A probability calculation subunit, configured to perform Bayesian analysis on each piece of image data in the image candidate set based on the ranking model to obtain the maximum a posteriori probability;

A list output subunit, configured to sort each piece of image data in the image candidate set based on the maximum a posteriori probability, and output the image recommendation list.

The working principle of each module in the image recommendation system based on deep hash retrieval in this embodiment is the same as the execution process of each step in the above method embodiment, and will not be repeated here.

Based on the foregoing embodiments, the present invention further provides a terminal device, and a functional block diagram of the terminal device may be shown in FIG. 6 . The terminal device of this embodiment may include one or more processors 100 (only one is shown in FIG. 6 ), a memory 101, and a computer program 102 stored in the memory 101 and operable on the one or more processors 100, For example, a program for image recommendation based on deep hash retrieval. When one or more processors 100 execute the computer program 102, various steps in the embodiment of the method for image recommendation based on deep hash retrieval can be implemented. Alternatively, when one or more processors 100 execute the computer program 102, they can implement the functions of the modules/units in the embodiment of the apparatus for image recommendation based on deep hash retrieval, which is not limited here.

In one embodiment, the so-called processor 100 may be a central processing unit (Central Processing Unit, CPU), and may also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.

In one embodiment, the storage 101 may be an internal storage unit of the electronic device, such as a hard disk or memory of the electronic device. The memory 101 can also be an external storage device of the electronic device, such as a plug-in hard disk equipped on the electronic device, a smart memory card (smart media card, SMC), a secure digital (secure digital, SD) card, a flash memory card (flash card) wait. Further, the memory 101 may also include both an internal storage unit of the electronic device and an external storage device. The memory 101 is used to store computer programs and other programs and data required by the terminal device. The memory 101 can also be used to temporarily store data that has been output or will be output.

Those skilled in the art can understand that the functional block diagram shown in Figure 6 is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the terminal equipment to which the solution of the present invention is applied. The specific terminal equipment It is possible to include more or fewer components than shown in the figures, or to combine certain components, or to have a different arrangement of components.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented through computer programs to instruct related hardware. The computer programs can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it may include the procedures of the embodiments of the above-mentioned methods. Wherein, any references to memory, storage, operational database or other media used in the various embodiments provided by the present invention may include non-volatile and/or volatile memory. Nonvolatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Dual Operating Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims (10)

1. An image recommendation method based on deep hash retrieval is characterized by comprising the following steps:

crawling image data from the Internet through a crawler program, and preprocessing the image data to obtain preprocessed image data;

inputting the preprocessed image data into a trained deep hash model to obtain a hash value, and determining an image candidate set based on the hash value;

obtaining historical interactive data of a user, inputting the historical interactive data of the user and the image candidate set into a preset sequencing model to obtain an image recommendation list, and feeding back the image recommendation list to the user.

2. The image recommendation method based on the deep hash search as claimed in claim 1, wherein the preprocessing the image data to obtain the preprocessed image data comprises:

and carrying out format detection, picture cleaning and picture indexing on the image data.

3. The image recommendation method based on deep hash search according to claim 1, wherein determining the candidate set of images based on the hash value comprises:

acquiring an index database, wherein the index database comprises image characteristics corresponding to the image data;

determining the image candidate set based on the hash value and the image feature.

4. The image recommendation method based on deep hash search according to claim 3, wherein the determining the image candidate set based on the hash value and the image feature comprises:

calculating the similarity between the image features of the image data based on the hash value, and acquiring the image features with the similarity higher than a preset threshold value to obtain candidate image features;

and obtaining candidate image data based on the candidate image characteristics, and taking the candidate image data as the image candidate set.

5. The image recommendation method based on deep hash retrieval according to claim 1, wherein the obtaining of the user history interaction data comprises:

and acquiring historical query items generated by browsing, clicking and collecting pictures by a user in a historical time period, and taking the historical query items as the historical interactive data of the user.

6. The image recommendation method based on the deep hash search as claimed in claim 1, wherein the inputting the user history interaction data and the image candidate set into a preset ranking model to obtain an image recommendation list comprises:

determining feedback behavior data corresponding to the historical query items according to the historical user interaction data;

and based on the sorting model, sorting each piece of image data in the image candidate set according to the feedback behavior data, and outputting the image recommendation list.

7. The image recommendation method based on deep hash search according to claim 6, wherein said sorting each image data in the image candidate set according to the feedback behavior data based on the sorting model, and outputting the image recommendation list comprises:

based on the ranking model, carrying out Bayesian analysis on each image data in the image candidate set to obtain a maximum posterior probability;

and sequencing each piece of image data in the image candidate set based on the maximum posterior probability, and outputting the image recommendation list.

8. An image recommendation system based on deep hash retrieval, the system comprising:

the image data processing module is used for crawling image data from the Internet through a crawler program and preprocessing the image data to obtain preprocessed image data;

the image candidate set determining module is used for inputting the preprocessed image data into a trained deep hash model to obtain a hash value, and determining an image candidate set based on the hash value;

and the image recommendation module is used for acquiring historical interaction data of a user, inputting the historical interaction data of the user and the image candidate set into a preset sequencing model to obtain an image recommendation list, and feeding the image recommendation list back to the user.

9. A terminal device, characterized in that the terminal device comprises a memory, a processor and an image recommendation program based on deep hash search, which is stored in the memory and can run on the processor, and when the processor executes the image recommendation program based on deep hash search, the steps of the image recommendation method based on deep hash search according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, wherein the computer-readable storage medium stores thereon an image recommendation program based on a depth hash search, and when the image recommendation program based on the depth hash search is executed by a processor, the image recommendation program based on the depth hash search implements the steps of the image recommendation method based on the depth hash search according to any one of claims 1 to 7.

Images