CN112905875A

CN112905875A - Offline interactive content recommendation method and system and storage medium

Info

Publication number: CN112905875A
Application number: CN201911228608.9A
Authority: CN
Inventors: 李夏
Original assignee: Shenzhen Intellifusion Technologies Co Ltd
Current assignee: Shenzhen Intellifusion Technologies Co Ltd
Priority date: 2019-12-04
Filing date: 2019-12-04
Publication date: 2021-06-04

Abstract

An offline interactive content recommendation method, system and storage medium, the method includes: acquiring a target image corresponding to an offline user; acquiring attribute information and human face characteristics of the offline user according to the target image; determining whether the offline user belongs to an old user recorded by the offline user system or not according to the facial features of the offline user; and if the offline user does not belong to the old user recorded by the offline user system, recommending corresponding content to the offline user according to the attribute information of the offline user. By implementing the embodiment of the application, even under the scenes that no online system is provided and the behavior data of the user cannot be acquired, such as shopping malls, book cities, tourist attractions and the like, the content (such as commodities) can be effectively recommended to the offline user; the accuracy of content recommendation is improved, and the occurrence of wrong recommendation is reduced; and, solving the cold start problem.

Description

Offline interactive content recommendation method and system and storage medium

Technical Field

The present application relates to the field of human-computer interaction technologies, and in particular, to an offline interactive content recommendation method and system, and a storage medium.

Background

In a conventional content recommendation system (e.g., a commodity recommendation system), content preferences of a user are generally mined by acquiring behavior data (e.g., search times, browsing times, page stay time, page access times, and the like) of the user in an online system, and then content recommendation is further performed to the user according to the content preferences of the user. In practice, it has been found that the conventional content recommendation system has at least the following technical drawbacks:

1. the behavior data of the user is captured by completely depending on the online system, if the recommended scene is offline, such as a market, a book city, a tourist attraction and other scenes without the online system, the behavior data of the user cannot be obtained, and further content recommendation cannot be performed;

2. only by analyzing the behavior data of the user in the online system and recommending the content, attribute information such as the sex, age, expression and the like of the user is difficult to obtain, so that the accuracy of content recommendation is influenced.

3. When users of different points of interest use the same online account, erroneous recommendations are likely to occur.

4. And the cold start problem, when the user is a non-registered user and has no any behavior data, effective recommendation cannot be carried out.

Disclosure of Invention

In order to solve at least one technical defect, embodiments of the present application disclose an offline interactive content recommendation method, system, and storage medium.

The embodiment of the application discloses a first aspect of an offline interactive content recommendation method, which comprises the following steps:

acquiring a target image corresponding to an offline user;

acquiring attribute information and human face characteristics of the offline user according to the target image;

determining whether the offline user belongs to an old user recorded by the offline user system or not according to the facial features of the offline user;

and if the offline user does not belong to the old user recorded by the offline user system, recommending corresponding content to the offline user according to the attribute information of the offline user.

As an optional implementation manner, in the first aspect of the embodiment of the present application, the obtaining, according to the target image, attribute information and a face feature of the offline user includes:

extracting a foreground image containing the offline user from the target image;

acquiring a face image of the offline user according to the foreground image;

inputting a first convolution neural network from the face image of the offline user to generate a feature region image set of the face image of the offline user; the first convolution neural network is used for extracting a characteristic region image from a face image;

inputting each characteristic region image in the characteristic region image set into a corresponding second convolutional neural network to generate a region face characteristic of the characteristic region image; the second convolutional neural network is used for extracting the regional face features of the corresponding feature region images;

generating the face features of the offline user according to the regional face features of each feature region image in the feature region image set;

calculating an initial feature map of the face image of the offline user; and calculating a response map of the initial feature map;

extracting the current response area according to the response graph; wherein the current response region is a connected region of which the response value is greater than a response threshold value in the response map;

calculating an attribute association area according to the average response area and the current response area, and pooling the attribute association area in an area of interest to obtain a pre-set undetermined characteristic map;

and predicting attribute information of the offline user according to the undetermined characteristic diagram.

As another optional implementation manner, in the first aspect of the embodiment of the present application, the determining, according to the facial features of the offline user, whether the offline user belongs to an old user recorded by the offline user system includes:

traversing the face features in the face feature library under the line according to the face features of the users under the line to determine whether the target face features exist in the face feature library under the line; the similarity between the target face features and the face features of the offline users meets a preset similarity threshold;

if the target face features exist in the face feature library, inquiring whether a user historical behavior record bound with the target face features is recorded in an offline user system;

if the offline user system does not record the user historical behavior record bound with the target human face characteristics, determining that the offline user does not belong to an old user recorded by the offline user system;

and if the offline user system records the user historical behavior record bound with the target human face characteristics, determining that the offline user belongs to the old user recorded by the offline user system.

As another optional implementation manner, in the first aspect of this embodiment of the present application, the method further includes:

if the offline user is determined to belong to an old user recorded by the offline user system, taking the user historical behavior record bound with the target human face characteristics as the user historical behavior record of the offline user, and determining a historical content set corresponding to the offline user according to the user historical behavior record of the offline user; each historical content in the historical content set corresponds to a respective user historical behavior type;

for each historical content in the historical content set, identifying a current credit value corresponding to the historical content; acquiring the heat weight matched with the user historical behavior type corresponding to the historical content according to the preset matching relationship between the historical behavior type and the heat weight;

calculating the product of the current credit value corresponding to the historical content and the heat weight matched with the historical behavior type of the user corresponding to the historical content, and taking the product as the user interest degree corresponding to the historical content;

determining the historical content with the highest interest degree of the user from the historical content set as historical interest content;

acquiring related content having a related relation with the historical interest content; and recommending the associated content to the offline user.

analyzing the target image to obtain space-time information corresponding to the offline user;

determining content resources corresponding to the space-time information by taking the space-time information as a basis;

the obtaining of the associated content having an association relation with the historical interest content includes:

and acquiring the associated content having the association relation with the historical interest content from the content resource.

As another optional implementation manner, in the first aspect of the embodiment of the present application, the recommending, to the offline user, corresponding content according to the attribute information of the offline user includes:

constructing a user feature vector of the offline user according to the attribute information of the offline user;

calculating the target similarity between the user feature vector of the offline user and the user feature vector of any user in a certain user cluster according to the Euclidean distance or the cosine similarity; wherein, the similarity between the user feature vectors of any two users in the user cluster is higher than a specified threshold;

judging whether the target similarity is higher than the specified threshold value, and if so, clustering the offline users to the user cluster;

and acquiring a content set corresponding to the user cluster, and recommending the content in the content set to the offline user.

A second aspect of the embodiments of the present application discloses an offline interactive content recommendation system, including:

the first acquisition unit is used for acquiring a target image corresponding to an offline user;

the second acquisition unit is used for acquiring attribute information and human face characteristics of the offline user according to the target image;

the first determining unit is used for determining whether the offline user belongs to an old user recorded by the offline user system according to the facial features of the offline user;

and the first recommending unit is used for recommending corresponding content to the offline user according to the attribute information of the offline user when the first determining unit determines that the offline user does not belong to the old user recorded by the offline user system.

As an optional implementation manner, in a second aspect of embodiments of the present application, the second obtaining unit includes:

a first extraction subunit, configured to extract a foreground image including the offline user from the target image;

the characteristic generating subunit is used for acquiring the face image of the offline user according to the foreground image; inputting a first convolution neural network from the face image of the offline user to generate a feature region image set of the face image of the offline user; the first convolution neural network is used for extracting a characteristic region image from a face image; inputting each characteristic region image in the characteristic region image set into a corresponding second convolutional neural network to generate a region face characteristic of the characteristic region image; the second convolutional neural network is used for extracting the regional face features of the corresponding feature region images; generating the face features of the offline user according to the regional face features of each feature region image in the feature region image set;

the attribute prediction subunit is used for calculating an initial feature map of the face image of the offline user; and calculating a response map of the initial feature map; extracting the current response area according to the response graph; wherein the current response region is a connected region of which the response value is greater than a response threshold value in the response map; calculating an attribute association area according to the average response area and the current response area, and pooling the attribute association area in an area of interest to obtain a pre-set undetermined characteristic map; and predicting attribute information of the offline user according to the undetermined characteristic diagram.

As another optional implementation manner, in a second aspect of embodiments of the present application, the first determining unit includes:

the traversal subunit is configured to traverse the face features in the offline face feature library according to the face features of the offline users, so as to determine whether a target face feature exists in the offline face feature library; the similarity between the target face features and the face features of the offline users meets a preset similarity threshold;

the query subunit is configured to query whether a user historical behavior record bound to the target face feature is recorded in the offline user system when the traversal subunit determines that the target face feature exists in the offline face feature library;

the determining subunit is used for querying the user historical behavior record which is not recorded by the offline user system and is bound with the target face feature in the querying subunit, and determining that the offline user does not belong to the old user recorded by the offline user system;

or, when the querying subunit queries that the offline user system records a user historical behavior record bound to the target face feature, determining that the offline user belongs to an old user recorded by the offline user system.

As another optional implementation manner, in the second aspect of the embodiment of the present application, the system further includes:

a second determining unit, configured to, when it is determined that the offline user belongs to an old user recorded by the offline user system, use a user historical behavior record bound to the target face feature as a user historical behavior record of the offline user, and determine, according to the user historical behavior record of the offline user, a historical content set corresponding to the offline user; each historical content in the historical content set corresponds to a respective user historical behavior type;

a third determining unit, configured to identify, for each historical content in the historical content set, a current score value corresponding to the historical content; acquiring the heat weight matched with the user historical behavior type corresponding to the historical content according to the preset matching relationship between the historical behavior type and the heat weight; calculating the product of the current credit value corresponding to the historical content and the heat weight matched with the historical behavior type of the user corresponding to the historical content, and taking the product as the user interest degree corresponding to the historical content; determining the historical content with the highest interest degree of the user from the historical content set as historical interest content;

a third obtaining unit, configured to obtain associated content having an association relationship with the historical interest content;

and the second recommending unit is used for recommending the associated content to the offline user.

the fourth determining unit is used for analyzing the target image to obtain the spatio-temporal information corresponding to the offline user; determining content resources corresponding to the space-time information by taking the space-time information as a basis;

the third obtaining unit is specifically configured to obtain, from the content resource, associated content having an association relationship with the historical interest content.

As another optional implementation manner, in a second aspect of embodiments of the present application, the first recommending unit includes:

the constructing subunit is configured to, when the first determining unit determines that the offline user does not belong to an old user recorded by the offline user system, construct a user feature vector of the offline user according to attribute information of the offline user;

a calculating subunit, configured to calculate, according to the euclidean distance or the cosine similarity, a target similarity between the user feature vector of the offline user and the user feature vector of any user in a certain user cluster; wherein, the similarity between the user feature vectors of any two users in the user cluster is higher than a specified threshold;

the clustering subunit is used for judging whether the target similarity is higher than the specified threshold value or not, and if so, clustering the offline users to the user cluster;

and the recommending subunit is used for acquiring a content set corresponding to the user cluster and recommending the content in the content set to the offline user.

A third aspect of the embodiments of the present application discloses an offline interactive content recommendation system, including:

a memory storing executable program code;

a processor coupled with the memory;

the processor calls the executable program code stored in the memory to execute the steps of the offline interactive content recommendation method disclosed in the first aspect of the embodiment of the present application.

A fourth aspect of the embodiments of the present application discloses a computer-readable storage medium, where computer instructions are stored on the computer-readable storage medium, and when the computer instructions are executed, the computer is caused to perform the steps of the offline interactive content recommendation method disclosed in the first aspect of the embodiments of the present application.

Compared with the prior art, the embodiment of the application has the following beneficial effects:

in the embodiment of the application, after the target image corresponding to the offline user is obtained, the attribute information and the face features of the offline user can be obtained according to the target image corresponding to the offline user; on the basis, if the user under the outgoing line is determined not to belong to the old user recorded by the user under the outgoing line system according to the face characteristics of the user under the outgoing line, the corresponding content is recommended to the user under the outgoing line system according to the attribute information of the user under the outgoing line system. Therefore, by implementing the embodiment of the application, even in the scenes such as shopping malls, book cities, tourist attractions and the like which are not provided with an online system and cannot acquire the online behavior data of the user, the content (such as commodities) can be effectively recommended to the offline user.

In addition, by implementing the embodiment of the application, the content (such as commodities) can be recommended according to the attribute information of the offline user, which is not only beneficial to improving the accuracy of content recommendation, but also beneficial to reducing the occurrence of wrong recommendation.

In addition, by implementing the embodiment of the application, even if the user is an unregistered offline user without any behavior data, effective recommendation can be performed, so that the cold start problem can be solved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

Fig. 1 is a schematic flowchart of an offline interactive content recommendation method disclosed in an embodiment of the present application;

fig. 2 is a flowchart illustrating another offline interactive content recommendation method according to an embodiment of the present application;

fig. 3 is a schematic structural diagram of an offline interactive content recommendation system according to an embodiment of the present application;

FIG. 4 is a schematic structural diagram of another offline interactive content recommendation system disclosed in an embodiment of the present application;

FIG. 5 is a schematic structural diagram of another offline interactive content recommendation system disclosed in an embodiment of the present application;

fig. 6 is a schematic structural diagram of another offline interactive content recommendation system disclosed in the embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that the terms "comprises" and "comprising," and any variations thereof, in the embodiments of the present application, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

The embodiment of the application discloses an offline interactive content recommendation method, an offline interactive content recommendation system and a storage medium, so that even in the scenes such as shopping malls, book cities, tourist attractions and the like which do not have an online system and cannot acquire behavior data of users, the offline interactive content recommendation method can effectively recommend the content (such as commodities) to the offline users; the accuracy of content recommendation is improved, and the occurrence of wrong recommendation is reduced; and, solving the cold start problem. The following detailed description is made with reference to the accompanying drawings.

Referring to fig. 1, fig. 1 is a flowchart illustrating an offline interactive content recommendation method according to an embodiment of the present application. The offline interactive content recommendation method described in fig. 1 is described from the perspective of a content recommendation system, which may belong to an offline user system in terms of physical implementation. The off-line user system is a system for providing intelligent service for off-line consumers or merchants in an off-line experience or consumption scene. As shown in fig. 1, the offline interactive content recommendation method may include the steps of:

101. the content recommendation system acquires a target image corresponding to the offline user.

In some embodiments, the content recommendation system may be provided with an offline interaction device (e.g., an offline interactive touch screen, an offline interaction robot, etc.), the offline user may perform content retrieval, browsing, clicking, etc. behavior operations through the offline interaction device, and the content recommendation system may collect and store content search, browsing, clicking, etc. behavior records performed by the offline user through the offline interaction device; in addition, the offline interaction device may be equipped with an image capturing device (e.g., a camera), and the content recommendation system may capture a target image corresponding to the offline user through the image capturing device equipped with the offline interaction device.

In some embodiments, the offline interaction device may be equipped with a human body sensing device (e.g., a human body proximity sensor, a thermal infrared human body sensor, etc.), and when the offline interaction device is equipped with the human body sensing device to sense the offline user, the content recommendation system may collect a target image corresponding to the offline user through an image collection device equipped with the offline interaction device.

In some embodiments, the target image corresponding to the offline user may be a color image or a grayscale image. Specific formats of the target image may include a JPG, BMP, RAW, and the like, and the embodiment of the present application is not limited.

102. And the content recommendation system acquires the attribute information and the face characteristics of the offline user according to the target image.

In some embodiments, the content recommendation system obtains the facial features of the offline user according to the target image, and may include steps 11) to 15), that is:

11) and the content recommendation system extracts a foreground image containing the offline user from the target image.

For example, the content recommendation system may identify a user profile of the offline user from the target image, and extract an area image surrounded by the user profile as a foreground image including the offline user, so that the foreground image including the offline user may be obtained quickly and conveniently.

For another example, the content recommendation system may employ a foreground extraction algorithm to accurately extract a foreground image including the offline user from the target image. Preferably, the foreground extraction algorithm can be realized by adopting a significance detection deep neural network model algorithm. It should be noted that, in the embodiment of the present application, the content recommendation system may also adopt other types of foreground extraction algorithms, and all foreground images including offline users may be accurately extracted from the target image, which is not described herein again.

12) And the content recommendation system acquires the face image of the offline user according to the foreground image.

For example, the content recommendation system can detect the face image of the offline user from the foreground image by using the fast-RCNN algorithm, and the method not only can adapt to environmental changes, but also can solve the problems of incomplete face and the like.

13) The content recommendation system inputs the face image of the offline user into the first convolution neural network to generate a feature area image set of the face image of the offline user; the first convolution neural network is used for extracting a characteristic region image from the face image.

For example, the feature region image may be an image for characterizing a feature of an arbitrary region of a human face, such as a left eye region image, a forehead region image, or the like.

In some embodiments, the content recommendation system may acquire a large number of sample face images and corresponding sample feature region images, and train to obtain the first convolutional neural network by taking the sample face images as input and the sample feature region images as output.

14) The content recommendation system inputs each characteristic region image in the characteristic region image set into a corresponding second convolutional neural network to generate region face characteristics of the characteristic region images; the second convolutional neural network is used for extracting the regional face features of the corresponding feature region images.

For example, the content recommendation system may input the feature region image of the nose into a second convolutional neural network for processing the nose image. Here, the regional face feature may be a face feature for describing a regional face image in the feature region image, such as a face feature of a nose region.

In some implementations, the second convolutional neural network can be trained by: firstly, a sample characteristic region image can be obtained; then, the sample feature region images can be classified according to the feature regions (such as nose, mouth, etc.) displayed by the sample feature region images; next, a second convolutional neural network corresponding to the feature region may be obtained by training using the sample feature region image belonging to the same feature region as an input. It should be noted that the acquired sample feature region image may be (but is not limited to) generated by the first convolutional neural network.

15) And the content recommendation system generates the facial features of the offline user according to the regional facial features of the feature region images in the feature region image set.

For example, the content recommendation system may perform deduplication processing on the region face features of each feature region image in the feature region image set, and may store the region face features subjected to deduplication processing, thereby generating the face features of the offline user.

In the embodiment of the present application, the above steps 11) to 15) are implemented, and the content recommendation system can effectively generate the facial features of the offline user.

In some embodiments, the content recommendation system obtains attribute information of the offline user according to the target image, and may include steps 21) to 26), that is:

21) and the content recommendation system extracts a foreground image containing the offline user from the target image.

22) And the content recommendation system acquires the face image of the offline user according to the foreground image.

23) The content recommendation system calculates an initial characteristic diagram of a face image of an offline user; and calculating a response map of the initial feature map.

For example, the content recommendation system may use a feature map obtained after mapping the output of the convolutional layer of the convolutional neural network to the initial feature map of the face image of the offline user as the response map.

24) The content recommendation system extracts a current response area according to the response graph; and the current response area is a connected area of which the response value is greater than the response threshold value in the response map.

In the convolutional neural network, the response value of the feature map is closely related to the learning target, and in the present embodiment, the response value of the feature map is closely related to the attribute information to be identified. In the feature map corresponding to a certain attribute information, the larger the response value, the higher the possibility that the attribute information exists in the region, that is, the larger the response value in the feature map, the stronger the relevance. The response value of each layer of feature map can be obtained through the forward propagation process of the convolutional neural network. And further, taking the connected region with the response value larger than a certain response threshold value in the response map as the current response region. The response threshold may be obtained empirically or through a limited number of experiments, such as setting the response threshold to an empirical value of 0.5.

25) And the content recommendation system calculates an attribute association area according to the average response area and the current response area, and performs region-of-interest pooling on the attribute association area to obtain a pre-set undetermined characteristic map.

26) And predicting the attribute information of the users under the line according to the undetermined characteristic diagram by the content recommendation system.

The content recommendation system can be implemented according to the prior art according to the face attribute of the user under the prediction line of the undetermined feature map, and details are not repeated in the embodiment of the application.

In this embodiment of the present application, the attribute information of the offline user may include a gender, an age, an expression, an attention, and the like of the offline user, and this embodiment of the present application is not limited.

In the embodiment of the present application, the above steps 21) to 26) are implemented, and the content recommendation system can effectively and completely predict the attribute information of the user under the line.

103. The content recommendation system determines whether the offline user belongs to an old user recorded by the offline user system according to the facial features of the offline user; if not, go to step 104; if yes, go to step 105.

In some embodiments, the content recommendation system determining whether the offline user belongs to an old user recorded by the offline user system according to the facial features of the offline user may include the following steps:

the content recommendation system traverses the face features in the face feature library according to the face features of the offline users to determine whether the target face features exist in the face feature library offline; the similarity between the target face features and the face features of the offline users meets a preset similarity threshold;

if the target face features do not exist in the offline face feature library, the content recommendation system can determine that the offline user does not belong to an old user recorded by the offline user system; or, if the target face features exist in the off-line face feature library, the content recommendation system may further query whether a user historical behavior record bound with the target face features is recorded in the off-line user system;

if the offline user system does not record the user historical behavior record bound with the target face features, the content recommendation system determines that the offline user does not belong to an old user recorded by the offline user system; or if the offline user system records the user historical behavior record bound with the target face feature, the content recommendation system determines that the offline user belongs to the old user recorded by the offline user system.

104. And the content recommendation system recommends corresponding content to the offline user according to the attribute information of the offline user.

In some embodiments, the content recommendation system recommends corresponding content to the offline user according to the attribute information of the offline user, and may include the following steps:

the content recommendation system constructs a user feature vector of the offline user according to the attribute information of the offline user;

the content recommendation system calculates the target similarity between the user feature vector of the offline user and the user feature vector of any user in a certain user cluster according to the Euclidean distance or cosine similarity; wherein, the similarity between the user feature vectors of any two users in the user cluster is higher than a specified threshold;

the content recommendation system judges whether the target similarity is higher than the specified threshold value, and if so, the offline users are clustered to the user cluster;

and the content recommendation system acquires a content set corresponding to the user cluster and recommends the content in the content set to the offline user.

In the embodiment of the present application, the attribute information of the offline user may be characterized by using the attribute value of the offline user, for example, the gender attribute of the offline user may be characterized by "1" and "0", where "1" may characterize male and "0" may characterize female; as another example, the age attribute of an offline user may be characterized by the age (e.g., 35) of the offline user. On this basis, the content recommendation system can sort the attribute values of the attribute information for representing the offline users according to the designated attribute value arrangement sequence, so as to construct the user feature vectors of the offline users.

For example, according to the cosine similarity, the content recommendation system may calculate the target similarity between the user feature vector of the offline user and the user feature vector of any user in a certain user cluster by: the content recommendation system calculates an included angle between the user feature vector of the offline user and the user feature vector of any user in a certain user cluster according to a cosine similarity calculation formula, and obtains a cosine value corresponding to the included angle, and the cosine value can be used as a target similarity for representing the similarity of the two user feature vectors. Wherein, the smaller the included angle is, the closer the cosine value is to 1, and the more the directions of the two user feature vectors are matched, the more similar the two user feature vectors are.

In some embodiments, the content recommendation system may automatically mine the correspondence between the user clusters and the content sets from a vast amount of internet data using a data mining algorithm. For example, the content recommendation system may first mine user attribute information of each user who has undergone user history behavior recording (e.g., search, browse) for the same or different content within a certain history time period from massive internet data through a data mining algorithm, then respectively construct user feature vectors corresponding to the users according to the user attribute information of the users, further select a part of the users from the users to cluster into a user cluster based on the user feature vectors corresponding to the users, and the similarity between the user feature vectors of any two users in the user cluster is higher than a specified threshold; and finally, aggregating the contents which are aimed by each user in the user cluster and have undergone the user historical behavior record in the historical time period to form a content set, and establishing the corresponding relation between the user cluster and the content set.

105. And the content recommendation system recommends corresponding content to the offline user according to the user historical behavior record of the offline user.

In some embodiments, the content recommendation system may mine the historical interest content of the offline user according to the historical behavior record of the offline user, and then recommend the content to the offline user based on the historical interest content, thereby facilitating the improvement of the probability of recommending the interest content to the offline user.

In the method described in fig. 1, even in a scenario such as a shopping mall, a book city, a tourist attraction, etc. which does not have an online system and cannot acquire behavior data of a user, content (such as a commodity) can be effectively recommended to the offline user.

In addition, by implementing the method described in fig. 1, content (such as a commodity) recommendation can be performed according to the attribute information of the offline user, which is not only beneficial to improving the accuracy of content recommendation, but also beneficial to reducing the occurrence of wrong recommendation.

In addition, by implementing the method described in fig. 1, even if the user is an unregistered offline user without any behavior data, effective recommendation can be performed, so that the cold start problem can be solved.

Referring to fig. 2, fig. 2 is a flowchart illustrating another offline interactive content recommendation method according to an embodiment of the present application. The offline interactive content recommendation method described in fig. 2 is described from the perspective of a content recommendation system. As shown in fig. 2, the offline interactive content recommendation method may include the steps of:

201. the content recommendation system acquires a target image corresponding to the offline user.

The implementation manner of step 201 may be similar to step 101, and is not described herein again.

202. And the content recommendation system acquires the attribute information and the face characteristics of the offline user according to the target image.

The implementation manner of step 202 may be similar to that of step 102, and is not described here.

203. The content recommendation system traverses the face features in the face feature library according to the face features of the offline users to determine whether the target face features exist in the face feature library offline; the similarity between the target face features and the face features of the offline users meets a preset similarity threshold; if the target face features exist, executing step 204; if the target face features do not exist, step 205 to step 206 are executed.

204. The content recommendation system inquires whether a user historical behavior record bound with the target face feature is recorded in an offline user system; if the user historical behavior record bound with the target face feature is not recorded, executing the step 205 to the step 206; and if the user historical behavior record bound with the target face features is recorded, executing the step 207 to the step 208.

205. The content recommendation system determines that the user under the outgoing line does not belong to the old user recorded by the user under the outgoing line system.

206. And the content recommending system recommends corresponding content to the offline user according to the attribute information of the offline user, and the process is ended.

The implementation manner of step 206 may be similar to step 104, and is not described herein.

207. The content recommendation system determines that the user who is off-line belongs to the old user recorded by the off-line user system.

208. The content recommendation system determines a historical content set corresponding to the target face features according to the user historical behavior record bound with the target face features; and each historical content in the historical content set corresponds to a respective user historical behavior type.

For any content in the historical content set, the corresponding user historical behavior type may include, but is not limited to, any one or combination of searches, clicks, and browsing.

209. The content recommendation system identifies a current credit value corresponding to each historical content in the historical content set; acquiring the heat weight matched with the user historical behavior type corresponding to the historical content according to the preset matching relationship between the historical behavior type and the heat weight; and calculating the product of the current credit value corresponding to the historical content and the heat weight matched with the historical behavior type of the user corresponding to the historical content, and taking the product as the user interestingness corresponding to the historical content.

In some embodiments, for each historical content in the historical content set, the content recommendation system may identify the total number of current good scores of the historical content, and obtain, as the current score value corresponding to the historical content, a score value in direct proportion to the total number of current good scores of the historical content based on the total number of current good scores of the historical content. That is, when the total number of current favorable scores of the history content is larger, the current score value corresponding to the history content is larger; otherwise, the smaller the total number of current good scores of the historical content is, the smaller the current score value corresponding to the historical content is.

It should be noted that the total number of current favorable comments of the historical content includes not only the number of favorable comments of the offline user on the historical content, but also the number of favorable comments of other users (e.g., online users) on the historical content.

In some embodiments, the matching heat weights of different historical behavior types may be the same or different, and this application example is not limited. For example, the heat weight for a search (of a historical behavior type) match is 0.5, the heat weight for a click (of a historical behavior type) match is also 0.5, and the heat weight for a browse (of a historical behavior type) match is 1.

In some embodiments, when there are multiple user historical behavior types corresponding to the historical content, calculating a product of a current credit value corresponding to the historical content and a heat weight matched with the user historical behavior type corresponding to the historical content as the user interestingness corresponding to the historical content may include: respectively calculating the products of the current credit values corresponding to the historical contents and the heat weights matched with each user historical behavior type corresponding to the historical contents to obtain each product; and taking the accumulated value of each product as the user interest degree corresponding to the historical content.

For example, suppose that the historical behavior types of the user corresponding to a certain historical content include three types, i.e., search, click and browse, and the heat weight of the search matching is 0.5, the heat weight of the click matching is also 0.5, and the heat weight of the browse matching is 1; and, assuming that the current score value corresponding to the historical content is 80; accordingly, the content recommendation system may calculate the user interest degree D1 corresponding to the historical content by the following formula:

D1＝((80*0.5)+(80*0.5)+(80*1))＝160。

for example, assume that the historical behavior types of the user corresponding to another historical content are two types, that is, clicking and browsing, and the hot weight of the click match is 0.5 and the hot weight of the browsing match is 1; and, assuming that the current score value corresponding to the historical content is 90; accordingly, the content recommendation system may calculate the user interest degree D2 corresponding to the historical content by the following formula:

D2＝((90*0.5)+(90*1))＝135。

210. the content recommendation system determines the historical content with the highest user interest degree from the historical content set as historical interest content; and acquiring the associated content having the association relation with the historical interest content.

In some embodiments, the content recommendation system may further perform the steps of:

the content recommendation system analyzes the target image to obtain the spatio-temporal information corresponding to the offline user; the spatio-temporal information at least may include a spatial position and time corresponding to an offline user, where the time may be a time point when the offline user appears at the spatial position, or the time may also be a staying time of the offline user at the spatial position, and the embodiment of the present application is not limited;

the content recommendation system determines content resources corresponding to the time-space information based on the time-space information;

accordingly, the content recommendation system obtaining the associated content having an association relationship with the historical interest content may include: and the content recommendation system acquires the associated content which has the association relation with the historical interest content from the content resource.

For example, the content recommendation system analyzes the target image, and the manner of obtaining the spatial position corresponding to the offline user may be:

the content recommendation system identifies a face area of a user under the line from the target image, and identifies the distance l of the user under the line relative to a lens of image acquisition equipment for shooting the target image according to the area of the face area of the user under the line; identifying the direction f of the offline user relative to a lens of image acquisition equipment which shoots the target image according to the position of the area of the face area of the offline user in the target image; and determining the instant three-dimensional coordinates of the offline user when the image acquisition equipment acquires the target image as the corresponding spatial position of the offline user according to the known three-dimensional coordinates of the lens of the image acquisition equipment which acquires the target image, the distance l of the offline user relative to the lens of the image acquisition equipment which acquires the target image and the direction f of the offline user relative to the lens of the image acquisition equipment which acquires the target image.

For example, when the time corresponding to the offline user is included in the spatiotemporal information and is a time point at which the offline user appears at the spatial position, the content recommendation system analyzes the target image, and the time corresponding to the offline user may be obtained as: the content recommendation system may use a time point when the image capture device captures the target image as a time point corresponding to the offline user.

For example, based on the spatio-temporal information, the content recommendation system may determine the content resource corresponding to the spatio-temporal information by: the content recommendation system determines the content resource corresponding to the space position and the time from the preset corresponding relationship among the space position, the time and a certain content resource according to the space position and the time corresponding to the offline user included in the spatio-temporal information, and uses the content resource corresponding to the spatio-temporal information as the content resource (such as clothes, books and the like) corresponding to the spatio-temporal information.

In one embodiment, the associated content having an association relationship with the historical interest content acquired from the content resource may be: the content which is acquired from the content resource and has a plurality of same content attributes with the historical interest content is taken as the related content. For example, when the historical interest content is a certain dress, the content attribute of the historical interest content may include a style, a color, a suitable crowd, a material, and the like of the dress; accordingly, the content recommendation system may obtain, from the content resource, a clothing that has a plurality of same content attributes on the content attributes of the clothing, such as style, color, suitable crowd, and material, as the associated content. For another example, when the historical interest content is a book, the content attribute of the historical interest content may include the publishing company, language, author, suitable age, literature type, paper type, etc. of the book; accordingly, the content recommendation system can acquire, from the content resource, a book having a plurality of identical content attributes in content attributes such as a publisher, a language, an author, an appropriate age, a literature type, and a paper type of the book, as the related content.

211. The content recommendation system recommends the associated content to the offline user.

In some embodiments, after the content recommendation system determines that the offline user belongs to an old user recorded by the offline user system, the content recommendation system may detect whether the quality of the target image meets a requirement (e.g., whether the face region of the offline user is complete), and if the quality of the target image meets the requirement, the retrieved target face features are fused and updated according to the face features of the offline user for the next retrieval.

In some embodiments, whether the offline user is determined to be a new user or an old user, the behavior record associated with the offline user will be uniquely bound to facial features characterizing the offline user and stored in the offline user system.

In the method described in fig. 2, even in a scenario such as a shopping mall, a book city, a tourist attraction, etc. which does not have an online system and cannot acquire behavior data of a user, content (such as a commodity) can be effectively recommended to the offline user.

In addition, by implementing the method described in fig. 2, content (such as a commodity) recommendation can be performed according to the attribute information of the offline user, which is not only beneficial to improving the accuracy of content recommendation, but also beneficial to reducing the occurrence of wrong recommendation.

In addition, by implementing the method described in fig. 2, even if the user is an unregistered offline user without any behavior data, effective recommendation can be performed, so that the cold start problem can be solved.

Referring to fig. 3, fig. 3 is a block diagram illustrating an offline interactive content recommendation system according to an embodiment of the present application. As shown in fig. 3, the offline interactive content recommendation system may include:

a first obtaining unit 301, configured to obtain a target image corresponding to an offline user;

a second obtaining unit 302, configured to obtain attribute information and facial features of an offline user according to a target image;

a first determining unit 303, configured to determine, according to the facial features of the offline user, whether the offline user belongs to an old user recorded by the offline user system;

and the first recommending unit 304 is configured to recommend corresponding content to the offline user according to the attribute information of the offline user when the first determining unit 303 determines that the offline user does not belong to an old user recorded by the offline user system.

The content recommendation system described in fig. 3 can effectively recommend content (such as a commodity) to an offline user even in a scene such as a shopping mall, a book city, a tourist attraction, etc. which does not have an online system and cannot acquire behavior data of the user.

In addition, by implementing the content recommendation system described in fig. 3, content (such as a commodity) recommendation can be performed according to the attribute information of the offline user, which is not only beneficial to improving the accuracy of content recommendation, but also beneficial to reducing the occurrence of wrong recommendation.

In addition, by implementing the content recommendation system described in fig. 3, even if the user is an unregistered offline user without any behavior data, effective recommendation can be performed, so that the cold start problem can be solved.

Referring to fig. 4, fig. 4 is a schematic block diagram of another offline interactive content recommendation system according to an embodiment of the present application. The content recommendation system described in fig. 4 is optimized by the content recommendation system described in fig. 3. In the offline interactive content recommendation system described in fig. 4, the second obtaining unit 302 includes:

a first extraction subunit 3021 configured to extract a foreground image including an offline user from the target image;

a feature generation subunit 3022, configured to obtain a face image of the offline user according to the foreground image; inputting a first convolution neural network from a face image of an offline user to generate a feature region image set of the face image of the offline user; the first convolution neural network is used for extracting a characteristic region image from a face image; inputting each characteristic region image in the characteristic region image set into a corresponding second convolutional neural network to generate region face characteristics of the characteristic region images; the second convolutional neural network is used for extracting the regional face features of the corresponding feature region images; generating the face features of the offline user according to the regional face features of each feature region image in the feature region image set;

an attribute prediction subunit 3023, configured to calculate an initial feature map of a face image of an offline user; and calculating a response map of the initial feature map; extracting a current response area according to the response graph; the current response area is a connected area of which the response value is greater than the response threshold value in the response map; calculating an attribute associated region according to the average response region and the current response region, and pooling the attribute associated region in an interested region to obtain a predetermined undetermined characteristic map; and predicting attribute information of the offline users according to the undetermined characteristic diagram.

As an alternative implementation, in the content recommendation system described in fig. 4, the first determining unit 303 includes:

a traversal subunit 3031, configured to traverse, according to the face features of the offline user, the face features in the offline face feature library to determine whether the target face feature exists in the offline face feature library; the similarity between the target face features and the face features of the offline users meets a preset similarity threshold;

the inquiry subunit 3032 is configured to inquire whether a user historical behavior record bound to the target face feature is recorded in the offline user system when the traversal subunit 3031 determines that the target face feature exists in the face feature library below the outgoing line;

the determining subunit 3033 is configured to query, by the querying subunit 3032, that the user system under the outgoing line does not record the user historical behavior record bound to the target face feature, and determine that the user under the outgoing line does not belong to an old user recorded by the user system under the outgoing line;

or, the query subunit 3032 is configured to determine that the user under the outgoing line belongs to an old user recorded by the user under the outgoing line system when the user system records a user historical behavior record bound to the target face feature.

As an alternative implementation manner, in the content recommendation system described in fig. 4, the first recommendation unit 304 includes:

a constructing subunit 3041, configured to, when the first determining unit 303 determines that the offline user does not belong to an old user recorded by the offline user system, construct a user feature vector of the offline user according to the attribute information of the offline user;

a calculating subunit 3042, configured to calculate, according to the euclidean distance or the cosine similarity, a target similarity between the user feature vector of the offline user and the user feature vector of any user in a certain user cluster; the similarity between the user feature vectors of any two users in the user cluster is higher than a specified threshold;

a clustering subunit 3043, configured to determine whether the target similarity is higher than the specified threshold, and if so, cluster the offline users into the user cluster;

a recommending subunit 3044, configured to obtain a content set corresponding to the user cluster, and recommend the content located in the content set to the offline user.

The content recommendation system described in fig. 4 can recommend content to the offline user according to the attribute information of the offline user, thereby improving the accuracy of recommending content to the offline user.

Referring to fig. 5, fig. 5 is a block diagram illustrating another offline interactive content recommendation system according to an embodiment of the present application. The content recommendation system described in fig. 5 is optimized by the content recommendation system described in fig. 4. In the offline interactive content recommendation system described in fig. 5, the method may further include:

a second determining unit 305, configured to determine, when it is determined that the user under the outgoing line belongs to an old user recorded by the offline user system, a historical content set corresponding to the target face feature according to the user historical behavior record bound to the target face feature; each historical content in the historical content set corresponds to a respective user historical behavior type;

a third determining unit 306, configured to identify, for each historical content in the historical content set, a current rating value corresponding to the historical content; acquiring the heat weight matched with the user historical behavior type corresponding to the historical content according to the preset matching relationship between the historical behavior type and the heat weight; calculating the product of the current credit value corresponding to the historical content and the heat weight matched with the historical behavior type of the user corresponding to the historical content, and taking the product as the user interest degree corresponding to the historical content; determining the historical content with the highest user interest degree from the historical content set as historical interest content;

a third obtaining unit 307 configured to obtain associated content having an association relationship with the history interest content;

and a second recommending unit 308 for recommending the associated content to the offline user.

As an alternative implementation, in the content recommendation system described in fig. 5, the method further includes:

a fourth determining unit 309, configured to analyze the target image to obtain spatio-temporal information corresponding to the offline user; and determining content resources corresponding to the spatio-temporal information based on the spatio-temporal information;

accordingly, the third obtaining unit 307 is specifically configured to obtain the associated content having an association relationship with the historical interest content from the content resource.

By implementing the content recommendation system described in fig. 5, the historical interest content of the offline user can be mined according to the historical behavior record of the offline user, and then the content is recommended to the offline user based on the historical interest content, so that the probability of recommending the interest content to the offline user is improved.

Referring to fig. 6, fig. 6 is a block diagram illustrating another offline interactive content recommendation system according to an embodiment of the present application. As shown in fig. 6, the offline interactive content recommendation system may include:

a memory 601 in which executable program code is stored;

a processor 602 coupled with the memory;

wherein, the processor 602 calls the executable program code stored in the memory 601 to execute the following steps:

acquiring a target image corresponding to an offline user;

acquiring attribute information and human face characteristics of an offline user according to a target image;

In some embodiments, the processor 602 obtains facial features of the offline user according to the target image, including:

extracting a foreground image containing an offline user from the target image;

acquiring a face image of the offline user according to the foreground image;

inputting a first convolution neural network from a face image of an offline user to generate a feature region image set of the face image of the offline user; the first convolution neural network is used for extracting a characteristic region image from a face image;

inputting each characteristic region image in the characteristic region image set into a corresponding second convolutional neural network to generate region face characteristics of the characteristic region images; the second convolutional neural network is used for extracting the regional face features of the corresponding feature region images;

and generating the facial features of the offline user according to the regional facial features of each feature region image in the feature region image set.

In some embodiments, the processor 602 obtains attribute information of the offline user according to the target image, including:

extracting a foreground image containing an offline user from the target image;

acquiring a face image of the offline user according to the foreground image;

calculating an initial characteristic diagram of the face image of the offline user; and calculating a response map of the initial feature map;

extracting a current response area according to the response graph; wherein, the current response area is a connected area of which the response value is greater than the response threshold value in the response map;

calculating an attribute associated region according to the average response region and the current response region, and pooling the attribute associated region in an interested region to obtain a predetermined undetermined characteristic map;

and predicting attribute information of the offline users according to the undetermined characteristic diagram.

In some embodiments, the processor 602 determines whether the offline user belongs to an old user recorded by the offline user system according to the facial features of the offline user, including:

traversing the face features in the face feature library according to the face features of the offline users to determine whether the target face features exist in the face feature library under the line; the similarity between the target face features and the face features of the offline users meets a preset similarity threshold;

if the user historical behavior record bound with the target face feature is not recorded in the offline user system, determining that the offline user does not belong to an old user recorded by the offline user system;

and if the user historical behavior record bound with the target face feature is recorded in the offline user system, determining that the offline user belongs to the old user recorded in the offline user system.

In some embodiments, processor 602 calls executable program code stored in memory 601 and may further perform the following steps:

if the user under the outgoing line is determined to belong to an old user recorded by an offline user system, taking the user historical behavior record bound with the target face characteristics as the user historical behavior record of the user under the outgoing line, and determining a historical content set corresponding to the user under the outgoing line according to the user historical behavior record of the user under the outgoing line; each historical content in the historical content set corresponds to a respective user historical behavior type;

identifying a current credit value corresponding to the historical content aiming at each historical content in the historical content set; acquiring the heat weight matched with the user historical behavior type corresponding to the historical content according to the preset matching relationship between the historical behavior type and the heat weight;

determining historical content with the highest user interest degree from the historical content set as historical interest content;

acquiring related content having a related relation with the historical interest content; and recommending the associated content to an offline user.

determining content resources corresponding to the space-time information based on the space-time information;

accordingly, the processor 602 obtains the associated content having an association relationship with the historical interest content, including:

In some embodiments, the processor 602 recommends the corresponding content to the offline user according to the attribute information of the offline user, including:

constructing a user characteristic vector of the offline user according to the attribute information of the offline user;

calculating the target similarity between the user characteristic vector of the offline user and the user characteristic vector of any user in a certain user cluster according to the Euclidean distance or the cosine similarity; wherein, the similarity between the user feature vectors of any two users in the user cluster is higher than a specified threshold;

judging whether the target similarity is higher than a specified threshold value, and if so, clustering offline users to the user cluster;

In the content recommendation system described in fig. 6, even in a scenario where no online system is available and the behavior data of the user cannot be obtained, such as a shopping mall, a book city, a tourist attraction, etc., the content (e.g., a commodity) recommendation system can be effectively performed to the offline user.

In addition, by implementing the content recommendation system described in fig. 6, content (such as a commodity) recommendation can be performed according to the attribute information of the offline user, which is not only beneficial to improving the accuracy of content recommendation, but also beneficial to reducing the occurrence of wrong recommendation.

In addition, by implementing the content recommendation system described in fig. 6, even if the user is an unregistered offline user without any behavior data, effective recommendation can be performed, so that the cold start problem can be solved.

The embodiment of the application discloses a computer-readable storage medium, wherein computer instructions are stored on the computer-readable storage medium, and when the computer instructions are executed, the computer executes each step of the offline interactive content recommendation method disclosed in the embodiment of the application, and the same technical effect can be achieved, and in order to avoid repetition, the description is omitted here.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.

The above disclosure is only for the purpose of illustrating the preferred embodiments of the present application and is not to be construed as limiting the scope of the present application, so that the present application is not limited thereto, and all equivalent variations and modifications can be made to the present application.

Claims

1. An offline interactive content recommendation method, the method comprising:

acquiring a target image corresponding to an offline user;

2. The content recommendation method according to claim 1, wherein the determining whether the offline user belongs to an old user recorded by an offline user system according to the facial features of the offline user comprises:

3. The content recommendation method according to claim 2, characterized in that the method further comprises:

4. The content recommendation method according to claim 3, characterized in that the method further comprises:

5. The content recommendation method according to any one of claims 1 to 4, wherein recommending corresponding content to the offline user according to the attribute information of the offline user comprises:

6. An offline interactive content recommendation system, comprising:

7. The content recommendation system according to claim 6, wherein said first determination unit comprises:

8. The content recommendation system according to claim 7, characterized in that said system further comprises:

9. An offline interactive content recommendation system, comprising:

a memory storing executable program code;

a processor coupled with the memory;

the processor calls the executable program code stored in the memory to execute the steps of the offline interactive content recommendation method according to any one of claims 1 to 5.

10. A computer readable storage medium having stored thereon computer instructions which, when executed, cause a computer to perform the steps of the offline interactive content recommendation method of any one of claims 1 to 5.