CN113807978A

CN113807978A - Hidden community attribute acquisition method and system based on attention-seeking neural network

Info

Publication number: CN113807978A
Application number: CN202111047006.0A
Authority: CN
Inventors: 张毅; 曹万华; 刘俊涛; 饶子昀; 王元斌; 王军伟; 周莹; 王振杰
Original assignee: 709th Research Institute of CSIC
Current assignee: 709th Research Institute of CSIC
Priority date: 2021-09-07
Filing date: 2021-09-07
Publication date: 2021-12-17

Abstract

The invention discloses a hidden community attribute acquisition method based on an attention-seeking neural network, which comprises the following steps: all words in a user social media data Word library are learned through a Word vector model Word2vec network to obtain embedded representation vectors of all words; obtaining an embedded representation of a target user based on a target embedded layer of a forward full-connection network by normalizing and weighting an embedded representation vector of a user social media data vocabulary; generating embedded representation of a neighbor user of a target user based on the user social network and the social activity information, and calculating a weighted social matrix according to the embedded representation of the neighbor user; and training a social popularity weighted attention-seeking neural network according to the weighted social matrix, and generating a hidden community attribute classification result of the target user by using the attention-seeking neural network and the embedded representation of the target user. The invention also provides a corresponding hidden community attribute acquisition system based on the attention-seeking neural network.

Description

Hidden community attribute acquisition method and system based on attention-seeking neural network

Technical Field

The invention belongs to the technical field of attribute mining, and particularly relates to a hidden community attribute acquisition method and system based on an attention-seeking neural network.

Background

User research in a network space is one of important tasks in the current internet personalized recommendation field, and with continuous expansion of social network scale, user attribute information in the network space has the characteristics of sparsity, fragmentity, heterogeneity and the like, so that acquisition of certain hidden community attribute information of a target user is very difficult, and further analysis and recommendation work is difficult. How to mine user potential or hidden community attribute information in a social network platform through effective technical means is crucial to analyzing potential preferences exhibited by target users with respect to contents to be recommended.

For user attribute cognitive analysis, most of attribute features adopted in the existing research are lightweight features, the contained information is not rich enough, and the attribute embedding representation and exploration of a target user are not sufficient, so that a large amount of information still exists for mining. Secondly, the attribute classification of the target users generated by the existing research is still too simple, the target users are usually the classification targets with simpler gender, education background and the like, the exploration on the fusion aspect of various media content information is still very lacked, and a larger explorable space still exists for the content behavior characterization method of the target users.

Disclosure of Invention

Aiming at the defects or the improvement requirements of the prior art, the invention provides a hidden community attribute acquisition method based on an attention-seeking neural network, which integrates text content information issued by target users and social information among the target users to acquire the hidden community attribute information and also considers information mining of interaction between target output content and a target of the social network. For example, two users having the same preferences and having mutual attention are likely to mutually favor or forward media data generated by each other, thereby forming a close target connection relationship. The invention realizes the uniform representation of the cross-space target attribute information, more efficiently excavates the hidden community attribute information of the target user in the network space, and is convenient for further analysis and recommendation work.

To achieve the above object, according to an aspect of the present invention, there is provided a hidden community attribute obtaining method based on an attention-seeking neural network, including:

step S1: all words in a user social media data Word library are learned through a Word vector model Word2vec network to obtain embedded representation vectors of all words;

step S2: obtaining an embedded representation of a target user based on a target embedded layer of a forward full-connection network by normalizing and weighting an embedded representation vector of a user social media data vocabulary;

step S3: generating embedded representation of a neighbor user of a target user based on the user social network and the social activity information, and calculating a weighted social matrix according to the embedded representation of the neighbor user;

step S4: and training a social popularity weighted attention-seeking neural network according to the weighted social matrix, and generating a hidden community attribute classification result of the target user by using the attention-seeking neural network and the embedded representation of the target user.

In an embodiment of the present invention, the step S1 includes: the user social media data vocabulary formed by the social media data set containing all the users is known as

Wherein c is_i∈R^fRepresenting the one-hot code of the ith Word in the vocabulary library, and the variable f ═ C |, representing the number of different vocabularies in the vocabulary library, and embedding the characterization vector for the vocabulary in C through Word2vec network learning

Vectorizing words in a vocabulary library from a collection of textual content, where each w_iThe word embedding representation of the ith word, and k is the vector dimension of each word after network learning.

In an embodiment of the present invention, the step S2 includes: obtaining the embedded characterization vectors E of all words_wThen, because the basic object for analyzing the hidden community attribute is the user, the vocabulary embedded representation needs to be converted into the target user embedded representation to enable the subsequent attention-seeking neural network to analyze by taking the target user as the object, and therefore, the target embedded layer based on the forward full-connection network embeds the vocabularies of all words into the representation E_wEmbedded representation converted into target user through full-connection form

Where n represents the number of target users, u_i′∈R^kAn embedded token representing the ith target user.

In an embodiment of the present invention, the step S2 specifically includes: for target user u^*And target user u^*All words co-occurring in the same social media content form the user's characterization word set, denoted as { u^*}＝{w_1′ w_2′… w_f′Each one of w_i′Belong to E_wFor a target user u^*In other words, the number f' of the corresponding elements in the token vocabulary set does not exceed E_wThe position of these words and the distance of the text from the target user are recorded as { u }^*The distances between all vocabularies and target users in the text data are normalized and then are given different weights { p }₁,…p_f′Is formed into u^*′＝p₁w_1′+p₂w_2′+…p_f′w_f′As an embedded representation of the target user.

In an embodiment of the present invention, the step S3 includes:

for target user u^*The various social relationships of (a) are converted into weight parameters;

metric target user u^*Social popularity among its neighbor users, social popularity coefficient m_i,pBy target user u^*Calculating the social times and social behavior categories of the users adjacent to the users;

repeating the above process for each user to obtain the adjacency weight between any two users, and h between users without social relationship_i,pm_i,p＝0；

Recording adjacency weights among all n users as a weighted social matrix A e R^n×nWherein h is_r,jm_r,je.A represents a social weight parameter between user r and user j.

In one embodiment of the invention, the pair of target users u^*The various social relationships are converted into weight parameters, specifically: recording target user u^*Generating neighbor users of social relations, wherein the users have social behaviors of paying attention, agreeing on, forwarding, commenting and the like, and recording the target user u^*Is [ u ] as a set of neighbor users^*]To [ u ]^*]In each neighbor user u_i1,…,ik∈[u^*]To obtain the target user u^*Embedding characterization u of_i' and Embedded characterization of these neighbor users (u)_i1′,u_i2′,…,u_ik') calculate user u^*And neighbor user set u^*]In each neighbor user u_ipCosine similarity of `:

in one embodiment of the invention, the metric target user u^*And the social popularity among the neighbor users thereof, specifically: remember h_i,p＝∑h(action_i,p)·mount_aciton,i,pH is a weighted value, and fusing the social times mount and the social behavior type action between the users, specifically h (action)_i,p) Representing a target user u^*And each of its neighbor users u_ip' weight of social action category action, mount_i,pRecording the times of corresponding social behavior action types, and calculating to obtain the target user u^*And all its neighbor users u^*]The adjacency weight between { h }_i,pm_i,p}。

In an embodiment of the present invention, the training of the attention-seeking neural network in step S4 includes:

calculating target user u^*And its neighbor user k_ik：e_ik＝ h_i,km_i,ka(Wu_i′,Wu_k') where a and W are parameters to be learned, W is a shared parameter matrix in the neural network, a is an attention calculation function, h_i, _km_i,kRepresenting a target user u^*And its neighbor user k, by pair e_ikPerforming softmax processing to obtain a target user u^*And an arbitrary user j_ij；

After training learning a and W, the attention coefficient alpha is calculated according to a and W_ijEmbedding tokens E for all users_uWeighted summation is carried out to obtain the target user u^*Fusion embedding characterization of

Wherein u is_j' go through E_u；

Training forward propagation network, inputting target user u^*Fusion embedding characterization of (h)_*And target user u^*Community attribute vector(s)₁,s₂,…,s_t) The community attribute vector is obtained by known information, t is the total number of the community attribute types, and when the target user u^*Has a certain community attribute of w hours s_wOtherwise, it is 0.

In one embodiment of the present invention, the actions are four categories, i.e., attention, comment, like, and forward, and the normalized weights are h (action is 0.4), h (action is 0.25), and h (action is 0.1).

According to another aspect of the present invention, there is also provided an attention-seeking neural network-based hidden community attribute acquisition system, including an embedded token vector generation module, a target user embedded token generation module, a weighted social matrix generation module, and a target user attribute classification module, where:

the embedded representation vector generation module is used for learning all words in the user social media data Word library through a Word vector model Word2vec network to obtain embedded representation vectors of all words;

the target user embedded representation generation module is used for carrying out normalized weighting on embedded representation vectors of user social media data vocabularies and obtaining embedded representations of target users based on a target embedded layer of a forward full-connection network;

the weight social matrix generating module is used for generating embedded representation of a neighbor user of a target user based on the user social network and social activity information, and calculating a weight social matrix according to the embedded representation of the neighbor user;

the target user attribute classification module is used for training a social popularity weighted attention-seeking neural network according to the weighted social matrix, and generating a hidden community attribute classification result of the target user by using the attention-seeking neural network and the embedded representation of the target user.

Generally, compared with the prior art, the technical scheme of the invention has the following beneficial effects:

(1) text features are extracted through the social media data and the social relations of the users, user social output contents and user community attribute features contained in network social interaction can be fully reserved, and compared with the traditional method of analyzing user personal network information which is difficult to distinguish true from false and an intricate social network, the method can more accurately and efficiently mine user attribute information;

(2) training an attention diagram neural network to construct a user hidden community attribute mining model, fusing social text information and interaction information, and describing the user hidden community attribute more comprehensively compared with a traditional method focusing on analyzing the user social network, so that the problem of data sparsity can be relieved;

(3) the social relationship weight between the users is calculated by combining the social popularity and the embedded representation of the neighbor users, the social behavior and the individual characteristics of the users are taken as the factors considering the social relationship weight, the limitation that only the existing social relationship is considered is broken through, and the social relationship attribute of the users is favorably mined as comprehensively as possible.

Drawings

Fig. 1 is a flowchart illustrating a hidden community attribute obtaining method based on an attention-seeking neural network according to the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

As shown in FIG. 1, the invention discloses a hidden community attribute acquisition method based on an attention-seeking neural network, which comprises the following steps:

specifically, the step S1 includes: the user social media data vocabulary formed by the social media data set containing all the users is known as

Wherein, c_i∈R^fOne-hot code representing the ith word in the vocabulary library, and the variable f ═ C | representing the number of different vocabularies in the vocabulary library. For C vocabulary, embedding a characterization vector through Word2vec network learning

Words in the vocabulary library are vectorized according to the text content set. Wherein each w_iThe word embedding representation of the ith word, and k is the vector dimension of each word after network learning.

specifically, the step S2 includes: obtaining the embedded characterization vectors E of all words_wThen, since the basic object for performing the hidden community attribute analysis is the user, the vocabulary embedded representation needs to be converted into the target user embedded representation, so that the subsequent attention-seeking neural network can perform analysis by taking the target user as the object. To this end, the vocabulary of all words is embedded into the representation E based on the target embedding layer of the forward full-connection network_wEmbedded representation converted into target user through full-connection form

Where n represents the number of target users, u_i′∈R^kEmbedding representing ith target userAnd (5) characterizing.

In particular, for target user u^*And target user u^*All words co-occurring in the same social media content form the user's characterization word set, denoted as { u^*}＝ {w_1′ w_2′ …w_f′Each one of w_i′Belong to E_wFor a target user u^*In other words, the number f' of the corresponding elements in the token vocabulary set does not exceed E_wThe size of (2). The location of these words and the text distance to the target user are recorded (in words, e.g., "Zhang three purchases computer", the distance between "buy" and "Zhang three" is 1, and the distance between "computer" and "Zhang three" is 2), according to { u }^*The distances between all vocabularies and target users in the text data are normalized and then are given different weights { p }₁,…p_f′Is formed into u^*′＝p₁w_1′+p₂w_2′+…p_f′w_f′As an embedded representation of the target user.

specifically, the step S3 includes: for target user u^*Into a weight parameter. First record the target user u^*Generating neighbor users of social relations, wherein the users have social behaviors of paying attention, agreeing on, forwarding, commenting and the like, and recording the target user u^*Is [ u ] as a set of neighbor users^*]To [ u ]^*]In each neighbor user u_i1,…,ik∈[u^*]To obtain the target user u^*Embedding characterization u of_i' and Embedded characterization of these neighbor users (u)_i1′,u_i2′,…,u_ik') calculate user u^*And neighbor user set u^*]In each neighbor user u_ipCosine similarity of `:

further measure target user u^*Social popularity among its neighbor users, social popularity coefficient h_i,pBy target user u^*And calculating the social times and social behavior categories of the neighbor users. Specifically, note h_i,p＝∑h(action_i,p)·mount_aciton,i,pH is a weighted value, and fusing the social times mount and the social behavior type action between the users, specifically h (action)_i,p) Representing a target user u^*And each of its neighbor users u_ip' weight of social action category action. The action is four categories of attention, comment, like and forward, and the normalized weight is h (action is 0.4), h (action is 0.25), h (action is 0.1). mounti_,pRecording the times of corresponding social behavior action types, and calculating to obtain the target user u^*And all its neighbor users u^*]The adjacency weight between { h }_i,pm_i,p}. The process is repeated for each user, so that the adjacency weight between any two users can be obtained, and h is between the users without social relationship_i,pm_i,p0. Recording adjacency weights among all n users as a weighted social matrix A e R^n×nWherein h is_r,jm_r,je.A represents a social weight parameter between user r and user j.

Specifically, the step S4 is implemented by the method of attempting a neural network including:

firstly, a target user u is calculated^*And its neighbor user k_ik：

e_ik＝h_i,km_i,ka(Wu_i′,Wu_k′)

Wherein the content of the first and second substances,a and W are parameters to be learned, W is a shared parameter matrix in the neural network, a is an attention calculation function, h_i,km_i,kRepresenting a target user u^*And social weight parameters between its neighbor users k.

By pair e_ikPerforming softmax processing to obtain a target user u^*And an arbitrary user j_ij。

Secondly, after learning a and W, training is performed based on the attention coefficient alpha calculated based on a and W_ijEmbedding tokens E for all users_uWeighted summation is carried out to obtain the target user u^*Fusion embedding characterization of

Wherein u is_j' go through E_u。

Finally, training the forward propagation network and inputting the target user u^*Fusion embedding characterization of (h)_*And target user u^*Community attribute vector(s)₁,s₂,…,s_t) The community attribute vector is derived from known information: t is the total number of community attribute categories, when the target user u^*Has a certain community attribute of w hours s_wOtherwise, it is 0. By trained target users u^*Fusion embedding characterization of (h)_*And target user u^*Community attribute vector(s)₁,s₂,…,s_t) The mapping between the social groups obtains the community attribute missing from any target user.

The process of the invention is illustrated below with reference to a specific embodiment:

(1) applying the embedded representation of Word2vec network learning words:

the vocabulary library formed by the social media data sets containing all users is known as C ═ C (C)₁,c₂,…,c_f)∈R^f×fWherein c is_i∈R^fOne-hot code representing the ith word in the vocabulary library, the variable f ═ C | representing the individual words in the vocabulary libraryAnd (4) counting. For the vocabulary in C, Word2vec network learning embedded representation is applied

Words in the vocabulary library are quantized according to the set of text content. Wherein each w_iWord2vec embedded representation of the ith Word.

Take the example of "Wu-Yi buys computer" and "Wang Yi Xiao Zan Wu", and the vocabulary library thereof is [ Wu-Yi, buy, computer, Wang Yi, Bai Zan)]And f is 5. Obtaining the embedded representation (w) of each Word of the media data by applying Word2vec network₁,w₂,w₃,w₄,w₅)。

(2) Obtaining a target user embedding representation based on a target embedding layer of a forward full-connection network:

deriving an embedded representation (w) of each word according to claim 2₁,w₂,w₃,w₄,w₅) And then, for each target user, Wu-Bi and Wang-Bi, respectively forming a representation vocabulary set of the user with all vocabularies which are co-occurring in the same event text, and keeping core information representing the hidden community attribute of the target user in the data as far as possible under the condition of eliminating redundant information to the maximum extent. For example "Wu-a" corresponds to { u₁}＝{w_1′w_2′w_3′Wu-jia, buy, computer, wang-jia corresponds to u-jia₂}＝{w_4′w_5′w_1′And { wang, bai zi, wu zi }. Wherein the text distance between 'purchase' and 'computer' from 'Wu-chi' is 1 and 2, the distance between 'Pengzai' and 'Wu-chi' from 'Wangzai' is 1 and 2, and different weights are given after reverse normalization, namely the target user of 'Wu-chi' is characterized as u₁′＝w_1′+0.66w_2′+0.33w_3′The target user of Wangzao is characterized as u₂′＝ w_4′+0.66w_5′+0.33w_1′. I.e. target user embedding representation E_u＝(u₁′,u₂′)。

(3) Generating a weighted social matrix based on the user social network and the social activity information:

various social relationships between the target users are translated into weight parameters. Firstly, recording neighbor users of each user generating social relations, wherein the users have social behaviors such as concern, praise, forward, comment and the like, and recording the neighbor user set of a target user 'Wu-somewhat' as u₁]To set of [ u ]₁]In each neighbor user u_i1,…,ik∈[u₁]To obtain their embedded characterization (u)_i1′,u_i2′,…,u_ik') obtain a target user Wu-somewhat and a neighbor user set [ u ]₁]Each user u in_ipCosine similarity of (c):

further taking the social times and the social behavior categories between the target user Wu-somewhat and the neighbor users as social heat to measure, and recording h_i,kΣ h (action) mount, where h is a weighted value, fusing the social times mount between users and the social behavior category action. The method of claim 4 wherein the target user "Wu-and-u" and all its neighbor users u are obtained according to the weight assignment_i1,…,ik∈[u₁]The adjacency weight between { h }_i,pm_i,p}. Repeating the above process can obtain the adjacency weight between any two users, and h between users without social relationship_i,km_i,k0. Recording the weights as the adjacency matrix A ∈ R^n×nWherein h is_i,jm_i,je.A represents a social weight parameter between user i and user j.

(4) Training a social popularity weighted attention-seeking neural network, and generating a hidden community attribute classification result of a target user:

the attention-seeking neural network is trained to complete the task of fusing textual content information with the target user's social information. Firstly, calculating a similarity coefficient e between a user i and a neighbor user k_ik：

e_ik＝h_i,km_i,ka(Wu_i′,Wu_k') where a and W are parameters that need to be learned, W is a shared parameter matrix in the neural network, and a is an attention calculation function. By pair e_ikPerforming softmax processing to obtain the attention coefficient alpha between the user i and any user j_ij. After training learning a and W, embedding a characterization E for a user according to an attention coefficient calculated based on a and W_uWeighted summation is carried out to obtain fusion embedding representation h of target user Wu-Chi_*。

Finally training the forward propagation network and inputting the target user u^*Fusion embedding characterization of (h)_*And target user u^*Community attribute vector(s)₁,s₂,…,s_t) T is the total number of community attribute categories, when the target user u^*Has a certain community attribute of w hours s_wOtherwise, it is 0. By trained target users u^*Fusion embedding characterization of (h)_*And target user u^*Community attribute vector(s)₁,s₂,…,s_t) The mapping between the two groups obtains certain community attributes of the target user 'wu-chi' missing.

Further, the invention also provides a hidden community attribute acquisition system based on an attention-seeking neural network, which comprises an embedded characterization vector generation module, a target user embedded characterization generation module, a weight social matrix generation module and a target user attribute classification module, wherein:

It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims

1. A hidden community attribute acquisition method based on an attention-seeking neural network is characterized by comprising the following steps:

2. The method for obtaining hidden community attributes based on attention-seeking neural network as claimed in claim 1, wherein the step S1 comprises:

the user social media data vocabulary formed by the social media data set containing all the users is known as

3. The method for obtaining hidden community attributes based on attention-seeking neural network as claimed in claim 1 or 2, wherein the step S2 comprises: obtaining the embedded characterization vectors E of all words_wThen, because the basic object for analyzing the hidden community attribute is the user, the vocabulary embedded representation needs to be converted into the target user embedded representation to enable the subsequent attention-seeking neural network to analyze by taking the target user as the object, and therefore, the target embedded layer based on the forward full-connection network embeds the vocabularies of all words into the representation E_wEmbedded representation converted into target user through full-connection form

4. The method for obtaining hidden community attributes based on attention-seeking neural network as claimed in claim 3, wherein the step S2 is specifically as follows: for target user u^*And target user u^*All words co-occurring in the same social media content form the user's characterization word set, denoted as { u^*}＝{w_1′ w_2′ … w_f′Therein ofEach w_i′Belong to E_wFor a target user u^*In other words, the number f' of the corresponding elements in the token vocabulary set does not exceed E_wThe position of these words and the distance of the text from the target user are recorded as { u }^*The distances between all vocabularies and target users in the text data are normalized and then are given different weights { p }₁,…p_f′Is formed into u^*′＝p₁w_1′+p₂w_2′+…p_f′w_f′As an embedded representation of the target user.

5. The method for obtaining hidden community attributes based on attention-seeking neural network as claimed in claim 1 or 2, wherein the step S3 comprises:

6. The method of claim 5, wherein the target user u is obtained from a hidden community attribute of the attention-driven neural network^*The various social relationships are converted into weight parameters, specifically:

recording target user u^*Generating neighbor users of social relations, wherein the users have social behaviors of paying attention, agreeing on, forwarding, commenting and the like, and recording the target user u^*Is a set of neighbor users[u^*]To [ u ]^*]In each neighbor user u_i1,…,ik∈[u^*]To obtain the target user u^*Embedding characterization u of_i' and Embedded characterization of these neighbor users (u)_i1′,u_i2′,…,u_ik') calculate user u^*And neighbor user set u^*]In each neighbor user u_ipCosine similarity of `:

7. the method of claim 6, wherein the metric target users u are selected from the group consisting of a group of users with a hidden social attribute, and a group of users with a hidden social attribute^*And the social popularity among the neighbor users thereof, specifically:

remember h_i,p＝∑h(action_i,p)·mount_aciton,i,pH is a weighted value, and fusing the social times mount and the social behavior type action between the users, specifically h (action)_i,p) Representing a target user u^*And each of its neighbor users u_ip' weight of social action category action, mount_i,pRecording the times of corresponding social behavior action types, and calculating to obtain the target user u^*And all its neighbor users u^*]The adjacency weight between { h }_i,pm_i,p}。

8. The method for obtaining hidden community attributes based on attention-seeking neural network as claimed in claim 1 or 2, wherein the training of the attention-seeking neural network in the step S4 comprises:

calculating target user u^*And its neighbor user k_ik：e_ik＝h_i,km_i,ka(Wu_i′,Wu_k') where a and W are parameters to be learned, W is a shared parameter matrix in the neural network, a is an attention calculation function, h_i,jm_i,kRepresenting a target user u^*And its neighbor user kSocial weighting parameter by pair e_ikPerforming softmax processing to obtain a target user u^*And an arbitrary user j_ij；

Wherein u is_j' go through E_u；

9. The hidden community attribute acquisition method based on attention-driven neural network as claimed in claim 8, wherein the actions are focus, comment, like-back and forward, respectively, and the normalized weights are h (action) 0.4, h (action) 0.25 and h (action) 0.1.

10. A hidden community attribute acquisition system based on an attention-seeking neural network is characterized by comprising an embedded token vector generation module, a target user embedded token generation module, a weight social matrix generation module and a target user attribute classification module, wherein: