CN115879515B - Document Network Topic Modeling Method, Variational Neighborhood Encoder, Terminal and Media - Google Patents

Abstract

The present invention provides a document network topic modeling method, a variational neighborhood encoder, a terminal and a medium. The method includes: obtaining a document network set, and respectively determining the document input representation of each document in the document network set; The document input representation of the document is input into the pre-trained variational neighborhood encoder for encoding processing to obtain the hidden layer representation of each document, and determine the representation of the central document according to the representation of the hidden layer; determine the representation of the central document according to the representation of the central document -topic distribution, and determine topic-term distribution based on said document-topic distribution. The present invention can effectively determine the representation of the central document based on the hidden layer representation of each document, can effectively determine the document-topic distribution based on the representation of the central document, and can effectively determine the topic-word distribution based on the document-topic distribution, so that To achieve the effect of topic modeling on the document network.

Description

Document network topic modeling method, variational neighborhood encoder, terminal and medium

Technical Field

The present invention relates to the technical field of topic modeling, and in particular to a document network topic modeling method, a variational neighborhood encoder, a terminal and a medium.

Background Art

A document network is a network composed of documents and their relationships, for example, a network composed of academic papers citing each other, a network composed of web page texts linking each other, etc. As an important part of text data, document networks can help people better understand the content distribution of documents by obtaining the topics of documents in document networks. Therefore, how to effectively perform topic modeling on documents in document networks is a problem that needs to be solved urgently.

Summary of the invention

The purpose of the embodiments of the present invention is to provide a document network topic modeling method, a variational neighborhood encoder, a terminal and a medium, aiming to solve the problem in the prior art of how to effectively perform topic modeling on documents in a document network.

The embodiment of the present invention is implemented as follows: a document network topic modeling method, the method comprising:

Obtaining a document network set, and determining a document input representation of each document in the document network set;

Inputting the document input representation of each document into the pre-trained variational neighborhood encoder for encoding processing to obtain a hidden layer representation, and determining the representation of the central document according to the hidden layer representation;

A document-topic distribution is determined based on the representation of the central document, and a topic-word distribution is determined based on the document-topic distribution.

Furthermore, the formula used to respectively determine the document input representation of each document in the document network set includes:

；

;

；

;

；

;

表示文档

与文档

之间最短路径的长度，

为文档

中单词

出现的次数，

是文本向量，

为0-1邻域向量，

是高阶邻域向量，

代表单词

在文档

中的权重，

表示中心文档。Among them, V represents the dictionary composed of words in the document set,

Representation Document

With Documentation

The length of the shortest path between

For Documentation

Chinese words

The number of occurrences,

is a text vector,

is a 0-1 neighborhood vector,

is a high-order neighborhood vector,

Representative words

In the documentation

The weight in

Represents the central document.

Furthermore, it is characterized in that before the document input representation of each document is input into the pre-trained variational neighborhood encoder for encoding processing, it also includes:

Obtaining a sample input representation of each sample document, and inputting the sample input representation of each sample document into the variational neighborhood encoder for encoding processing to obtain a sample inference distribution parameter;

Determining a sample topic representation according to the sample inference distribution parameter, and reconstructing each sample document according to the sample topic representation to obtain a reconstructed document;

Determine a priori loss according to the sample inference distribution parameter and the prior normal distribution parameter of each sample document, and determine reconstruction loss according to each sample document and the reconstructed document;

The variational neighborhood encoder is parameter updated according to the prior loss and the reconstruction loss until the variational neighborhood encoder converges, thereby obtaining the pre-trained variational neighborhood encoder.

Furthermore, the formula used for determining the prior loss according to the sample inference distribution parameters and the prior normal distribution parameters of each sample document and determining the reconstruction loss according to each sample document and the reconstructed document includes:

；

;

为各样本文档的邻域文档，

为由隐藏主题重新生成的邻域文档，

表示总损失，

表示重构损失，

表示先验损失，

表示权重参数，

为样本文档中的单词，

为重构样本中的单词，KL(·)表示所述样本推断分布参数和先验正态分布参数的KL散度，μ和σ分别为所述变分邻域编码器中推断网络推断出的推断分布的均值与方差，

和

为所述先验正态分布参数的均值与方差，

为正态分布。in,

is the neighborhood document of each sample document,

is the neighborhood document regenerated from the hidden topic,

represents the total loss,

represents the reconstruction loss,

represents the prior loss,

represents the weight parameter,

are the words in the sample documents,

To reconstruct the words in the sample, KL(·) represents the KL divergence between the sample inference distribution parameter and the prior normal distribution parameter, μ and σ are the mean and variance of the inference distribution inferred by the inference network in the variational neighborhood encoder, respectively.

and

are the mean and variance of the prior normal distribution parameters,

is a normal distribution.

Further, determining the representation of the central document according to the hidden layer representation includes:

Reparameterizing and processing the hidden layer representation with an attention mechanism to obtain a topic representation of the central document;

The dot product attention mechanism is used to aggregate the neighborhood documents of each document and the topic representation of the central document to obtain the representation of the central document.

Furthermore, the document input representation of each document is input into the pre-trained variational neighborhood encoder for encoding, and the formula used to obtain the hidden layer representation includes:

；

;

表示激活函数，

均表示所述变分邻域编码器中编码器对应全连接层的训练参数，

，

，

和

，

表示全体实数空间，t为主题个数，m为词典大小，

表示对数化方差，

表示中心文档的隐藏层表示，

表示从规模与μ和σ相同的多元标准正态分布中随机生成得到的样本表示。in,

represents the activation function,

Both represent the training parameters of the encoder corresponding to the fully connected layer in the variational neighborhood encoder,

,

,

and

,

represents the entire real number space, t is the number of topics, m is the dictionary size,

represents the logarithmic variance,

represents the hidden layer representation of the central document,

represents a sample representation randomly generated from a multivariate standard normal distribution of the same size as μ and σ .

Furthermore, the formula used to determine the document-topic distribution according to the representation of the central document includes:

；

;

表示中心文档d的邻域文档，

表示标准对数正态分布，weight表示邻域文档对中心文档的影响程度，

为中心文档d与邻域文档

之间的最短路径长度，

为中心文档与邻域文档的关联程度，

为中心文档隐藏层表示的转置，

为邻域文档的隐藏层表示，

为中心文档隐藏层表示与邻域文档隐藏层表示的注意力系数，θ为文档-主题分布，

为未归一化的中心文档主题表示，

表示归一化函数。Where N ( d ) represents the set of neighborhood documents that have paths with the central document d .

represents the neighborhood documents of the central document d ,

represents the standard log-normal distribution, weight represents the influence of the neighboring document on the central document,

is the central document d and the neighboring documents

The shortest path length between

is the correlation degree between the central document and the neighboring documents,

is the transpose of the hidden layer representation of the central document,

is the hidden layer representation of the neighborhood documents,

is the attention coefficient between the hidden layer representation of the central document and the hidden layer representation of the neighborhood document, θ is the document-topic distribution,

is the unnormalized central document topic representation,

Represents the normalization function.

Another object of an embodiment of the present invention is to provide a variational neighborhood encoder, wherein the variational neighborhood encoder is applied to any one of the document network topic modeling methods, and comprises:

The input layer is used to determine the document input representation of each document in the document network set respectively;

The encoding layer is used to encode the document input representation of each document to obtain the hidden layer representation of each document, and to re-parameterize and process the hidden layer representation with an attention mechanism to obtain the topic representation of the central document;

An attention layer, used to aggregate the neighborhood documents of each document and the topic representation of the central document using dot product attention to obtain the representation of the central document;

A decoder is configured to determine a document-topic distribution based on the representation of the central document, and determine a topic-word distribution based on the document-topic distribution.

Another object of an embodiment of the present invention is to provide a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

Another object of an embodiment of the present invention is to provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

The embodiments of the present invention can effectively encode each document by respectively determining the document input representation of each document in the document network set, and can effectively infer the hidden layer representation corresponding to each document by inputting the document input representation of each document into a pre-trained variational neighborhood encoder for encoding processing, and can effectively determine the representation of the central document based on the hidden layer representation, and can effectively determine the document-topic distribution based on the representation of the central document, and can effectively determine the topic-word distribution based on the document-topic distribution, so as to achieve the topic modeling effect of the document.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of a document network topic modeling method provided by a first embodiment of the present invention;

2 is a flow chart of a document network topic modeling method provided by a second embodiment of the present invention;

3 is a schematic diagram of the structure of a variational neighborhood encoder provided by a third embodiment of the present invention;

FIG. 4 is a schematic diagram of the structure of a terminal device provided in a fourth embodiment of the present invention.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

In order to illustrate the technical solution of the present invention, a specific embodiment is provided below for illustration.

Embodiment 1

Please refer to FIG. 1 , which is a flowchart of a document network topic modeling method provided by a first embodiment of the present invention. The document network topic modeling method can be applied to any terminal device or system. The document network topic modeling method includes the following steps:

Step S10, obtaining a document network set, and determining the document input representation of each document in the document network set respectively;

，由D中的词构成的词典记作V，其中包含m个单词，文档网络集D可表示为文档-词矩阵

。其中，

代表单词

在文档

中的权重(例如：TF-IDF)。Among them, given a document network set containing n documents

, the dictionary composed of words in D is denoted as V , which contains m words. The document network set D can be represented as a document-word matrix

.in,

Representative words

In the documentation

The weights in (e.g. TF-IDF).

，其中元素

为1表示

，

存在边，为0表示不存在边，文档网络G表示为

，其中D表示文档网络集，A表示D中文档的邻域矩阵，X表示文档-词矩阵，V表示D中的词构成的词典；The relationship between documents in D is represented by a 0-1 neighborhood matrix, that is,

, where the elements

1 means

,

There is an edge, and 0 means there is no edge. The document network G is represented as

, where D represents the document network set, A represents the neighborhood matrix of the documents in D , X represents the document-word matrix, and V represents the dictionary composed of the words in D ;

中元素的定义如公式(1)所示。其中，

表示文档

与文档

之间路径的长度；In this step, in order to model high-order graph structure information, the neighborhood matrix A should not only record the direct connection relationship between two documents, that is, the first-order neighborhood information, but also the second-order or even higher-order neighborhood information.

The definition of the elements in is shown in formula (1).

Representation Document

With Documentation

The length of the path between them;

；公式(1)

; Formula (1)

为文档

中单词

出现的次数：For the document-word matrix X , logarithmic regularization is used for initialization, as shown in formula (2).

For Documentation

Chinese words

Number of occurrences:

；公式(2)

; Formula (2)

结合，得到各文档的文档输入表示：Finally, as shown in formula (3), the text vector x , the neighborhood vector a or the high-order neighborhood vector

Combined, we get the document input representation of each document:

；公式(3)

; Formula (3)

Step S20, inputting the document input representation of each document into the pre-trained variational neighborhood encoder for encoding processing to obtain the hidden layer representation of each document, and determining the representation of the central document according to the hidden layer representation;

Optionally, in this step, determining the representation of the central document according to the hidden layer representation includes:

Reparameterizing and processing the hidden layer representation with an attention mechanism to obtain a topic representation of the central document;

Use the dot product attention mechanism to aggregate the neighborhood documents of each document and the topic representation of the central document to obtain the representation of the central document;

。The pre-trained variational neighbor encoder (VADJE) encodes the central document through a fully connected layer, infers the hidden layer representation of each document, and then obtains the hidden layer representation of the central document through reparameterization and attention mechanism.

.

This embodiment uses the normal distribution as the prior distribution, the variational neighborhood encoder uses the inverse tangent function as the activation function of the fully connected layer in the encoding stage, and uses Xavier Glorot to initialize the training parameters. The fully connected layer and reparameterization process in the encoding stage are shown in formula (4):

；公式(4)

; Formula (4)

表示激活函数，

均表示所述变分邻域编码器中编码器对应全连接层的训练参数，

，

，

和

，

表示全体实数空间，t为主题个数，m为词典大小，

表示对数化方差，

表示中心文档的隐藏层表示，

表示从规模与μ和σ相同的多元标准正态分布中随机生成得到的样本表示。重参数化的思想是通过从标准正态分布

中采样一个变量，再对该变量进行仿射变换得到所需隐变量，用于类VAE模型中反向传播的问题。in,

represents the activation function,

Both represent the training parameters of the encoder corresponding to the fully connected layer in the variational neighborhood encoder,

,

,

and

,

represents the entire real number space, t is the number of topics, m is the dictionary size,

represents the logarithmic variance,

represents the hidden layer representation of the central document,

represents the sample representation randomly generated from a multivariate standard normal distribution of the same size as μ and σ . The idea of reparameterization is to generate

A variable is sampled from the dataset, and then an affine transformation is performed on the variable to obtain the required latent variable, which is used for the back propagation problem in the VAE-like model.

Step S30, determining a document-topic distribution according to the representation of the central document, and determining a topic-word distribution according to the document-topic distribution;

与中心文档的主题表示

，得到未归一化的中心文档主题表示

，随后用softmax函数将

转换为文档的文档-主题分布θ，具体过程如公式(5)所示：Among them, after reparameterization, the variational neighborhood encoder uses dot product attention to aggregate neighborhood documents

Theme representation with central document

, get the unnormalized central document topic representation

, and then use the softmax function to

Converted to the document-topic distribution θ of the document, the specific process is shown in formula (5):

；公式(5)

; Formula (5)

表示中心文档d的邻域文档，

表示标准对数正态分布，weight表示邻域文档对中心文档的影响程度，

为中心文档d与邻域文档

之间的最短路径长度，

为中心文档与邻域文档的关联程度，

为中心文档隐藏层表示的转置，

为邻域文档的隐藏层表示，

为中心文档隐藏层表示与邻域文档隐藏层表示的注意力系数，θ为文档-主题分布，

为未归一化的中心文档主题表示，

表示归一化函数。Where d represents the central document, N ( d ) represents the set of neighborhood documents that have paths with the central document d ,

represents the neighborhood documents of the central document d ,

represents the standard log-normal distribution, weight represents the influence of the neighboring document on the central document,

is the central document d and the neighboring documents

The shortest path length between

is the correlation degree between the central document and the neighboring documents,

is the transpose of the hidden layer representation of the central document,

is the hidden layer representation of the neighborhood documents,

is the attention coefficient between the hidden layer representation of the central document and the hidden layer representation of the neighborhood document, θ is the document-topic distribution,

is the unnormalized central document topic representation,

Represents the normalization function.

In this step, in the decoding stage, the variational neighborhood encoder generates not only the central document itself but also its neighborhood documents using the hidden topics of the central document based on the document relations existing in the document network, as shown in formula (6):

；公式(6)

; Formula (6)

为激活函数，

和

为解码器中对应全连接层的可训练参数，

为由隐藏主题重新生成的邻域文档，将解码器中的权重与偏置参数进行softmax变化可以得到主题-词分布β。in,

is the activation function,

and

are the trainable parameters of the corresponding fully connected layer in the decoder,

For the neighborhood documents regenerated from the hidden topics, the topic-word distribution β can be obtained by performing softmax transformation on the weights and bias parameters in the decoder.

In this embodiment, the document network generation step based on the variational neighborhood encoder includes:

和

，

与

分别表示VADJE的均值推断网络和标准差推断网络；When generating a document, the corresponding distribution parameters are first obtained through the inference network in the variational neighborhood encoder

and

,

and

Represent the mean inference network and standard deviation inference network of VADJE respectively;

。对于给定文本，每个单词从对应的文本的词分布中生成，而文本的词分布可由文档的主题分布

与主题的词分布

得到，且为多项分布，即：Generating topic distribution of documents by reparameterization

For a given text, each word is generated from the word distribution of the corresponding text, and the word distribution of the text can be obtained from the topic distribution of the document

Word distribution with topic

We get , and it is a multinomial distribution, that is:

；

;

表示中心文档d的单词，

表示多项分布，文档连接生成时，将其建模为伯努利二元变量，根据文档的主题分布计算连接存在的概率，即

，其中

表示神经网络的全连接层，

表示伯努利分布。in,

represents the word of the central document d ,

Represents a multinomial distribution. When a document connection is generated, it is modeled as a Bernoulli binary variable, and the probability of the connection existing is calculated based on the topic distribution of the document, that is,

,in

represents the fully connected layer of the neural network,

represents a Bernoulli distribution.

；Specifically, for each document

;

：

；Generate a mean vector

:

;

：

；Generate log covariance

:

;

：

；Generate samples from a multivariate standard normal distribution

:

;

：

；Generating text topic distribution

:

;

；For each word

;

；Generate words

;

；For every pair of documents d and

;

。Generate a connection:

.

In this embodiment, by respectively determining the document input representation of each document in the document network set, each document can be effectively encoded based on the document input representation, and by inputting the document input representation of each document into the pre-trained variational neighborhood encoder for encoding processing, the hidden layer representation corresponding to each document can be effectively inferred, and the representation of the central document can be effectively determined based on the hidden layer representation, and the document-topic distribution can be effectively determined based on the representation of the central document, and the topic-word distribution can be effectively determined based on the document-topic distribution, so as to achieve the topic modeling effect of the document.

Embodiment 2

Please refer to FIG. 2 , which is a flow chart of a document network topic modeling method provided by a second embodiment of the present invention. This embodiment is used to further refine the steps before step S20 in the first embodiment, including the steps of:

Step S40, obtaining a sample input representation of each sample document, and inputting the sample input representation of each sample document into the variational neighborhood encoder for encoding processing to obtain a sample inference distribution parameter;

Wherein, based on formula (1) to formula (3), the sample input representation of each sample document is obtained respectively, and the sample input representation of each sample document is input into the variational neighborhood encoder for encoding processing to obtain the sample inference distribution parameter;

Step S50, determining a sample topic representation according to the sample inference distribution parameter, and reconstructing each sample document according to the sample topic representation to obtain a reconstructed document;

The sample topic representation is obtained by reparameterizing and processing the attention mechanism on the sample topic representation. After reparameterization, the variational neighborhood encoder uses dot product attention to aggregate the neighborhood documents and sample topic representations of each sample document to obtain the sample representation. Then, the sample representation is converted into a sample document-topic distribution using a softmax function. The sample topic-word distribution is determined based on the sample document-topic distribution. Based on the sample topic-word distribution, each sample document is reconstructed to obtain a reconstructed document.

Step S60, determining a priori loss according to the sample inference distribution parameter of each sample document and the prior normal distribution parameter, and determining reconstruction loss according to each sample document and the reconstructed document;

Among them, in the model training stage, for each document, the loss function of the variational neighborhood encoder is divided into two parts: reconstruction loss and prior loss: the reconstruction loss is the binomial cross entropy between the reconstructed document and the original document, and the prior loss is the KL divergence between the inferred distribution obtained by the inference network and the prior normal distribution, as shown in formula (7):

；公式(7)

; Formula (7)

为各样本文档的邻域文档（d也是自己的邻域文档之一），

为由隐藏主题重新生成的邻域文档，KL(·)表示所述样本推断分布参数和先验正态分布参数的KL散度，μ和σ为所述变分邻域编码器中推断网络推断出的推断分布的均值与方差，所述推断分布的形式为正态分布，参数包括μ和σ，

和

为所述先验正态参数的均值与方差，

为正态分布。in,

is the neighborhood document of each sample document ( d is also one of its own neighborhood documents),

is a neighborhood document regenerated from a hidden topic, KL(·) represents the KL divergence between the sample inference distribution parameter and the prior normal distribution parameter, μ and σ are the mean and variance of the inference distribution inferred by the inference network in the variational neighborhood encoder, the inference distribution is in the form of a normal distribution, and the parameters include μ and σ ,

and

are the mean and variance of the prior normal parameters,

is a normal distribution.

Step S70, updating parameters of the variational neighborhood encoder according to the prior loss and the reconstruction loss until the variational neighborhood encoder converges to obtain the pre-trained variational neighborhood encoder;

If the current iteration number of the variational neighborhood encoder is greater than or equal to the number threshold, the variational neighborhood encoder is determined to be converged, and the number threshold can be set according to requirements.

In this embodiment, by inputting the sample input representation of each sample document into the variational neighborhood encoder for encoding processing, the sample inference distribution parameters corresponding to each sample document can be effectively obtained, and the sample topic representation can be effectively determined based on the sample inference distribution parameters. Based on the sample topic representation, each sample document can be effectively reconstructed to obtain a reconstructed document. Based on the sample inference distribution parameters and the prior normal distribution parameters of each sample document, the prior loss of the variational neighboring encoder can be effectively determined. Based on each sample document and the reconstructed document, the reconstruction loss of the variational neighborhood encoder can be effectively determined. Based on the prior loss and the reconstruction loss, the variational neighborhood encoder parameters can be effectively updated, thereby improving the accuracy of the parameters in the variational neighborhood encoder and improving the accuracy of document network topic modeling.

Embodiment 3

Please refer to FIG3 , which is a schematic diagram of the structure of a variational neighborhood encoder provided by a third embodiment of the present invention, including:

The input layer is used to determine the document input representation of each document in the document network set respectively; wherein, for each document in the document network, the purpose of the input layer is to obtain the input representation corresponding to each document.

The encoding layer is used to encode the document input representation of each document to obtain the hidden layer representation of each document, and to re-parameterize and process the hidden layer representation with the attention mechanism to obtain the topic representation of the central document.

The encoding layer includes an encoder 10 and a reparameterization layer 11. The encoder is used to encode the central document through a fully connected layer to infer the hidden layer representation. The reparameterization layer 11 is used to obtain the topic representation of the central document through reparameterization and attention mechanism. Normal distribution is used as the prior distribution in the encoder 10.

The attention layer 12 is used to use dot product attention to aggregate the neighborhood documents of each document and the topic representation of the central document to obtain the representation of the central document; wherein, after reparameterization, the variational neighborhood encoder uses dot product attention to aggregate the topic representation of the neighborhood documents and the central document to obtain the representation of the central document.

The decoder 13 is used to determine the document-topic distribution according to the representation of the central document, and determine the topic-word distribution according to the document-topic distribution.

In this embodiment, by respectively determining the document input representation of each document in the document network set, each document can be effectively encoded based on the document input representation, and the hidden layer representation corresponding to each document can be effectively inferred by inputting the document input representation of each document into the pre-trained variational neighborhood encoder for encoding processing. Based on the hidden layer representation, the representation of the central document can be effectively determined, based on the representation of the central document, the document-topic distribution can be effectively determined, and based on the document-topic distribution, the topic-word distribution can be effectively determined, so as to achieve the topic modeling effect of the document.

Embodiment 4

FIG4 is a block diagram of a terminal device 2 provided in the fourth embodiment of the present application. As shown in FIG4, the terminal device 2 of this embodiment includes: a processor 20, a memory 21, and a computer program 22 stored in the memory 21 and executable on the processor 20, such as a program of a document network topic modeling method. When the processor 20 executes the computer program 22, the steps in each embodiment of the above-mentioned document network topic modeling method are implemented.

Exemplarily, the computer program 22 may be divided into one or more modules, which are stored in the memory 21 and executed by the processor 20 to complete the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 22 in the terminal device 2. The terminal device may include, but is not limited to, a processor 20 and a memory 21.

The processor 20 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

The memory 21 may be an internal storage unit of the terminal device 2, such as a hard disk or memory of the terminal device 2. The memory 21 may also be an external storage device of the terminal device 2, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 2. Further, the memory 21 may also include both an internal storage unit and an external storage device of the terminal device 2. The memory 21 is used to store the computer program and other programs and data required by the terminal device. The memory 21 may also be used to temporarily store data that has been output or is to be output.

In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units.

If the integrated module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Among them, the computer-readable storage medium can be non-volatile or volatile. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims (7)

1. A document network topic modeling method, characterized in that the method comprises the following steps: Obtaining a document network set, and determining a document input representation of each document in the document network set; Inputting the document input representation of each document into the pre-trained variational neighborhood encoder for encoding processing to obtain a hidden layer representation of each document, and determining the representation of the central document according to the hidden layer representation; Determining a document-topic distribution based on the representation of the central document, and determining a topic-word distribution based on the document-topic distribution; The formula used to respectively determine the document input representation of each document in the document network set includes:

;

;

; Among them, V represents the dictionary composed of words in the document set,

Representation Document

With Documentation

The length of the shortest path between

For Documentation

Chinese words

The number of occurrences,

is a text vector,

is a 0-1 neighborhood vector,

is a high-order neighborhood vector,

Representative words

In the documentation

The weight in

Represents the central document; Before inputting the document input representation of each document into the pre-trained variational neighborhood encoder for encoding, the method further includes: Obtaining a sample input representation of each sample document, and inputting the sample input representation of each sample document into the variational neighborhood encoder for encoding processing to obtain a sample inference distribution parameter; Determining a sample topic representation according to the sample inference distribution parameter, and reconstructing each sample document according to the sample topic representation to obtain a reconstructed document; Determine a priori loss according to the sample inference distribution parameter and the prior normal distribution parameter of each sample document, and determine reconstruction loss according to each sample document and the reconstructed document; updating parameters of the variational neighborhood encoder according to the prior loss and the reconstruction loss until the variational neighborhood encoder converges to obtain the pre-trained variational neighborhood encoder; The formula used for determining the prior loss according to the sample inference distribution parameters and the prior normal distribution parameters of each sample document, and determining the reconstruction loss according to each sample document and the reconstructed document includes:

; in,

is the neighborhood document of each sample document,

is the neighborhood document regenerated from the hidden topic,

represents the total loss,

represents the reconstruction loss,

represents the prior loss,

represents the weight parameter,

are the words in the sample documents,

To reconstruct the words in the sample, KL(·) represents the KL divergence between the sample inference distribution parameter and the prior normal distribution parameter,

and

are respectively the mean and variance of the inferred distribution inferred by the inference network in the variational neighborhood encoder,

and

are the mean and variance of the prior normal distribution parameters,

is a normal distribution. 2. The document network topic modeling method according to claim 1, wherein determining the representation of the central document according to the hidden layer representation comprises: Reparameterizing and processing the hidden layer representation with an attention mechanism to obtain a topic representation of the central document; The dot product attention mechanism is used to aggregate the neighborhood documents of each document and the topic representation of the central document to obtain the representation of the central document. 3. The document network topic modeling method according to claim 2, characterized in that the document input representation of each document is input into the pre-trained variational neighborhood encoder for encoding processing, and the formula used to obtain the hidden layer representation of each document includes:

; in,

represents the activation function,

Both represent the training parameters of the encoder corresponding to the fully connected layer in the variational neighborhood encoder,

,

,

and

,

represents the space of all real numbers,

is the number of topics,

is the dictionary size,

represents the logarithmic variance,

represents the hidden layer representation of the central document,

represents a sample representation randomly generated from a multivariate standard normal distribution of the same size as μ and σ . 4. The document network topic modeling method according to claim 3, characterized in that the formula used to determine the document-topic distribution based on the representation of the central document includes:

; in,

Representation and central documentation

A collection of neighboring documents with paths between them,

Representation Center Document

Neighborhood documents,

represents the standard lognormal distribution,

Indicates the degree of influence of the neighborhood document on the central document,

Central Document

With Neighborhood Documents

The shortest path length between

is the correlation degree between the central document and the neighboring documents,

is the transpose of the hidden layer representation of the central document,

is the hidden layer representation of the neighborhood documents,

is the attention coefficient between the hidden layer representation of the central document and the hidden layer representation of the neighboring document,

is the document-topic distribution,

is the unnormalized central document topic representation,

Represents the normalization function. 5. A variational neighborhood encoder, characterized in that it is applied to the document network topic modeling method according to any one of claims 1 to 4, and the variational neighborhood encoder comprises: The input layer is used to determine the document input representation of each document in the document network set respectively; The encoding layer is used to encode the document input representation of each document to obtain the hidden layer representation of each document, and to re-parameterize and process the hidden layer representation with an attention mechanism to obtain the topic representation of the central document; An attention layer, used to aggregate the neighborhood documents of each document and the topic representation of the central document using dot product attention to obtain the representation of the central document; A decoder is configured to determine a document-topic distribution based on the representation of the central document, and determine a topic-word distribution based on the document-topic distribution. 6. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 4 when executing the computer program. 7. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 4.

Images