CN114492669B

CN114492669B - Keyword recommendation model training method, recommendation device, equipment and medium

Info

Publication number: CN114492669B
Application number: CN202210143648.9A
Authority: CN
Inventors: 刘羲; 舒畅; 陈又新
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2022-02-16
Filing date: 2022-02-16
Publication date: 2024-06-28
Anticipated expiration: 2042-02-16
Also published as: WO2023155304A1; CN114492669A

Abstract

The embodiment of the disclosure provides a keyword recommendation model training method, a keyword recommendation model training device, keyword recommendation model training equipment and keyword recommendation model medium, and relates to the technical field of artificial intelligence. The keyword recommendation model training method comprises the following steps: obtaining an index sample, constructing a keyword training data set according to the index sample, obtaining word vectors of words according to the index sample, obtaining primary environment word vectors according to the word vectors, obtaining input environment word vectors according to the primary environment word vectors, generating keyword vectors according to the input environment word vectors and the word vectors, inputting the keyword vectors into a prediction classification layer to obtain a recommended prediction value, and adjusting parameters according to detection errors between the recommended prediction value and classification labels to obtain a keyword recommendation model. According to the method, the device and the system, the input environment word vectors related to the word vectors are obtained, and the learning difficulty of the keywords and the environment information corresponding to the keywords are combined, so that the selected keywords meet the learning requirements of users, and the relevance and the accuracy of keyword recommendation are improved.

Description

Keyword recommendation model training method, recommendation device, equipment and medium

Technical Field

The invention relates to the technical field of artificial intelligence, in particular to a keyword recommendation model training method, a keyword recommendation device, keyword recommendation equipment and keyword recommendation media.

Background

Along with the wider application of foreign languages in life and work of people, language learning is fast, convenient and efficient, and continuous pursuit of people is accompanied by development of artificial intelligence, and research results for improving language learning efficiency by using artificial intelligence are increasingly presented. For example, for English learning, word learning is the basis, after a user reads an English text, an artificial intelligence system is utilized to automatically extract key words (i.e. keywords) of the text for the user to learn, and according to English reading history information research on a user group, words most likely to learn by 20 users are provided, and recommendation ordering is performed according to learning probability from large to small.

In the related art, there are two solutions, the first is to obtain a recommendation model for an unsupervised learning scheme, and by counting the word frequencies of all the texts, the inverse text frequency IDF (inverse document frequency) of each word is obtained, the difficulty level of the words is represented by IDFs, the IDFs are recommended to the user after being sequenced from high to low, although the recommendation model obtained by the scheme is relatively convenient to calculate, the IDFs can reflect the difficulty trend of the words, but words with higher IDFs are often very rare words and are not suitable for general user learning. The other is to obtain a recommendation model for a supervised learning scheme, such as a sequence labeling scheme, and input all words of a text, and try to find the relation degree between each word and other words through a long-short-term memory artificial neural network or an attention mechanism network structure so as to find the word which the user wants to learn most. When the recommendation model obtained by the scheme processes long texts, the parameter quantity is large, and the influence of too many irrelevant words is introduced, so that the recommendation effect of keywords is greatly reduced. For example, the single word is directly predicted, the relation among the words is not considered, the scheme is simple and quick, but the word most suitable for learning cannot be accurately found due to the lack of interaction with other words.

Disclosure of Invention

The main purpose of the disclosed embodiments is to provide a keyword recommendation model training method, a recommendation device, a device and a medium, wherein the keyword recommendation model obtained by training the keyword recommendation model training method can realize automatic extraction of keywords in a text, and the relevance and accuracy of keyword recommendation are improved.

To achieve the above object, a first aspect of an embodiment of the present disclosure provides a keyword recommendation model training method, including:

Acquiring at least one text sample, and generating an index sample containing index information of words according to the text sample;

Constructing a keyword training data set according to the index sample, wherein the keyword training data set comprises the index sample and a classification label;

Inputting the index sample into a word embedding processing layer of the keyword recommendation model to perform word embedding operation to obtain word vectors of the words;

inputting the word vector into a primary environment information extraction layer of the keyword recommendation model to extract primary environment information, so as to obtain a primary environment word vector;

Performing first splicing processing on the primary environment word vector and the word vector by using an input environment information extraction layer of the keyword recommendation model to generate the input environment word vector corresponding to the word;

performing second splicing processing according to the input environment word vector and the word vector to obtain a keyword vector;

inputting the keyword vector into a prediction classification layer of the keyword recommendation model to obtain a recommendation predicted value;

And adjusting parameters in the keyword recommendation model according to the detection error between the recommendation predicted value and the classification label until a loss function meets a convergence condition, so as to obtain the keyword recommendation model, wherein the keyword recommendation model is used for performing keyword recommendation processing.

In some embodiments, the step of inputting the word vector into the primary environmental information extraction layer of the keyword recommendation model to extract primary environmental information, and obtaining a primary environmental word vector includes:

Acquiring preselected words of the word vectors according to the index information by using the primary environment information extraction layer, wherein the preselected words are a first number of words adjacent to the words corresponding to the word vectors, and the word vectors corresponding to the preselected words are preselected word vectors;

acquiring the reverse text frequency of the pre-selected word;

and calculating a primary environment word vector corresponding to the word vector according to the pre-selected word vector and the inverse text frequency.

In some embodiments, the performing a second stitching process according to the input environmental word vector and the word vector to obtain a keyword vector includes:

acquiring the inverse text frequency of the word vector;

And performing second splicing processing on the input environment word vector, the word vector and the inverse text frequency of the word vector to generate the keyword vector.

In some embodiments, the obtaining, by the primary context information extraction layer, the pre-selected word of the word vector according to the index information includes:

copying the words of the index sample to obtain a second number of copied words, wherein the second number is calculated by the first number;

performing displacement operation on the second number of copied words column by column to obtain a displacement word matrix;

Acquiring a list word of the displacement word matrix;

acquiring the list word corresponding to the word vector as the pre-selected word according to the index information

In some embodiments, the constructing a keyword training dataset from the index sample comprises:

Acquiring at least one user keyword sample and a corresponding classification label, wherein the classification label is a target label or a non-target label

Calculating the probability that each classified label is a target label to obtain label probability distribution;

Combining the user keyword samples according to the tag probability distribution to obtain the index samples corresponding to the text samples;

and constructing a keyword training data set according to the index sample.

Acquiring a text mask;

performing length filling processing on the index sample according to the text mask to obtain the keyword training data set; wherein the text lengths of the index samples in the keyword training dataset are consistent.

In order to achieve the above object, a third aspect of the present disclosure provides a keyword recommendation method, including:

Acquiring a text to be recommended;

generating an index text according to the text to be recommended;

And inputting the index text into a keyword recommendation model to perform keyword recommendation processing to obtain target keywords, wherein the keyword recommendation model is trained by using the keyword recommendation model training method according to any one of the first aspect.

To achieve the above object, a third aspect of the present disclosure provides a keyword recommendation model training apparatus, including:

the text sample acquisition module is used for acquiring at least one text sample and generating an index sample containing index information of words according to the text sample;

the training data set construction module is used for constructing a keyword training data set according to the index sample, wherein the keyword training data set comprises the index sample and a classification label;

The word vector processing module is used for inputting the index sample into a word embedding processing layer of the keyword recommendation model to perform word embedding operation to obtain a word vector of the word;

The primary environment word vector calculation module is used for inputting the word vector into a primary environment information extraction layer of the keyword recommendation model to extract primary environment information so as to obtain a primary environment word vector;

The input environment word vector calculation module is used for carrying out first splicing processing on the primary environment word vector and the word vector by utilizing an input environment information extraction layer of the keyword recommendation model to generate the input environment word vector corresponding to the word;

The keyword vector generation module is used for carrying out second splicing processing according to the input environment word vector and the word vector to obtain a keyword vector;

the classification prediction module is used for inputting the keyword vector into a prediction classification layer of the keyword recommendation model to obtain a recommendation prediction value;

And the parameter adjustment module is used for adjusting parameters in the keyword recommendation model according to the detection error between the recommendation predicted value and the classification label until the loss function meets the convergence condition, so as to obtain the keyword recommendation model, wherein the keyword recommendation model is used for performing keyword recommendation processing.

To achieve the above object, a fourth aspect of the present disclosure proposes an electronic device including:

At least one memory;

At least one processor;

At least one program;

the program is stored in a memory and the processor executes the at least one program to implement the method of the present disclosure as set forth in the first or second aspect above.

To achieve the above object, a fifth aspect of the present disclosure proposes a storage medium that is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute:

the method as described in the first or second aspect above.

According to the keyword recommendation model training method, index samples are obtained, a keyword training data set is constructed according to the index samples, words of the keyword recommendation model are input into a word embedding processing layer through the index samples, word vectors of the words are obtained, the word vectors are input into a primary environment information extraction layer, primary environment word vectors are obtained, then the primary environment word vectors are input into the input environment information extraction layer, input environment word vectors are obtained, keyword vectors are generated according to the input environment word vectors and the word vectors, the keyword vectors are input into a prediction classification layer, a recommendation prediction value is obtained, parameters in the keyword recommendation model are adjusted according to detection errors between the recommendation prediction value and classification labels, and the keyword recommendation model is obtained. According to the keyword recommendation model training method, the keyword recommendation model is trained to obtain the input environment word vectors related to the word vectors, and the selected keywords are enabled to meet the learning requirements of users by combining the learning difficulty of the keywords and the environment information corresponding to the keywords, so that the relevance and the accuracy of keyword recommendation are improved.

Drawings

Fig. 1 is a flowchart of a keyword recommendation model training method provided in an embodiment of the present disclosure.

Fig. 2 is a further flowchart of a keyword recommendation model training method provided by an embodiment of the present disclosure.

Fig. 3 is a schematic diagram of a method for selecting a pre-selected word of a keyword recommendation model training method according to an embodiment of the present disclosure.

Fig. 4 is a further flowchart of a keyword recommendation model training method provided by an embodiment of the present disclosure.

Fig. 5 is a schematic diagram of acquiring a pre-selected word according to a displacement structure according to a keyword recommendation model training method provided in an embodiment of the present disclosure.

Fig. 6 is a schematic diagram of a keyword recommendation model structure of a keyword recommendation model training method according to an embodiment of the present disclosure.

Fig. 7 is a schematic diagram of a prediction flow of a keyword recommendation model training method according to an embodiment of the disclosure.

Fig. 8 is a further flowchart of a keyword recommendation model training method provided by an embodiment of the present disclosure.

Fig. 9 is a flowchart of a keyword recommendation method provided in an embodiment of the present disclosure.

Fig. 10 is a block diagram of a keyword recommendation model training apparatus provided in an embodiment of the present disclosure.

Fig. 11 is a block diagram of a keyword recommendation apparatus provided in an embodiment of the present disclosure.

Fig. 12 is a schematic hardware structure of an electronic device according to an embodiment of the disclosure.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

It should be noted that although functional block division is performed in a device diagram and a logic sequence is shown in a flowchart, in some cases, the steps shown or described may be performed in a different order than the block division in the device, or in the flowchart.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to be limiting of the invention.

First, several nouns involved in the present application are parsed:

Inverse text frequency IDF (inverse document frequency): namely TF-IDF (Term Frequency-Inverse Document Frequency), is a word segmentation algorithm, and consists of two parts: TF (Term Frequency), characteristic Term Frequency, i.e., word Frequency, and IDF (Inverse Document Frequency, inverse text Frequency). The term frequency TF refers to the number of times that feature words appear in the selected text, which means that when the term frequency is calculated, word combinations in the text need to be divided, and the number of words is counted after the division. IDF refers to a measure of the general importance of a feature word. The inverse text frequency of the feature words is estimated by counting the degree to which the feature words appear in the established corpus. The IDF can effectively reduce the weight of the high-frequency characteristic words with smaller effect, so that the influence on text classification is weakened, meanwhile, the characteristic words with lower word frequency and larger effect are evaluated, and the larger weight is given, so that the accuracy of text classification is improved.

In the related art, two solutions exist, namely, a recommendation model is obtained for an unsupervised learning scheme, the word frequency of all texts is counted to obtain IDFs of each word, the IDFs are used for representing the difficulty level of the words, the IDFs are recommended to a user after being sequenced from high to low, although the recommendation model obtained by the scheme is convenient to calculate, the IDFs can reflect the difficulty trend of the words, but words with higher IDFs are often very rare words and are not suitable for general user learning. The other is to obtain a recommendation model for a supervised learning scheme, such as a sequence labeling scheme, and input all words of a text, and try to find the relation degree between each word and other words through a long-short-term memory artificial neural network or an attention mechanism network structure so as to find the word which the user wants to learn most. When the recommendation model obtained by the scheme processes long texts, the parameter quantity is large, and the influence of too many irrelevant words is introduced, so that the recommendation effect of keywords is greatly reduced. For example, the single word is directly predicted, the relation among the words is not considered, the scheme is simple and quick, but the word most suitable for learning cannot be accurately found due to the lack of interaction with other words.

Based on the above, the embodiment of the disclosure provides a keyword recommendation model training method, a recommendation method and device, electronic equipment and a storage medium, which can realize automatic extraction of keywords in a text and promote relevance and accuracy of keyword recommendation.

The embodiment of the disclosure provides a keyword recommendation model training method, a recommendation method and a device, electronic equipment and a storage medium, and specifically, the following embodiment is used for explaining, firstly, the keyword recommendation model training method and the recommendation method in the embodiment of the disclosure.

The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Wherein artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) is the theory, method, technique, and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend, and expand human intelligence, sense the environment, acquire knowledge, and use knowledge to obtain optimal results.

Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

The embodiment of the disclosure provides a keyword recommendation model training method and a recommendation method, relates to the technical field of artificial intelligence, and particularly relates to the technical field of artificial intelligence. The keyword recommendation model training method and the keyword recommendation model training method provided by the embodiment of the disclosure can be applied to a terminal, a server side and software running in the terminal or the server side. In some embodiments, the terminal may be a smart phone, tablet, notebook, desktop, or smart watch, etc.; the server can be an independent server, and can also be a cloud server for providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDNs), basic cloud computing services such as big data and artificial intelligent platforms, and the like; the software may be an application that implements a keyword recommendation model training method, a recommendation method, or the like, but is not limited to the above form.

The application is operational with numerous general purpose or special purpose computer system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

Fig. 1 is an optional flowchart of a keyword recommendation model training method provided in an embodiment of the present disclosure, where the method in fig. 1 may include, but is not limited to, steps S101 to S108.

Step S101, at least one text sample is acquired, and an index sample containing index information of words is generated according to the text sample.

In an embodiment, taking english learning as an example, a large number of text samples are required for training a keyword recommendation model, where the text samples are english articles, and the english articles are made up of words, so that index information of each word is generated according to the english articles, and all the words in the english articles can be located according to the index information. For example, a word database may be generated and then the text sample may be converted to an index sample in which the word is represented by its position in the database, improving the processing efficiency of the sample.

Step S102, a keyword training data set is constructed according to the index sample, wherein the keyword training data set comprises the index sample and the classification label.

In an embodiment, a keyword training data set is formed by using a large number of index samples, and the training purpose is to enable a trained keyword recommendation model to output recommendation probability of all words in a text to be recommended as keywords after an index text corresponding to the text to be recommended is input. Based on this training goal, the keyword training data set constructed in this embodiment includes: an index sample and a class label, wherein the class label is used for indicating the probability that each word in the index sample is used as a keyword, for example, a "01" indicates that the word is not selected as the keyword, and a "1" indicates that the word is selected as the keyword.

Step S103, inputting the index sample into a word embedding processing layer of the keyword recommendation model to perform word embedding operation, and obtaining word vectors of words.

In one embodiment, when the keyword recommendation model is trained, the index sample is input to a word embedding processing layer in the keyword recommendation model, and word embedding operation is performed on all words in the index sample to obtain word vectors of the corresponding words. The word embedding operation is a vectorized representation of words, with individual words represented in a predefined vector space as real vectors, each word mapped to a word vector.

In one embodiment, for example, a text contains words such as "cat", "dog", "love", etc., and these words are mapped to a vector space, where the word vector corresponding to cat is (0.1,0.2,0.3), the word vector corresponding to dog is (0.2,0.2,0.4), the word vector corresponding to love is (-0.4, -0.5, -0.2), etc., and the word vector representations are merely illustrative. In this embodiment, the word vector is obtained by vectorizing the word, so that the efficiency and accuracy of word processing by the computer can be improved, for example, "cat" and "dog" represent animals, and "love" represents an emotion, but for the computer, all three words are represented by 0,1 as binary character strings, and cannot be directly calculated. Thus, the word is converted into the word vector by the word embedding method, and the machine computer can directly calculate the word. The word embedding operation in this embodiment, which is the process of mapping words into vector representations, implements word embedding operations on all words in the index sample by using the word embedding processing layer in the keyword recommendation model.

Step S104, inputting the word vector into a primary environment information extraction layer of the keyword recommendation model to extract primary environment information, and obtaining a primary environment word vector.

When the user selects a word in an article for learning in the actual language learning process, the difficulty of the word is considered by light filling, and the word is selected according to the related word in the current visual receptive field, namely, the user compares the seen word with a plurality of words around the word to determine whether to select the word for learning.

For example: the difficulty of the word A is not high, but the difficulty of a plurality of words around the word A is low, and the word A can be selected by a user with high probability; on the contrary, the difficulty of the word B is higher, but the difficulty of the words around the word is also higher, so that the user can not select the word with high probability. Therefore, in an embodiment, a word vector is input into a primary environmental information extraction layer of a keyword recommendation model through a visual receptive field range of a user to obtain a primary environmental word vector, and the primary environmental word vector is used for reflecting surrounding word information of the word (i.e. word vector) and is used as a primary environmental influence factor to determine whether the word can be selected as a learning word, and the learning difficulty of the keyword and the environmental information corresponding to the keyword are combined, so that the selected keyword meets the learning requirement of the user, and the relevance and accuracy of keyword recommendation are improved.

In one embodiment, the primary environmental information extraction layer of the keyword recommendation model inputs the word vector to perform primary environmental information extraction to obtain a primary environmental word vector, and referring to fig. 2, the method includes, but is not limited to, steps S1041 to S1043:

in step S1041, a pre-selected word of the word vector is acquired according to the index information by using the primary environmental information extraction layer.

In an embodiment, the pre-selected word, i.e. the word surrounding the word, may be a first number of words adjacent to the word corresponding to the word vector, the word surrounding may be a front-back position or a front-back upper-lower position of the word, i.e. a front-back sliding window is set for a first number, the words of the first number around the word position are taken as the pre-selected word, or the words of the first number around the word position are taken as the radius, the words of the front-back upper-lower position are taken as the pre-selected word, and the selection mode of the pre-selected word is not specifically limited, so that the word within the range of the word visual perception field is mainly selected as a primary environmental influence factor. Wherein, the word vector corresponding to the preselected word is the preselected word vector.

Referring to FIG. 3, a method for selecting a pre-selected word in accordance with one embodiment of the present application is shown.

The first number is 5 in the figure, and the selection mode is exemplified as the front and back positions, namely, the word is taken as the center, and each word of 5 left and right words is selected as a preselected word. For example, in the figure, the words "shot" are the first five words "Surgeon", "Genaral", "defends", "US" and "boost", respectively, and the last five words "plan", "as", "much", "of" and "the", respectively, then "Surgeon", "Genaral", "defends", "US", "boost", "plan", "as", "much", "of" and "the" are pre-selected words of the word "shot".

In an embodiment of the present application, since each word in the index sample needs to be selected as a corresponding pre-selected word, for example, when the first number is 5, each word is selected with 5 words as a center, and 5 words as a radius, for example, the length of a peace army word of a text is 1000 words, 1000 times of processing are needed, and when the number of the text is more, or the text is longer, the calculation time complexity is higher, and the calculation performance is lower, in order to further improve the efficiency of selecting the pre-selected word, the embodiment provides a method for acquiring the pre-selected word according to a displacement structure, which includes, but is not limited to, steps S401 to S404:

Step S401, the words of the index sample are copied to obtain a second number of copied words.

In an embodiment, the second number is calculated from the first number, e.g. the second number=the first number×2+1. Referring to fig. 5, in an embodiment of the present application, a schematic diagram of pre-selected words is obtained according to a displacement structure, in which words are represented by numbers and words are copied, and when the first number is 5, the second number is 11, that is, all words of text are copied 11 times.

Step S402, performing displacement operation on the second number of duplicated words column by column to obtain a displacement word matrix.

In an embodiment, referring to fig. 5, the copied words obtained by copying are shifted column by column, that is, the first word of the next text is aligned with the previous word corresponding to the word in the previous text, and so on, to obtain a shifted word matrix after column by column shifting, where if the word length of the text is 1000, that is, each text may be represented as a row vector of 1×1000, the length of the obtained shifted word matrix is 11×1000.

Step S403, a column word of the displacement word matrix is acquired.

In one embodiment, the displacement word matrix is divided according to columns to obtain column words corresponding to each column.

Step S404, obtaining a list word corresponding to the list of the word vector according to the index information as a preselected word.

Referring to fig. 5, since the first number of values is 5, the word of the 6 th line is selected, the value is taken by columns, the column word corresponding to each word is the pre-selected word of the word, for example, "1", "2", "3", "4" and "5" and "7", "8", "9", "10" and "11", the "6" of the 6 th line is in the corresponding column in the shift word matrix, that is, the pre-selected word of "6", and so on. According to the method, the time complexity can be obviously reduced, and only the preselected word needs to be copied by a second number for each word.

In step S1042, the inverse text frequency of the preselected word is obtained.

In one embodiment, for an IDF of a word W, the number of total text in the corpus (e.g., the keyword training dataset) divided by the number of text containing the word is obtained, and the obtained quotient is obtained by taking the logarithm, and if the text containing the word W is fewer, the larger the IDF, the better the class distinction capability of the word W is.

In this embodiment, the IDF of the preselected word is calculated, expressed as:

IDF_w＝log1+M

Wherein IDF _w represents the inverse text frequency of word W, |D| represents the total number of text in the keyword training dataset, M represents the number of text containing word W, and the purpose of adding 1 to the denominator is to avoid the situation that the denominator is 0 because the word is not in the corpus.

Step S1043, calculating a primary environment word vector corresponding to the word vector according to the pre-selected word vector and the inverse text frequency.

In one embodiment, the pre-selected word vector may be multiplied by the corresponding inverse text frequency and then weighted and summed to obtain the primary ambient word vector corresponding to the word vector.

Step S105, performing first splicing processing on the primary environment word vector and the word vector by using an input environment information extraction layer of the keyword recommendation model, and generating an input environment word vector corresponding to the word.

In one embodiment, to further extract the context information, an input context word vector is further extracted for the primary context word vector. For example, the input environment information extraction layer is used for performing first splicing processing on the primary environment word vector and the word vector to generate an input environment word vector corresponding to the word.

And S106, performing second splicing processing according to the input environment word vector and the word vector to obtain a keyword vector.

In one embodiment, in order to introduce word frequency information of a word vector, firstly, an inverse text frequency of the word vector is obtained, and then, an input environment word vector, the word vector and an IDF corresponding to the word vector are subjected to second splicing processing to generate a keyword vector for subsequent classification prediction.

Step S107, inputting the keyword vector into a prediction classification layer of the keyword recommendation model to obtain a recommendation predicted value.

And S108, adjusting parameters in the keyword recommendation model according to the detection error between the recommendation predicted value and the classification label until the loss function meets the convergence condition, so as to obtain the keyword recommendation model.

In one embodiment, the keyword vector is input into a prediction classification layer of the keyword recommendation model to obtain a recommended predicted value y ', and the recommended predicted value y' is compared with a classification label (i.e., a true value y) corresponding to the word to obtain a detection error between the recommended predicted value and the classification label. Wherein the recommended predicted value y 'and the actual value y may be represented by a number sequence, for example, a number sequence of the number of words wrapped in the text sample, and whether each word is represented by "0" or "1" as a keyword, wherein "0" indicates that the word is not selected as a keyword, and "1" indicates that the word is selected as a keyword, for example, 1000 words are included in a text sample, and the recommended predicted value y' and the actual value y are each represented by a number sequence of 1000 "0" or "1".

In this embodiment, according to the detection error between the recommendation predicted value and the classification label, the parameters in the keyword recommendation model are adjusted until the loss function reaches a convergence condition, where the convergence condition may be that the loss value of the loss function is smaller than a preset threshold value, so as to obtain the keyword recommendation model, and the loss function that may be selected is a cross entropy loss function. The cross entropy loss function is a log likelihood function, and a commonly used activation function is a sigmoid activation function, so that the loss function can solve the problem of too slow weight updating, and can be used in two-class and multi-class tasks when the weight updating is fast and the error is small when the error is large.

Referring to fig. 6, a schematic diagram of a keyword recommendation model according to an embodiment of the application is shown.

As can be seen in the figure, the keyword recommendation model of this embodiment at least includes:

A word embedding processing layer: receiving an index sample, performing word embedding operation on the index sample, and outputting word vectors of words in the index sample;

a primary environmental information extraction layer: receiving word vectors of words in the index sample, and extracting primary environment information to obtain primary environment word vectors;

an input environmental information extraction layer: receiving a primary environmental word vector, and extracting input environmental information to obtain the input environmental word vector;

A splicing layer: inputting an input environment word vector and a word vector to generate a keyword vector;

a predictive classification layer: the method can be a two-prediction classification layer or a classification layer for performing other types of predictions, receives keyword vectors and outputs recommended prediction values.

In the training process, parameters in the keyword recommendation model are adjusted according to detection errors between the recommendation predicted value and the classification labels, and finally the keyword recommendation model is obtained.

Referring to fig. 7, a schematic diagram of a prediction process according to an embodiment of the application is shown.

In the figure, it is assumed that a word sequence in a text sample is ："Surgeon"、"Genaral"、"defends"、"US"、"booster"、"shot"、"plan"、"as"、"much"、"of"、"the"、"world"、"awaits"、"vaccines"...,, word vectors of each word in the text sample are obtained by using a word embedding processing layer, and then the word vectors are input into a primary environment information extraction layer. The extraction of the pre-selected words, for example for the words "shot", is performed using a set first number (illustrated in the figure as 5), the first five words being "Surgeon", "Genaral", "defends", "US" and "boost", respectively, and the last five words being "plan", "as", "much", "of" and "the", respectively, the pre-selected words being the words "Surgeon", "Genaral", "defends", "US", "boost", "plan", "as", "much", "of" and "the". By calculating the IDF of each pre-selected word, multiplying the pre-selected word vector by the corresponding inverse text frequency, and then carrying out weighted summation, the primary environment word vector (shown schematically in the figure) corresponding to the word vector can be obtained. And then inputting the primary environmental word vector into an input environmental information extraction layer of the keyword recommendation model to obtain an input environmental word vector, for example, the primary environmental word vector, the square of the primary environmental word vector, the word vector and the square of the word vector which are illustrated in the figure are spliced to obtain the input environmental word vector. And then splicing the input environment word vector, the word vector and the IDF corresponding to the word vector to generate a keyword vector for classification prediction to obtain a recommended predicted value, wherein in the figure, 0 or 1 is used for indicating a predicted result, 0 indicates that the word is not selected as a keyword, and 1 indicates that the word is selected as a keyword.

In one embodiment, in order to improve training efficiency of the keyword prediction model, data compression processing is performed on the text sample, referring to fig. 8, specifically including steps S801 to S804:

Step S801, at least one user keyword sample and a corresponding classification label are obtained, where the classification label is a target label or a non-target label.

In one embodiment, the keywords selected by different users are different in the same text sample, i.e., each text sample corresponds to a plurality of user keyword samples of different users (only differing in the category labels to which each word corresponds).

In step S802, the probability that each classified label is a target label is calculated, and a label probability distribution is obtained.

In one embodiment, the probability calculation is performed on the classification label of each word in the text sample, for example, the first word of a certain article is read by 10 users and is selected by 3 people, and the probability of the real keyword of the word is 0.3. In this way a label probability distribution is derived from the classification labels of the user keyword samples, i.e. the probability of each word in the text sample being selected as a keyword is included.

Step S803, combining the user keyword samples by using the tag probability distribution to obtain an index sample corresponding to the text sample.

Step S804, constructing a keyword training data set according to the index sample.

In an embodiment, all the user keyword samples in the same text sample are combined into one text sample according to the probability classification labels, and an index sample of the text sample is obtained, namely, one text sample does not consider the user, only corresponds to one index sample, and then a keyword training data set is generated according to the index sample. The class labels in the dataset are no longer the "0" or "1" sequences described above, but rather the word is used as a probability value for the keyword. It will be appreciated that the model training method is the same even though the behavior of the class labels is different. In this embodiment, for example, 2 ten thousand texts, selecting keyword selection information of 1000 users, generating 2000 ten thousand text samples, compressing 2000 ten thousand data into 2 ten thousand data according to the method, and finally obtaining 2 ten thousand text samples (classification labels are probability values of words as keywords), wherein the method according to the embodiment is capable of reducing the number of samples, accelerating training and not losing processing precision.

In one embodiment, to speed up the reasoning process of the keyword predictive model, the keyword predictive model is enabled to batch process text samples. Because the word lengths of different texts are different, the input of the different texts into the keyword prediction model can lead to the change of model parameters and output, and only the input and prediction of a single article can be supported, therefore, in the embodiment, a text mask layer is added into the keyword prediction model, a text mask is obtained, and the length filling processing (such as the full complement of the insufficient length by 0) is carried out on an index sample according to the text mask, so that a keyword training data set is obtained; the text lengths of index samples in the keyword training data set are consistent, so that the keyword prediction model supports batch input of a plurality of texts, and the reasoning process of the keyword prediction model is accelerated.

According to the keyword recommendation model training method, index samples are obtained, a keyword training data set is constructed according to the index samples, words of the keyword recommendation model are input into a word embedding processing layer through the index samples, word vectors of the words are obtained, the word vectors are input into a primary environment information extraction layer, primary environment word vectors are obtained, then the primary environment word vectors are input into the input environment information extraction layer, input environment word vectors are obtained, keyword vectors are generated according to the input environment word vectors and the word vectors, the keyword vectors are input into a prediction classification layer, a recommendation prediction value is obtained, parameters in the keyword recommendation model are adjusted according to detection errors between the recommendation prediction value and classification labels, and the keyword recommendation model is obtained. According to the method, the device and the system, the input environment word vectors related to the word vectors are obtained, and the learning difficulty of the keywords and the environment information corresponding to the keywords are combined, so that the selected keywords meet the learning requirements of users, and the relevance and the accuracy of keyword recommendation are improved.

In an embodiment of the present application, a keyword recommendation method is further provided, referring to fig. 9, including but not limited to steps S901 to S903:

step S901, obtaining a text to be recommended;

step S902, generating an index text according to a text to be recommended;

step S903, inputting the index text into a keyword recommendation model to obtain a classification result of keyword recommendation, wherein the keyword recommendation model is trained by using the keyword recommendation model training method described in the above embodiment. The keywords obtained in the index text can be ranked according to the classification probability according to the requirement of the user for learning the words, for example, the first 20 words in the ranking are recommended to the user for learning. The keywords selected by the keyword recommendation model meet the learning requirement of the user, and the relevance and accuracy of keyword recommendation are improved.

The embodiment of the present disclosure further provides a keyword recommendation model training device, which may implement the keyword recommendation model training method, and referring to fig. 10, the device includes:

a text sample acquiring module 101, configured to acquire at least one text sample, and generate an index sample including index information of words according to the text sample;

The training data set construction module 102 is configured to construct a keyword training data set according to the index sample, where the keyword training data set includes the index sample and the classification label;

The word vector processing module 103 is configured to perform word embedding operation on a word embedding processing layer for inputting the index sample into the keyword recommendation model, so as to obtain a word vector of the word;

the primary environmental word vector calculation module 104 is configured to input a word vector into a primary environmental information extraction layer of the keyword recommendation model to extract primary environmental information, so as to obtain a primary environmental word vector;

The input environmental word vector calculation module 105 is configured to perform a first splicing process on the primary environmental word vector and the word vector by using an input environmental information extraction layer of the keyword recommendation model, so as to generate an input environmental word vector corresponding to the word;

The keyword vector generation module 106 is configured to perform a second splicing process according to the input environmental word vector and the word vector, so as to obtain a keyword vector;

the classification prediction module 107 is configured to input a keyword vector into a prediction classification layer of the keyword recommendation model to obtain a recommendation prediction value;

And the parameter adjustment module 108 is configured to adjust parameters in the keyword recommendation model according to the detection error between the recommendation predicted value and the classification label until the loss function meets the convergence condition, thereby obtaining the keyword recommendation model.

In one embodiment, the primary environmental word vector calculation module 104 of the keyword recommendation model training device uses the primary environmental information extraction layer to obtain the pre-selected word of the word vector according to the index information, then obtains the inverse text frequency of the pre-selected word, and finally calculates the primary environmental word vector corresponding to the word vector according to the pre-selected word vector and the inverse text frequency.

In an embodiment, the primary environmental word vector calculation module 104 of the keyword recommendation model training device is further configured to copy the words in the index sample according to the second number, perform a displacement operation on the copied words column by column to obtain a displacement word matrix, obtain the column word of the displacement word matrix, and finally obtain the column word corresponding to the word vector as the pre-selected word according to the index information.

In one embodiment, the input environmental word vector calculation module 105 of the keyword recommendation model training apparatus performs a first concatenation process on the primary environmental word vector and the word vector by using the input environmental information extraction layer, so as to generate an input environmental word vector corresponding to the word.

In an embodiment, the keyword vector generation module 106 of the keyword recommendation model training apparatus obtains the inverse text frequency of the word vector, performs the second stitching process on the input environmental word vector, the word vector, and the inverse text frequency of the word vector, and generates the keyword vector.

The specific implementation manner of the keyword recommendation model training device in this embodiment is substantially identical to the specific implementation manner of the keyword recommendation model training method described above, and will not be described herein.

The embodiment of the present disclosure further provides a keyword recommendation apparatus, which may implement the keyword recommendation method, and referring to fig. 11, the apparatus includes:

a text to be recommended acquisition module 111, configured to acquire a text to be recommended;

an index text generation module 112, configured to generate an index text according to the text to be recommended;

The keyword recommendation module 113 is configured to input the index text into a keyword recommendation model, obtain a classification result of the keyword recommendation, and train the keyword recommendation model by using the keyword recommendation model training method described in the above embodiment.

The specific implementation manner of the keyword recommendation apparatus in this embodiment is substantially identical to the specific implementation manner of the keyword recommendation method described above, and will not be described herein. The embodiment of the disclosure also provides an electronic device, including:

At least one memory;

At least one processor;

At least one program;

The program is stored in the memory, and the processor executes the at least one program to implement the keyword recommendation model training method and the recommendation method described above. The electronic device can be any intelligent terminal including a mobile phone, a tablet Personal computer, a Personal digital assistant (PDA for short), a vehicle-mounted computer and the like.

Referring to fig. 12, fig. 12 illustrates a hardware structure of an electronic device according to another embodiment, the electronic device includes:

The processor 1201 may be implemented by a general-purpose CPU (central processing unit), a microprocessor, an application-specific integrated circuit (ApplicationSpecificIntegratedCircuit, ASIC), or one or more integrated circuits, etc. for executing related programs to implement the technical solutions provided by the embodiments of the present disclosure;

The memory 1202 may be implemented in the form of a ROM (read only memory), a static storage device, a dynamic storage device, or a RAM (random access memory). The memory 1202 may store an operating system and other application programs, and when the technical solutions provided in the embodiments of the present disclosure are implemented by software or firmware, relevant program codes are stored in the memory 1202, and the processor 1201 invokes a keyword recommendation model training method and a recommendation method for executing the embodiments of the present disclosure;

an input/output interface 1203 for implementing information input and output;

The communication interface 1204 is configured to implement communication interaction between the device and other devices, and may implement communication in a wired manner (e.g., USB, network cable, etc.), or may implement communication in a wireless manner (e.g., mobile network, WIFI, bluetooth, etc.); and

A bus 1205 for transferring information between various components of the device such as the processor 1201, memory 1202, input/output interface 1203, and communication interface 1204;

Wherein the processor 1201, the memory 1202, the input/output interface 1203 and the communication interface 1204 enable communication connection between each other inside the device via a bus 1205.

The embodiment of the disclosure also provides a storage medium, which is a computer readable storage medium, and the computer readable storage medium stores computer executable instructions for causing a computer to execute the keyword recommendation model training method and the recommendation method.

According to the keyword recommendation model training method and device, the keyword recommendation method and device, the electronic equipment and the storage medium, the keyword recommendation model is used for enabling the selected keywords to meet the learning requirement of a user by acquiring the input environment word vectors related to the word vectors and combining the learning difficulty of the keywords and the environment information corresponding to the keywords, and the relevance and the accuracy of keyword recommendation are improved.

The memory, as a non-transitory computer readable storage medium, may be used to store non-transitory software programs as well as non-transitory computer executable programs. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory remotely located relative to the processor, the remote memory being connectable to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The embodiments described in the embodiments of the present disclosure are for more clearly describing the technical solutions of the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure, and as those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are equally applicable to similar technical problems.

It will be appreciated by those skilled in the art that the solutions shown in fig. 1-9 are not limiting to the embodiments of the present disclosure, and may include more or fewer steps than shown, or certain steps may be combined, or different steps.

The above described apparatus embodiments are merely illustrative, wherein the units illustrated as separate components may or may not be physically separate, i.e. may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Those of ordinary skill in the art will appreciate that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof.

The terms "first," "second," "third," "fourth," and the like in the description of the application and in the above figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.

It should be understood that in the present application, "at least one (item)" means one or more, and "a plurality" means two or more. "and/or" for describing the association relationship of the association object, the representation may have three relationships, for example, "a and/or B" may represent: only a, only B and both a and B are present, wherein a, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b or c may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.

In several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented by way of the same. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in the form of electrical, mechanical, or the like.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including multiple instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory RAM), a magnetic disk, or an optical disk, or other various media capable of storing a program.

Preferred embodiments of the disclosed embodiments are described above with reference to the accompanying drawings, and thus do not limit the scope of the claims of the disclosed embodiments. Any modifications, equivalent substitutions and improvements made by those skilled in the art without departing from the scope and spirit of the embodiments of the present disclosure shall fall within the scope of the claims of the embodiments of the present disclosure.

Claims

1. The keyword recommendation model training method is characterized by comprising the following steps of:

performing first splicing processing on the primary environment word vector, the square of the primary environment word vector, the word vector and the square of the word vector by using an input environment information extraction layer of the keyword recommendation model, and generating the input environment word vector corresponding to the word;

Acquiring the inverse text frequency of the word vector, and performing second splicing processing according to the input environment word vector, the word vector and the inverse text frequency corresponding to the word vector to obtain a keyword vector;

According to the detection error between the recommendation predicted value and the classification label, adjusting parameters in the keyword recommendation model until a loss function meets a convergence condition, and obtaining the keyword recommendation model, wherein the keyword recommendation model is used for performing keyword recommendation processing;

The step of inputting the word vector into a primary environment information extraction layer of the keyword recommendation model for primary environment information extraction to obtain a primary environment word vector, comprising the following steps:

acquiring the reverse text frequency of the pre-selected word;

Calculating a primary environment word vector corresponding to the word vector according to the pre-selected word vector and the inverse text frequency;

the obtaining, by the primary environmental information extraction layer, the pre-selected word of the word vector according to the index information includes:

Acquiring a list word of the displacement word matrix;

And acquiring the list word corresponding to the word vector according to the index information to serve as the pre-selected word.

2. The keyword recommendation model training method of claim 1, wherein constructing a keyword training dataset from the index sample comprises:

and constructing a keyword training data set according to the index sample.

3. The keyword recommendation model training method of claim 1, wherein constructing a keyword training dataset from the index sample comprises:

Acquiring a text mask;

4. A keyword recommendation method, comprising:

Acquiring a text to be recommended;

generating an index text according to the text to be recommended;

and inputting the index text into a keyword recommendation model to perform keyword recommendation processing to obtain target keywords, wherein the keyword recommendation model is trained by using the keyword recommendation model training method according to any one of claims 1 to 3.

5. A keyword recommendation model training device, characterized by comprising:

The input environment word vector calculation module is used for carrying out first splicing processing on the primary environment word vector, the square of the primary environment word vector, the word vector and the square of the word vector by utilizing an input environment information extraction layer of the keyword recommendation model, so as to generate the input environment word vector corresponding to the word;

The keyword vector generation module is used for acquiring the inverse text frequency of the word vector, and performing second splicing processing according to the input environment word vector, the word vector and the inverse text frequency corresponding to the word vector to obtain a keyword vector;

The parameter adjustment module is used for adjusting parameters in the keyword recommendation model according to the detection error between the recommendation predicted value and the classification label until a loss function meets a convergence condition, so as to obtain the keyword recommendation model, and the keyword recommendation model is used for performing keyword recommendation processing;

acquiring the reverse text frequency of the pre-selected word;

Acquiring a list word of the displacement word matrix;

6. An electronic device, comprising:

At least one memory;

At least one processor;

At least one program;

the program is stored in the memory, and the processor executes the at least one program to implement:

a keyword recommendation model training method as claimed in any one of claims 1 to 3, or a keyword recommendation method as claimed in claim 4.

7. A storage medium that is a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions for causing a computer to perform: