CN117709339A - Language model enhancement method and system based on live broadcasting room user behavior network - Google Patents
Language model enhancement method and system based on live broadcasting room user behavior network Download PDFInfo
- Publication number
- CN117709339A CN117709339A CN202311739893.7A CN202311739893A CN117709339A CN 117709339 A CN117709339 A CN 117709339A CN 202311739893 A CN202311739893 A CN 202311739893A CN 117709339 A CN117709339 A CN 117709339A
- Authority
- CN
- China
- Prior art keywords
- user
- text
- language model
- user behavior
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 230000006399 behavior Effects 0.000 claims abstract description 126
- 239000013598 vector Substances 0.000 claims abstract description 93
- 238000012549 training Methods 0.000 claims abstract description 76
- 238000012512 characterization method Methods 0.000 claims abstract description 30
- 238000004458 analytical method Methods 0.000 claims abstract description 21
- 230000002708 enhancing effect Effects 0.000 claims description 21
- 230000011218 segmentation Effects 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 4
- 238000005065 mining Methods 0.000 claims description 4
- 230000014509 gene expression Effects 0.000 abstract description 13
- 230000000694 effects Effects 0.000 abstract description 8
- 230000007704 transition Effects 0.000 description 12
- 230000006870 function Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000013507 mapping Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000005295 random walk Methods 0.000 description 2
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 241000209140 Triticum Species 0.000 description 1
- 235000021307 Triticum Nutrition 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application discloses a language model enhancement method and system based on a live broadcasting room user behavior network. According to the technical scheme provided by the embodiment of the application, the user behavior network is constructed based on the live broadcasting room information, the user identification information, the user behavior information and the user text information; performing entity embedding characterization training based on a user behavior network, and outputting text vectors corresponding to user text information; and constructing a similar text data set corresponding to the text information of the user according to the text vector, and performing language model enhancement training based on the similar text data set. By adopting the technical means, entity embedding characterization training is performed by constructing the user behavior network, so that similar text data sets are accurately mined according to text vector characterization, expansion of similar samples of different sources is realized, language model training is performed by the method, the content understanding capability of a language model in different context scenes can be enhanced, accurate language analysis is performed on user behaviors and text expression, and the application effect of the language model is improved.
Description
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to a language model enhancement method and system based on a live broadcasting room user behavior network.
Background
At present, with the continuous development of live broadcast applications, functions of the live broadcast applications are more and more varied, and ways of users to participate in interaction in the terminal are also gradually increased. Users can enter a favorite living broadcast room to enjoy a host, play a role in the living broadcast room, send a barrage, interact with other living broadcast room members, and the like. To better understand the characteristics of a user, a common approach is to analyze the user's speech and published text using language models.
However, for live scenes with partial entertainment, the user's text expression is random, the text is short, and the expressed information is fragmented, which is rich in a lot of noise. The conventional language model is difficult to accurately analyze the characters, so that insufficient understanding of the contents of users and living rooms is caused, and the language analysis effect is deviated.
Disclosure of Invention
The embodiment of the application provides a language model enhancement method and a language model enhancement system based on a live broadcasting room user behavior network, which can improve the analysis precision of a language model and solve the technical problem of large user language analysis error in a live broadcasting scene.
In a first aspect, an embodiment of the present application provides a method for enhancing a language model based on a live room user behavior network, including:
constructing a user behavior network based on live broadcasting room information, user identification information, user behavior information and user text information;
performing entity embedding characterization training based on a user behavior network, and outputting text vectors corresponding to user text information;
and constructing a similar text data set corresponding to the text information of the user according to the text vector, and performing language model enhancement training based on the similar text data set.
In a second aspect, embodiments of the present application provide a language model enhancement system based on a live room user behavior network, including:
the network construction module is configured to construct a user behavior network based on live broadcasting room information, user identification information, user behavior information and user text information;
the entity embedding module is configured to perform entity embedding characterization training based on the user behavior network and output text vectors corresponding to the user text information;
and the model enhancement module is configured to construct a similar text data set corresponding to the text information of the user according to the text vector, and perform language model enhancement training based on the similar text data set.
In a third aspect, an embodiment of the present application provides a language model enhancement device based on a live room user behavior network, including:
a memory and one or more processors;
the memory is configured to store one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of language model enhancement based on live room user behavior network as described in the first aspect.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium storing computer-executable instructions that, when executed by a computer processor, are configured to perform the method of language model enhancement based on a live-room user behavior network as described in the first aspect.
In a fifth aspect, embodiments of the present application provide a computer program product, which includes instructions that, when executed on a computer or a processor, cause the computer or the processor to perform the method for enhancing a language model based on a live room user behavior network according to the first aspect.
According to the embodiment of the application, the user behavior network is constructed based on the live broadcasting room information, the user identification information, the user behavior information and the user text information; performing entity embedding characterization training based on a user behavior network, and outputting text vectors corresponding to user text information; and constructing a similar text data set corresponding to the text information of the user according to the text vector, and performing language model enhancement training based on the similar text data set. By adopting the technical means, entity embedding characterization training is performed by constructing the user behavior network, so that similar text data sets are accurately mined according to text vector characterization, expansion of similar samples of different sources is realized, language model training is performed by the method, the content understanding capability of a language model in different context scenes can be enhanced, accurate language analysis is performed on user behaviors and text expression, and the application effect of the language model is improved.
Drawings
FIG. 1 is a flowchart of a method for enhancing language model based on a live room user behavior network according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a user behavior network in an embodiment of the present application;
FIG. 3 is a flow chart of language model enhanced training in an embodiment of the present application;
FIG. 4 is a schematic diagram of an enhanced language model application in an embodiment of the present application;
FIG. 5 is a schematic structural diagram of a language model enhancement system based on a live room user behavior network according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a language model enhancement device based on a live room user behavior network according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the following detailed description of specific embodiments thereof is given with reference to the accompanying drawings. It is to be understood that the specific embodiments described herein are merely illustrative of the application and not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the matters related to the present application are shown in the accompanying drawings. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart depicts operations (or steps) as a sequential process, many of the operations can be performed in parallel, concurrently, or at the same time. Furthermore, the order of the operations may be rearranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figures. The processes may correspond to methods, functions, procedures, subroutines, and the like.
The language model enhancement method based on the live broadcasting room user behavior network aims at precisely mining similar samples of different sources through text vector characterization so as to train the content understanding capability of the language model in different context scenes.
In a live broadcast scene, a user rewards a host, plays a wheat and sends out a public screen in a live broadcast room, interacts with other live broadcast room members, acquires various recharging tasks and consumption coupons issued by a platform, and is used for recharging, gift sending and other actions in a terminal, and corresponding text data can be generated. The live broadcasting server analyzes the language expression of the user through a language model by collecting various types of text data generated by the live broadcasting room, and further carries out corresponding application according to the language analysis result.
Generally, in order to improve the analysis accuracy of the language model, the language model is enhanced and trained by collecting the words issued by the user in the live application end, so as to improve the language analysis accuracy in the live scene. However, for live scenes with partial entertainment, the user's text expression is arbitrary, the text is short, and the expressed information is fragmented, which is rich in a lot of noise. The conventional language model is difficult to accurately analyze the characters, so that insufficient understanding of the contents of users and living rooms is caused, and the language analysis effect is deviated. Conventional language model enhancement training using this portion of live room text data containing a lot of noise may not perform better but may deteriorate the model due to the noise of the text. It is therefore difficult to easily characterize the user's interests by means of text analysis, understanding the events occurring within the user and the live room, which presents challenges for efficient operation of the live room and the user.
Based on the method, the language model enhancement method based on the live broadcasting room user behavior network is provided, and the technical problem of user language analysis errors in live broadcasting scenes is solved.
Examples:
fig. 1 is a flowchart of a language model enhancing method based on a live-broadcasting room user behavior network according to an embodiment of the present application, where the language model enhancing method based on the live-broadcasting room user behavior network provided in the embodiment may be implemented by a language model enhancing device based on the live-broadcasting room user behavior network, and the language model enhancing device based on the live-broadcasting room user behavior network may be implemented by software and/or hardware, and the language model enhancing device based on the live-broadcasting room user behavior network may be formed by two or more physical entities or may be formed by one physical entity. In general, the language model enhancement device based on the live room user behavior network may be a computing device such as a live server device, a server host, etc.
The following describes an example in which the language model enhancing device based on the live-broadcasting-room user behavior network is taken as a main body for executing the language model enhancing method based on the live-broadcasting-room user behavior network. Referring to fig. 1, the language model enhancement method based on the live-room user behavior network specifically includes:
s110, constructing a user behavior network based on the live broadcasting room information, the user identification information, the user behavior information and the user text information.
When the language model is enhanced, the user behavior network is established through the user behaviors in the live broadcast scene, so that the similar text data set in the live broadcast scene is mined according to the user behavior network to conduct model enhancement training.
The heterogeneous network is a network formed by multiple types of nodes such as users, living rooms, different user behaviors and the like, and the types of the edges forming the network are different due to the fact that the types of the nodes are different, so that the heterogeneous network is called as the heterogeneous network. Through the user behavior network, the context relation among different nodes can be identified for entity embedding training.
Referring to fig. 2, a schematic structural diagram of a user behavior network according to an embodiment of the present application is provided, and corresponding live broadcast room information is collected based on different types of nodes formed by the user behavior network. The collected live room information includes live room information (such as room 1, room 2, room 3, etc.), user identification information (such as user 1, user 2, user 3, etc.), user behavior information (such as speaking, focusing, entering, viewing, etc.), and user text information (text 1, text 2, and text 3). By collecting the information of each type, a user behavior network can be correspondingly constructed.
It can be understood that, for the user text information of the user in different scenes, the similar text information can be clustered by combining the context relation of the user behavior, so that part of the text with noise and difficult understanding can be clustered by the context relation, and the part of the information can be conveniently analyzed and understood based on the clustering result. Based on the above, the embodiment of the application collects the live broadcasting room information of different types of nodes under the corresponding user behaviors according to the context relation on the basis of collecting the user text information by constructing the user behavior network, so as to accurately cluster the user text information.
In practical application, the method can collect corresponding types of live broadcast room information according to various context relations of the user text information in the live broadcast scene, and the method is used for constructing a user behavior network, so that more accurate and comprehensive user text information clustering is realized.
S120, performing entity embedding characterization training based on the user behavior network, and outputting text vectors corresponding to the user text information.
Further, based on the user behavior network, the embodiment of the application determines the text vector for representing the text information of the user through entity embedding representation training. And according to the determined text vector, performing accurate similar text clustering.
Wherein, based on the user behavior network, entity embedding characterization training is carried out, comprising:
taking live broadcasting room information, user identification information, user behavior information and user text information as entities, and performing entity embedding characterization training based on context relations among the entities to obtain embedded vectors for characterizing each entity in a user behavior network.
According to the embodiment of the application, live broadcasting room information, user identification information, user behavior information and user text information are taken as entities, a random walk method is adopted for sampling a network structure in entity embedding characterization training, and then embedding vector characterization of different entities in a network is learned, and embedding vectors for characterizing the entities are determined. In the training process, the initial embedded vector of each entity is assigned by using the corresponding array, the similarity of the embedded vectors is calculated according to the context relation, and the loss function is calculated according to the similarity. The adjustment of the embedded vector can be performed based on the loss function, and the loss function is recalculated, and so on until the loss converges. For each entity, training is performed in the manner described above to determine its embedded vector. Taking the user behavior "user 1-enter-room 1" as an example, the embedded vector characterizing "user 1" plus the embedded vector characterizing "enter" should be equal to the embedded vector characterizing "room 1". And by analogy, limiting the loss function of each entity through the context relation, and training to obtain the embedded vector for accurately representing each entity.
Optionally, for each entity in the user behavior network, an entity relationship transition probability is calculated for the entity to transition to an entity adjacent to the entity according to the context. The entity and its neighboring entities may both be from the user behavior network, the neighboring entity being an entity directly connected to the entity, for example: entity a is directly connected to entity B, which may be referred to herein as an entity adjacent to entity a.
For each entity, embodiments of the present application may determine an entity relationship transition probability of the entity to its neighboring entities, the entity relationship transition probability being comprised of an entity relationship ratio and an inverse triplet probability. The entity relationship ratio is used for representing the ratio of any entity relationship from an entity to an adjacent entity in all entity relationships, and the reverse triplet probability is used for representing the importance of any entity relationship in all triples. In this way, the transition probabilities that an entity can transition to its neighboring entities through each entity relationship can be determined based on each entity relationship transition probability.
When calculating the entity relation transition probability of the entity to the adjacent entity of the entity, acquiring a designated entity relation between the entity and the adjacent entity of the entity aiming at each entity in the user behavior network; determining the ratio of the appointed entity relationship in the entity relationship between the entity and the adjacent entity of the entity aiming at the appointed entity relationship in all the acquired entity relationships; counting the occurrence times of the specified entity relationship in the triples of the user behavior network; determining reverse triplet probability corresponding to the specified entity relation according to the statistics times and the number of the triples; and obtaining the designated entity relationship transition probability based on the designated entity relationship ratio and the reverse triplet probability.
And determining all reference entities corresponding to the target entity according to the relation transition probability of each entity of the target entity and the preset jump step number of the target entity. Here, based on the entity relationship transition probability obtained above, the embodiment of the present application aims to determine a corresponding reference entity for a target entity by using the entity relationship transition probability. The reference entity may be an entity describing the target entity generated by means of random walk, i.e. the reference entity may be not only a neighboring entity directly connected to the target entity, but also an entity indirectly connected to the target entity, for example: entity A is directly connected with entity B, entity B is directly connected with entity C, but entity A is not directly connected with entity C, and entity A is indirectly connected with entity C through entity B, and entity C can be called as a reference node of entity A. In a specific operation, a reference entity corresponding to the target entity can be determined by setting a preset jump number, for example: setting the jump number as 1, and taking the adjacent node directly connected with the entity as a reference entity; setting the number of the jump steps to be 2, taking the entity as a starting point, taking the entity corresponding to the two jump steps as a reference entity, and the like.
Then, calculating an embedded vector of the target entity based on the target entity and all reference entities corresponding to the target entity; the embedded vector reflects the entity relationship between the target entity and all reference entities. Here, in the embodiment of the present application, the embedded vector may be used to characterize the entity. Since in a user behavior network the entities may be described in text form, for the data originally obtained, it is often necessary for the computer to process it for convenience to convert it into a vector representation, i.e. to encode the entities into a vector space, such that each entity is represented by a vector of the vector space. For the initial vectorized representation of the originally obtained entity, i.e. mapping the entity into vector space, common methods or models, such as existing semantic mapping methods, etc., may be selected, without limitation. Because the vector mapping of the entities at present cannot fully reflect the association between the entities, the embodiment of the application performs operation or multiple iterative operations by determining the reference entity corresponding to the entity, so that the calculated vector of the entity can be fused or embody the vector characteristics of the reference entity, and the original vector representation of the entity is optimized.
It should be noted that there are many embodiments of entity embedding training, and the specific entity embedding vector characterization method is not limited in this embodiment, and is not repeated here.
Further, outputting a text vector corresponding to the user text information, including: and determining an embedded vector of an entity corresponding to the user text information in the user behavior network as a text vector of the user text information.
Then, based on the determined embedded vector, an embedded vector characterizing the user text information is selected therefrom as a text vector of the user text information. It can be understood that, since many user text information in a live broadcast scene is in noise and is difficult to understand, the embodiment of the application performs entity embedding characterization training on various entities by utilizing the context relation of the user behavior network, so that the user text information with normal expression and the user text information with noise expression, which have similar semantics, can be characterized by similar embedding vectors by virtue of the context relation, even if two user text information have large difference due to noise influence, the two user text information can be characterized by similar embedding vectors by virtue of the context relation of the user behavior network, so that accurate text clustering can be performed subsequently. And for the embedded vectors of other entities, the embedded vectors are only used for calculating text vectors, and the embedded vectors do not need to be output in practical application.
S130, constructing a similar text data set corresponding to the user text information according to the text vector, and performing language model enhancement training based on the similar text data set.
Finally, text clustering can be adaptively performed based on the determined text vectors, and the user text information with the normal expressions and the user text information with the noise expressions with similar semantics are clustered together for language model enhancement training. The similar text data set can adopt a nearest neighbor matching algorithm to calculate the similarity of the text vectors obtained through the determination, find similar text vectors and construct the similar text data set.
Wherein constructing a similar text data set corresponding to the user text information according to the text vector comprises:
and carrying out vector similarity comparison according to the text vectors of the user text information, and selecting corresponding user text information according to the vector similarity comparison result to construct a similar text data set.
It can be understood that if the vector similarity of the two text vectors is the same, even if the actual text expression modes of the user text information corresponding to the two text vectors are different, such as different text expression situations including noise, reverse order, hyponym, homonym, and the like, the text vectors of the user text information with similar context semantics can reach a certain similarity based on the text vectors obtained by the entity embedded vector characterization. Therefore, all text vectors with similarity in a set proportion can be clustered to obtain a similar text data set.
Further, referring to fig. 3, language model enhancement training based on similar text datasets includes:
s1301, acquiring a word segmentation language sample and a mask training sample based on a similar text data set;
s1302, inputting word segmentation language samples and mask training samples into a pre-trained language model, and training the language model through a multi-head self-care network layer.
Specifically, the language model includes a Bert model using a Transformer encoder, and since self-attion mechanism, upper and lower layers of the model are directly and entirely connected to each other. While OpenAI GPT uses a transducer decoder, but it is a limited transducer structure that requires left to right, ELMo uses bi-directional LSTM, but is simply spliced at the highest layer of the two unidirectional LSTM. Only the Bert model is a structure which is truly bidirectional in all sentence feature extraction layers, and can capture the whole semantic information of the sentence context at the same time. The language training sample mentioned in the embodiment of the application is a corresponding text training sample, and the corresponding language model is obtained by continuously learning and training the text training sample. On language model training, the standard left-to-right predictive next word is not used as the target task, but two new tasks are proposed. The first task, they refer to as MLM, i.e. 15% of words in the input word sequence are randomly blocked, then the task is to predict these words in the blocks, and it can be seen that compared to the traditional language model prediction objective function, MLM can predict the probability of this word from the full context information of the words in the blocks, not just unidirectional information. Such as "today, is, a, nice, day" for the input word. When the pre-training is performed, a word is randomly blocked, such as blocked to 'is', and then the predictive training of the model is performed by using the context information 'today', 'a', 'nice', 'day'. The learning of the final model parameters is achieved by repeating the above.
For the conventional language model, the relation between sentences is not considered. In order for the model to learn the relationships between sentences, a second objective task is to predict the next sentence. The method is characterized in that the method is a binary classification problem, 50% of time is spent in inputting a sentence and splicing the next sentence, classification labels are positive examples, and the other 50% are spent in inputting a sentence and splicing the non-next random sentences, and labels are negative examples. Finally, the objective function of the whole training is that the two samples are subjected to maximum likelihood learning.
Through the language model training mode, the pre-trained language model is subjected to enhanced training again by using the similar text data set, so that the understanding analysis capability of texts with various noise and difficult understanding can be obtained. For the pre-training part of the language model, the language model training can also be performed based on a conventional training sample, and the pre-training language model is not limited in a fixed manner in the embodiment of the application, and is not described herein.
Therefore, by combining various behaviors of the user with language model training, the understanding of the language model on the contexts of the user such as behaviors in different live rooms can be enhanced.
Optionally, after language model enhancement training based on the similar text data set, further comprising:
and acquiring the text data of the live broadcasting room, inputting the text data of the live broadcasting room into a language model to obtain a language analysis result, and carrying out similar user clustering, content tag addition among the main broadcasting rooms or mining among the live broadcasting rooms of the user based on the language analysis result.
Referring to fig. 4, in practical application, after language model enhancement training is performed on various types of user text information which are difficult to understand in a live broadcast scene by the above-mentioned language model enhancement method, similar user text information can be found in the live broadcast scene, and then user clustering is performed according to the similar user text information. Similarly, the content type of the live broadcast room is understood by analyzing the text information of the user of the live broadcast room, and further different content type labels are added to the live broadcast room. In addition, according to the text information of the users in different live rooms, the text information of the current user is combined, so that the live rooms in which the users possibly interest can be analyzed to accurately recommend. Therefore, by means of the enhanced language model, through accurate user text information analysis and understanding, the application effect of the language model in the live broadcast scene can be improved, and the live broadcast service and operation and maintenance effect can be improved.
The user behavior network is constructed based on the live broadcasting room information, the user identification information, the user behavior information and the user text information; performing entity embedding characterization training based on a user behavior network, and outputting text vectors corresponding to user text information; and constructing a similar text data set corresponding to the text information of the user according to the text vector, and performing language model enhancement training based on the similar text data set. By adopting the technical means, entity embedding characterization training is performed by constructing the user behavior network, so that similar text data sets are accurately mined according to text vector characterization, expansion of similar samples of different sources is realized, language model training is performed by the method, the content understanding capability of a language model in different context scenes can be enhanced, accurate language analysis is performed on user behaviors and text expression, and the application effect of the language model is improved.
Based on the above embodiments, fig. 5 is a schematic structural diagram of a language model enhancement system based on a live-room user behavior network provided in the present application. Referring to fig. 5, the language model enhancement system based on the live-room user behavior network provided in this embodiment specifically includes: a network construction module 21, an entity embedding module 22 and a model enhancement module 23.
Wherein the network construction module 21 is configured to construct a user behavior network based on the live room information, the user identification information, the user behavior information, and the user text information;
the entity embedding module 22 is configured to perform entity embedding characterization training based on the user behavior network, and output text vectors corresponding to user text information;
model enhancement module 23 is configured to construct a similar text dataset corresponding to the user text information from the text vector, and to perform language model enhancement training based on the similar text dataset.
Specifically, entity embedding characterization training is performed based on a user behavior network, including:
taking live broadcasting room information, user identification information, user behavior information and user text information as entities, and performing entity embedding characterization training based on context relations among the entities to obtain embedded vectors for characterizing each entity in a user behavior network.
Wherein outputting a text vector corresponding to the user text information comprises:
and determining an embedded vector of an entity corresponding to the user text information in the user behavior network as a text vector of the user text information.
Specifically, constructing a similar text data set corresponding to user text information according to the text vector includes:
and carrying out vector similarity comparison according to the text vectors of the user text information, and selecting corresponding user text information according to the vector similarity comparison result to construct a similar text data set.
Specifically, language model enhancement training based on similar text datasets, including:
acquiring word segmentation language samples and mask training samples based on similar text data sets;
the word segmentation language sample and the mask training sample are input into a pre-trained language model, and the language model is trained through a multi-head self-care network layer.
After language model enhancement training based on the similar text data set, further comprising:
and acquiring the text data of the live broadcasting room, inputting the text data of the live broadcasting room into a language model to obtain a language analysis result, and carrying out similar user clustering, content tag addition among the main broadcasting rooms or mining among the live broadcasting rooms of the user based on the language analysis result.
The user behavior network is constructed based on the live broadcasting room information, the user identification information, the user behavior information and the user text information; performing entity embedding characterization training based on a user behavior network, and outputting text vectors corresponding to user text information; and constructing a similar text data set corresponding to the text information of the user according to the text vector, and performing language model enhancement training based on the similar text data set. By adopting the technical means, entity embedding characterization training is performed by constructing the user behavior network, so that similar text data sets are accurately mined according to text vector characterization, expansion of similar samples of different sources is realized, language model training is performed by the method, the content understanding capability of a language model in different context scenes can be enhanced, accurate language analysis is performed on user behaviors and text expression, and the application effect of the language model is improved.
The language model enhancement system based on the live broadcasting room user behavior network provided by the embodiment of the application can be configured to execute the language model enhancement method based on the live broadcasting room user behavior network provided by the embodiment of the application, and has corresponding functions and beneficial effects.
On the basis of the actual embodiment, the embodiment of the application further provides a language model enhancing device based on the live-broadcasting room user behavior network, and referring to fig. 6, the language model enhancing device based on the live-broadcasting room user behavior network includes: processor 31, memory 32, communication module 33, input device 34 and output device 35. The memory is configured as a computer readable storage medium, and is configured to store a software program, a computer executable program, and modules, corresponding to the program instructions/modules of the language model enhancement method based on the live-room user behavior network according to any embodiment of the present application (for example, a network building module, an entity embedding module, and a model enhancement module in the language model enhancement system based on the live-room user behavior network). The communication module is configured to perform data transmission. The processor executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory, namely, the language model enhancement method based on the live broadcasting room user behavior network is realized. The input means may be configured to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the device. The output means may comprise a display device such as a display screen. The language model enhancing device based on the live broadcasting room user behavior network provided by the embodiment can be configured to execute the language model enhancing method based on the live broadcasting room user behavior network provided by the embodiment, and has corresponding functions and beneficial effects.
On the basis of the above embodiments, the present application further provides a computer-readable storage medium storing computer-executable instructions that, when executed by a computer processor, are configured to perform a method for enhancing a language model based on a live-room user behavior network, where the storage medium may be any of various types of memory devices or storage devices. Of course, the computer-readable storage medium provided in the embodiments of the present application, whose computer-executable instructions are not limited to the language model enhancement method based on the live-room user behavior network described above, may also perform the related operations in the language model enhancement method based on the live-room user behavior network provided in any embodiment of the present application.
On the basis of the above embodiments, the embodiments of the present application further provide a computer program product, where the technical solution of the present application is essentially or a part contributing to the prior art or all or part of the technical solution may be embodied in the form of a software product, and the computer program product is stored in a storage medium, and includes several instructions to cause a computer device, a mobile terminal or a processor therein to perform all or part of the steps of the language model enhancing method based on the live-room user behavior network according to the embodiments of the present application.
Claims (10)
1. A method for enhancing a language model based on a live room user behavior network, comprising:
constructing a user behavior network based on live broadcasting room information, user identification information, user behavior information and user text information;
performing entity embedding characterization training based on the user behavior network, and outputting text vectors corresponding to the user text information;
and constructing a similar text data set corresponding to the user text information according to the text vector, and performing language model enhancement training based on the similar text data set.
2. The method for enhancing a language model based on a live house user behavior network according to claim 1, wherein the training for entity embedding characterization based on the user behavior network comprises:
and taking the live broadcasting room information, the user identification information, the user behavior information and the user text information as entities, and performing entity embedding characterization training based on the context relation among the entities to obtain embedded vectors for characterizing each entity in the user behavior network.
3. The method for enhancing a language model based on a live-room user behavior network according to claim 2, wherein the outputting a text vector corresponding to the user text information comprises:
and determining an embedded vector of an entity corresponding to the user text information in the user behavior network as a text vector of the user text information.
4. The method for enhancing a language model based on a live-room user behavior network according to claim 1, wherein said constructing a similar text data set corresponding to said user text information from said text vector comprises:
and carrying out vector similarity comparison according to the text vectors of the user text information, and selecting the corresponding user text information according to a vector similarity comparison result to construct a similar text data set.
5. The method for language model enhancement based on live-room user behavior network of claim 1, wherein the training for language model enhancement based on the similar text data set comprises:
acquiring a word segmentation language sample and a mask training sample based on the similar text data set;
inputting the word segmentation language sample and the mask training sample into a pre-trained language model, and training the language model through a multi-head self-care network layer.
6. The method for language model enhancement based on live-room user behavior network of claim 5, further comprising, after language model enhancement training based on the set of similar text data:
and acquiring live broadcasting room text data, inputting the live broadcasting room text data into the language model to obtain a language analysis result, and carrying out similar user clustering, content label addition between the main broadcasting rooms or mining between the live broadcasting rooms based on the language analysis result.
7. A language model enhancement system based on a live room user behavior network, comprising:
the network construction module is configured to construct a user behavior network based on live broadcasting room information, user identification information, user behavior information and user text information;
the entity embedding module is configured to perform entity embedding characterization training based on the user behavior network and output text vectors corresponding to the user text information;
and the model enhancement module is configured to construct a similar text data set corresponding to the user text information according to the text vector, and perform language model enhancement training based on the similar text data set.
8. A language model enhancement device based on a live room user behavior network, comprising:
a memory and one or more processors;
the memory is configured to store one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the live room user behavior network-based language model enhancement method of any one of claims 1-6.
9. A computer-readable storage medium storing computer-executable instructions that, when executed by a computer processor, are configured to perform the live-room user behavior network-based language model enhancement method of any one of claims 1-6.
10. A computer program product comprising instructions which, when executed on a computer or processor, cause the computer or processor to perform the method of enhancing a language model based on a live room user behavior network as claimed in any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311739893.7A CN117709339A (en) | 2023-12-15 | 2023-12-15 | Language model enhancement method and system based on live broadcasting room user behavior network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311739893.7A CN117709339A (en) | 2023-12-15 | 2023-12-15 | Language model enhancement method and system based on live broadcasting room user behavior network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117709339A true CN117709339A (en) | 2024-03-15 |
Family
ID=90158556
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311739893.7A Pending CN117709339A (en) | 2023-12-15 | 2023-12-15 | Language model enhancement method and system based on live broadcasting room user behavior network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117709339A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117909494A (en) * | 2024-03-20 | 2024-04-19 | 北京建筑大学 | Abstract consistency assessment model training method and device |
-
2023
- 2023-12-15 CN CN202311739893.7A patent/CN117709339A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117909494A (en) * | 2024-03-20 | 2024-04-19 | 北京建筑大学 | Abstract consistency assessment model training method and device |
| CN117909494B (en) * | 2024-03-20 | 2024-06-07 | 北京建筑大学 | Abstract consistency assessment model training method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Uc-Cetina et al. | Survey on reinforcement learning for language processing | |
| CN111341341B (en) | Training method of audio separation network, audio separation method, device and medium | |
| US11574145B2 (en) | Cross-modal weak supervision for media classification | |
| CN111666416B (en) | Method and device for generating semantic matching model | |
| EP3913521A1 (en) | Method and apparatus for creating dialogue, electronic device and storage medium | |
| CN116720004B (en) | Recommendation reason generation method, device, equipment and storage medium | |
| CN111753076B (en) | Dialogue method, dialogue device, electronic equipment and readable storage medium | |
| CN111898379B (en) | Slot filling model training method, electronic equipment and storage medium | |
| CN111708876A (en) | Method and device for generating information | |
| CN117709339A (en) | Language model enhancement method and system based on live broadcasting room user behavior network | |
| CN115803734A (en) | Natural language enrichment using action interpretation | |
| US11157246B2 (en) | Code recommender for resolving a new issue received by an issue tracking system | |
| WO2024093578A1 (en) | Voice recognition method and apparatus, and electronic device, storage medium and computer program product | |
| CN117036834B (en) | Data classification method and device based on artificial intelligence and electronic equipment | |
| CN116644180A (en) | Training method and training system for text matching model and text label determining method | |
| CN112668608A (en) | Image identification method and device, electronic equipment and storage medium | |
| CN115186738B (en) | Model training method, device and storage medium | |
| CN115273856A (en) | Voice recognition method and device, electronic equipment and storage medium | |
| CN112149426B (en) | Reading task processing method and related equipment | |
| CN115311598A (en) | Video description generation system based on relation perception | |
| CN114461943A (en) | Deep learning-based multi-source POI semantic matching method and device and storage medium thereof | |
| CN114970494A (en) | Comment generation method and device, electronic equipment and storage medium | |
| CN112861474A (en) | Information labeling method, device, equipment and computer readable storage medium | |
| CN113469204A (en) | Data processing method, device, equipment and computer storage medium | |
| CN117521674B (en) | Method, device, computer equipment and storage medium for generating countermeasure information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |