CN113076753A - Emotion analysis model training optimization method, system and storage medium - Google Patents

Emotion analysis model training optimization method, system and storage medium Download PDF

Info

Publication number
CN113076753A
CN113076753A CN202110236422.9A CN202110236422A CN113076753A CN 113076753 A CN113076753 A CN 113076753A CN 202110236422 A CN202110236422 A CN 202110236422A CN 113076753 A CN113076753 A CN 113076753A
Authority
CN
China
Prior art keywords
emotion
text
model
label
self
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110236422.9A
Other languages
Chinese (zh)
Inventor
辛永欣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Yingxin Computer Technology Co Ltd
Original Assignee
Shandong Yingxin Computer Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Yingxin Computer Technology Co Ltd filed Critical Shandong Yingxin Computer Technology Co Ltd
Priority to CN202110236422.9A priority Critical patent/CN113076753A/en
Publication of CN113076753A publication Critical patent/CN113076753A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses an emotion analysis model training optimization method, which comprises the following steps: firstly, acquiring a text and an emotion label, and inputting the text and the emotion label into a self-attention mechanism module of a model; secondly, performing feature fusion on the text and the emotion labels through a self-attention mechanism module to obtain fusion representation identifiers; thirdly, the multilayer perceptron module of the fusion representation identification input model is calculated to obtain the matching degree of the text and the emotion label; optimizing a loss function based on the matching degree to enable the model to reach a convergence state, and obtaining an optimized model; fourthly, performing emotion analysis operation on the input text to be predicted through the optimization model; by the aid of the method, emotion analysis of the text is achieved, and analysis accuracy is improved.

Description

Emotion analysis model training optimization method, system and storage medium
Technical Field
The invention relates to the technical field of emotion analysis, in particular to an emotion analysis model training optimization method, system and storage medium.
Background
Emotion Analysis (Sentiment Analysis) refers to a process of analyzing, processing and extracting subjective texts with emotion colors by utilizing natural language processing and text mining technologies; at present, the text sentiment analysis research covers a plurality of fields including natural language processing, text mining, information retrieval, information extraction, machine learning, ontology and the like, and is concerned by a plurality of scholars and research institutions, and the research is continuously one of the hot problems of the research of the fields of natural language processing and text mining in recent years.
Most of the existing emotion analysis solutions are regarded as a multi-classification task, namely after a text to be analyzed is input, a sentence is expressed into a vector through a BERT model, multi-classification prediction is carried out on the basis of the vector, the probabilities of positive, neutral and negative emotions are output, and finally, the emotion category with the highest probability is taken as an output category; the prior art has the defects that only input text information is used in a representation stage, emotion labels are only used as the basis of probability calculation in a final output layer, and the representation of the emotion labels is lacked, so that emotion analysis is not accurate enough.
Disclosure of Invention
The invention mainly solves the technical problem of providing an emotion analysis model training optimization method, system and storage medium, and can solve the problem of inaccurate emotion analysis caused by lack of expression of emotion labels.
In order to solve the technical problems, the invention adopts a technical scheme that: an emotion analysis model training optimization method is provided, wherein the model comprises a self-attention mechanism module and a multi-layer perceptron module, and the method comprises the following steps:
firstly, acquiring a text and an emotion label, and inputting the text and the emotion label into the self-attention mechanism module of the model;
secondly, performing feature fusion on the text and the emotion label through the self-attention mechanism module to obtain a fusion representation identifier;
inputting the fusion representation identification into the multilayer perceptron module and calculating to obtain the matching degree of the text and the emotion label; optimizing a loss function based on the matching degree to enable the model to reach a convergence state, so as to obtain an optimized model;
and fourthly, performing emotion analysis on the input text to be predicted through the optimization model.
As an improvement, the obtaining the text and the emotion label further comprises the following steps:
the text and emotion labels are concatenated using separators.
Further specifically, the text comprises at least one text and the emotion tag comprises at least one emotion tag.
As an improvement, the feature fusion is performed on the text and the emotion tag through the self-attention mechanism module to obtain a fusion representation identifier, further comprising the following steps:
performing feature fusion on the text through the self-attention mechanism module;
performing feature fusion on the emotion label through the self-attention mechanism module;
and performing feature fusion on the text subjected to feature fusion and the emotion label subjected to feature fusion through the self-attention mechanism module to obtain the fusion representation identifier.
As an improvement, the optimizing the loss function based on the matching degree to make the model reach a convergence state further includes the following steps:
and optimizing a loss function by adopting a gradient back-transmission algorithm based on the matching degree to enable the model to reach a convergence state.
As an improvement, the emotion analysis is performed on the input text to be predicted through the optimization model, and the method further includes the following steps:
acquiring a text to be predicted and a plurality of emotion labels, and outputting the matching degree of the text to be predicted and each emotion label by the optimization model;
and arranging the matching degrees of the text to be predicted and each emotion label in a descending order, and outputting the emotion label corresponding to the first sorted matching degree as the emotion label of the text to be predicted.
The invention also provides an emotion analysis model training optimization system, wherein the model comprises a self-attention mechanism module and a multilayer perceptron module, and the emotion analysis model training optimization system comprises:
a feature fusion unit: the system comprises a self-attention mechanism module, a text and emotion label acquisition module, a self-attention mechanism module and a self-attention mechanism module, wherein the self-attention mechanism module is used for acquiring the text and emotion label;
a model training unit: the fusion representation identifier is input into the multilayer perceptron module and is calculated to obtain the matching degree of the text and the emotion label; optimizing a loss function by adopting a gradient pass-back algorithm based on the matching degree to enable the model to reach a convergence state, so as to obtain an optimized model;
an emotion analysis unit: the method comprises the steps of obtaining a text to be predicted and a plurality of emotion labels, and outputting the matching degree of the text to be predicted and each emotion label through the optimization model; the emotion label sorting module is used for sorting the matching degree of the text to be predicted and each emotion label in a descending order and outputting the emotion label corresponding to the first sorted matching degree.
The invention also provides a computer storage medium for storing computer software instructions for the method for optimizing the training of the emotion analysis model, which comprises a program for executing the method for optimizing the training of the emotion analysis model.
The invention has the beneficial effects that:
1. according to the emotion analysis model training optimization method, the emotion labels and the text are input into the BERT model together, and feature fusion is performed, so that emotion analysis on the text is more accurate.
2. The emotion analysis model training optimization system provided by the invention adopts the feature fusion unit to enable the text information to comprise emotion label information, so that the input information is more sufficient, and the emotion analysis is more accurate.
3. The data transmission storage medium realizes the emotion analysis of the text by executing the emotion analysis model training optimization method.
Drawings
In order to more clearly illustrate the detailed description of the invention or the technical solutions in the prior art, the drawings needed in the detailed description or the prior art description are briefly introduced below; throughout the drawings, like elements or portions are generally identified by like reference numerals; in the drawings, elements or portions are not necessarily drawn to scale.
FIG. 1 is a flowchart of an emotion analysis model training optimization method according to embodiment 1 of the present invention;
FIG. 2 is a schematic view of feature fusion according to example 1 of the present invention;
FIG. 3 is a schematic diagram of an emotion analysis model training optimization system according to embodiment 2 of the present invention.
The parts in the drawings are numbered as follows:
1-feature fusion unit, 2-model training unit, 3-emotion analysis unit and 100-emotion analysis model training optimization system.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the accompanying drawings, and it is obvious that the described embodiments are some, not all embodiments of the present invention; all other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the description of the present invention, it is to be noted that, unless otherwise explicitly specified or limited, the term "connected" is to be interpreted broadly, e.g. as a fixed connection, a detachable connection, or an integral connection; can be mechanically or electrically connected; the two elements can be directly connected or indirectly connected through an intermediate medium, or the two elements can be communicated with each other, or the two elements can be in wireless connection or wired connection; the specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
In the description of the present invention, it should be noted that, for example, MLP (Muti-Layer probability) is a multi-Layer perceptron module, and bert (bidirectional Encoder Representation from transforms) is a bidirectional coding Representation based on a converter, and is a pre-trained language Representation model.
Example 1
This embodiment 1 provides an emotion analysis model training optimization method, as shown in fig. 1, including the following steps:
in the S100 step, connecting the text X and the emotion label L by using a separator, and inputting a BERT model; the BERT model comprises a self-attention mechanism module and a multi-layer perceptron module; combining a text X and an emotion label L into a sample, and inputting three samples because scoring of three emotion labels L is used during optimization training; the three samples were placed in the same group; in this embodiment, the samples that need to be input are arranged as follows:
sample 1: < 'I like sunny days', 'forward';
sample 2: < 'I like sunny days', 'negative direction';
sample 3: < 'I like sunny days', 'neutral' >.
In the step S200, as shown in fig. 2, the text X inside, the emotion tag L inside, the text X and the emotion tag L are fully fused by the attention mechanism module of the BERT model to obtain a fusion representation identifier CLS, so that the text X information is fused with the emotion tag L information, and the emotion tag L information is also fused with the text X information; after the multi-layer fusion representation of the BERT model, the output aspect characteristics of each coding layer of the BERT model are fused, the key semantic characteristics between layers are extracted through the convolutional layers, the influence of redundant information is reduced, the information learned by each coding layer is fully utilized, and finally the text X information and the emotion label L information are fused by the fusion representation identification CLS;
the fused representation identifier CLS may be represented as:
H[CLS]＝BERT(X，L)。
in the step S300, a multi-Layer perceptron Module (MLP) of a BERT model is input into a fusion representation CLS and is calculated to obtain the matching degree (score) of the text X and the emotion label L; the matching degree calculation is expressed as:
Score＝MLP(H[CLS])；
optimizing a loss function by adopting a gradient back-transmission algorithm according to the matching degree to enable the BERT model to reach a convergence state, so as to obtain an optimized BERT model; the loss function is expressed as:
Figure RE-GDA0003056157570000061
wherein, Back _ size is the number of input samples, SlFor the degree of match of the text X and the correct emotion label L, SaAnd SbFor the matching degree of the text X and the other two emotion labels L, minimizing Loss means maximizing the difference between the matching degree of the text X and the correct emotion label L and the matching degree of the text X and the wrong emotion label L, so that the matching degree of the BERT model on the correct emotion label L is improved.
In the step S400, positive, neutral and negative emotion labels L and text X are respectively input into the optimized BERT model to calculate the matching degree of the emotion labels L and the text X, and then the emotion label L with the highest matching degree is taken as the emotion label L of the text X to be output; in this embodiment, to predict the emotional tendency of "i feel good on sunny day", there are three inputs:
input 1: < 'I feel good on sunny days', 'forward';
input 2: < 'I feel good in sunny days', 'neutral';
input 3: < 'I feel good on sunny days', 'negative direction';
and (3) fusing the three inputs in a BERT model to obtain a fusion representation identification CLS, inputting the fusion representation identification CLS into an MLP (maximum likelihood ratio) to calculate the matching degree to obtain [0.8, 0.2 and 0.1], wherein the input 1 has the highest matching degree, and then outputting the emotional tendency of 'I feel good in sunny days' as 'forward'.
Example 2
The embodiment 2 provides an emotion analysis model training optimization system, wherein a BERT model comprises a self-attention mechanism module and a multilayer perceptron module; as shown in fig. 3, the emotion analysis model training optimization system 100 includes:
feature fusion unit 1: the system comprises a BERT model, a feature fusion module, a fusion expression identification and a text information fusion module, wherein the BERT model is used for performing feature fusion on the acquired text and emotion labels through the self-attention mechanism module of the BERT model to obtain fusion expression identification, so that the text information is fused with emotion label information, and the emotion label information is also fused with text information; the fusion representation mark fuses text information and emotion label information;
model training unit 2: a multi-Layer perceptron Module (MLP) for inputting the fusion representation identification into the BERT model and calculating to obtain the matching degree of the text and the emotion label; optimizing a loss function by adopting a gradient back-transmission algorithm based on the matching degree to enable the BERT model to reach a convergence state, so as to obtain an optimized BERT model;
emotion analysis section 3: the method comprises the steps that a text to be predicted and a plurality of emotion labels are input into an optimized BERT model, and the optimized BERT model calculates the matching degree of the text to be predicted and each emotion label; and comparing the matching degree of each emotion label with the text to be predicted, outputting the emotion label corresponding to the highest matching degree as the emotion label of the text to be predicted, and realizing emotion analysis of the text to be predicted.
Example 3
This embodiment 3 provides a computer-readable storage medium, storing computer software instructions for implementing the emotion analysis model training optimization method described in embodiment 1 above, and comprising a program designed for executing the emotion analysis model training optimization method; specifically, the executable program may be built in the emotion analysis model training optimization system 100, so that the emotion analysis model training optimization system 100 may implement the emotion analysis model training optimization method of embodiment 1 by executing the built-in executable program.
Furthermore, the computer-readable storage medium provided by the present embodiments may take any combination of one or more readable storage media, where a readable storage medium includes an electronic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof.
The serial numbers of the embodiments disclosed in the above embodiments are merely for description and do not represent the merits of the embodiments.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes performed by the present specification and drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims (8)

1. An emotion analysis method emotion analysis model training optimization method is characterized in that a model comprises a self-attention mechanism module and a multilayer perceptron module, and the method comprises the following steps:
the self-attention mechanism module is used for acquiring texts and emotion labels and inputting the texts and the emotion labels into the model;
performing feature fusion on the text and the emotion label through the self-attention mechanism module to obtain a fusion representation identifier;
inputting the fusion representation identification into the multilayer perceptron module and calculating to obtain the matching degree of the text and the emotion label;
optimizing a loss function based on the matching degree to enable the model to reach a convergence state, so as to obtain an optimized model;
and performing emotion analysis operation on the input text to be predicted through the optimization model.
2. The emotion analysis model training optimization method of claim 1, wherein the text and the emotion labels are subjected to feature fusion by the attention mechanism module to obtain a fusion representation identifier, further comprising the steps of:
performing feature fusion on the text through the self-attention mechanism module;
performing feature fusion on the emotion label through the self-attention mechanism module;
and performing feature fusion on the text subjected to feature fusion and the emotion label subjected to feature fusion through the self-attention mechanism module to obtain the fusion representation identifier.
3. The method for optimizing emotion analysis model training according to claim 1, wherein the optimizing a loss function based on the degree of matching brings the model to a converged state, further comprising the steps of:
and optimizing a loss function by adopting a gradient pass-back algorithm based on the matching degree to enable the model to reach a convergence state.
4. The method for optimizing training of emotion analysis models according to claim 1, wherein the emotion analysis operation is performed on the inputted text to be predicted through the optimization model, and further comprising the steps of:
acquiring a text to be predicted and a plurality of emotion labels, and outputting the matching degree of the text to be predicted and each emotion label by the optimization model;
and arranging the matching degrees of the text to be predicted and each emotion label in a descending order, and outputting the emotion label corresponding to the first sorted matching degree as the emotion label of the text to be predicted.
5. The emotion analysis model training optimization method of claim 1, wherein the obtaining of the text and emotion labels further comprises the steps of:
the text and emotion labels are concatenated using separators.
6. The emotion analysis model training optimization method of claim 5, wherein the text comprises at least one text, and the emotion labels comprise at least one emotion label.
7. An emotion analysis model training optimization system, wherein a model comprises a self-attention mechanism module and a multi-layer perceptron module, the system comprising:
a feature fusion unit: the system comprises a self-attention mechanism module, a text and emotion label acquisition module, a self-attention mechanism module and a self-attention mechanism module, wherein the self-attention mechanism module is used for acquiring the text and emotion label;
a model training unit: the fusion representation identifier is input into the multilayer perceptron module and is calculated to obtain the matching degree of the text and the emotion label; optimizing a loss function by adopting a gradient pass-back algorithm based on the matching degree to enable the model to reach a convergence state, so as to obtain an optimized model;
an emotion analysis unit: the method comprises the steps of obtaining a text to be predicted and a plurality of emotion labels, and outputting the matching degree of the text to be predicted and each emotion label through the optimization model; the emotion label sorting module is used for sorting the matching degree of the text to be predicted and each emotion label in a descending order and outputting the emotion label corresponding to the first sorted matching degree.
8. A computer storage medium storing computer software instructions for a method for optimizing training of an emotion analysis model according to any one of claims 1 to 6, the computer software instructions comprising a program for executing the method for optimizing training of an emotion analysis model.
CN202110236422.9A 2021-03-03 2021-03-03 Emotion analysis model training optimization method, system and storage medium Pending CN113076753A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110236422.9A CN113076753A (en) 2021-03-03 2021-03-03 Emotion analysis model training optimization method, system and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110236422.9A CN113076753A (en) 2021-03-03 2021-03-03 Emotion analysis model training optimization method, system and storage medium

Publications (1)

Publication Number Publication Date
CN113076753A true CN113076753A (en) 2021-07-06

Family

ID=76609834

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110236422.9A Pending CN113076753A (en) 2021-03-03 2021-03-03 Emotion analysis model training optimization method, system and storage medium

Country Status (1)

Country Link
CN (1) CN113076753A (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109492101A (en) * 2018-11-01 2019-03-19 山东大学 File classification method, system and medium based on label information and text feature
US20190163742A1 (en) * 2017-11-28 2019-05-30 Baidu Online Network Technology (Beijing) Co., Ltd. Method and apparatus for generating information
CN111680154A (en) * 2020-04-13 2020-09-18 华东师范大学 Comment text attribute level emotion analysis method based on deep learning
CN112307757A (en) * 2020-10-28 2021-02-02 中国平安人寿保险股份有限公司 Emotion analysis method, device and equipment based on auxiliary task and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190163742A1 (en) * 2017-11-28 2019-05-30 Baidu Online Network Technology (Beijing) Co., Ltd. Method and apparatus for generating information
CN109492101A (en) * 2018-11-01 2019-03-19 山东大学 File classification method, system and medium based on label information and text feature
CN111680154A (en) * 2020-04-13 2020-09-18 华东师范大学 Comment text attribute level emotion analysis method based on deep learning
CN112307757A (en) * 2020-10-28 2021-02-02 中国平安人寿保险股份有限公司 Emotion analysis method, device and equipment based on auxiliary task and storage medium

Similar Documents

Publication Publication Date Title
CN107133220B (en) Geographic science field named entity identification method
CN107992597B (en) Text structuring method for power grid fault case
WO2018028077A1 (en) Deep learning based method and device for chinese semantics analysis
CN110210037A (en) Category detection method towards evidence-based medicine EBM field
CN109359293A (en) Mongolian name entity recognition method neural network based and its identifying system
CN110188202A (en) Training method, device and the terminal of semantic relation identification model
CN110750974B (en) Method and system for structured processing of referee document
CN110263325A (en) Chinese automatic word-cut
CN110532398A (en) Family&#39;s map method for auto constructing based on multitask united NNs model
CN117151222B (en) Domain knowledge guided emergency case entity attribute and relation extraction method thereof, electronic equipment and storage medium
CN112784601B (en) Key information extraction method, device, electronic equipment and storage medium
CN115017266A (en) Scene text retrieval model and method based on text detection and semantic matching and computer equipment
CN114153971A (en) Error-containing Chinese text error correction, identification and classification equipment
CN108536781B (en) Social network emotion focus mining method and system
CN115168541A (en) Chapter event extraction method and system based on frame semantic mapping and type perception
CN110297986A (en) A kind of Sentiment orientation analysis method of hot microblog topic
CN117828105A (en) Electric power defect text knowledge graph construction method based on joint extraction
CN116522945A (en) Model and method for identifying named entities in food safety field
CN115392254A (en) Interpretable cognitive prediction and discrimination method and system based on target task
CN111783464A (en) Electric power-oriented domain entity identification method, system and storage medium
CN117972121B (en) Relation extraction method based on multi-mode large language model
CN114756678A (en) Unknown intention text identification method and device
CN114388108A (en) User feedback analysis method based on multi-task learning
CN112905750A (en) Generation method and device of optimization model
CN116702753A (en) Text emotion analysis method based on graph attention network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210706