CN114580430A

CN114580430A - Method for extracting fish disease description emotion words based on neural network

Info

Publication number: CN114580430A
Application number: CN202210172472.XA
Authority: CN
Inventors: 张思佳; 吴杰; 丛子涵; 姜鑫; 于英囡; 孙华; 刘明剑
Original assignee: Dalian Ocean University
Current assignee: Dalian Ocean University
Priority date: 2022-02-24
Filing date: 2022-02-24
Publication date: 2022-06-03
Anticipated expiration: 2042-02-24
Also published as: CN114580430B

Abstract

A fish disease description emotion word extraction method based on a neural network belongs to the technical field of emotion word analysis. The method is based on the prior knowledge, and learns the part of emotional knowledge in the text semantic information through a neural network, thereby assisting remote disease diagnosis. Specifically, a series of fish disease descriptions provided by a user are input, then manually marked fish disease aspect categories and emotion polarities are added to form a data set, the data set is transmitted into a pre-training model and is converted into word vectors, and the word vectors are transmitted into a sequence model to process a time sequence relation in sentences. And finally, transmitting the processed semantic information into a classification model to finish the extraction and analysis of the emotional words in the fish disease description. Compared with the existing fish disease diagnosis method based on the expert system, the method is used for extracting the emotional word part in the semantic information in order to reduce the dependence on the prior knowledge and the rules.

Description

Method for extracting fish disease description emotion words based on neural network

Technical Field

The invention relates to the technical field of sentiment word analysis based on aspects, in particular to a fish disease description sentiment word extraction method based on a neural network.

Background

With the development of computer technology, the number of people using the internet is increasing at a high speed, and by far, global mobile phone users exceed 50 hundred million, and internet users reach 45 hundred million. Among these are 42 billion social media users. These numbers account for the majority of the world's general population. It is conceivable that the internet will produce an unthinkable quantity every day. The data of the level gives the opportunity of the rapid development of artificial intelligence, and meanwhile, the artificial intelligence also deeply changes the life style of people.

Natural Language Processing (NLP) is an important direction in the field of artificial intelligence, and is a method for studying how to make a machine understand the discipline of human Language and to realize effective interaction between human and computer by Natural Language. Meanwhile, with the increase of the number of network users, more and more people can publish their own opinions on the social platform to share their own ideas. Thus, there are numerous languages with emotional colors and tendencies on a variety of open platforms. Analysis of these statements is of great significance to reality. The method can predict the preference of the client and the emotional mood of people, and can estimate the risk. Therefore, the emotion analysis task is very critical at the present stage.

In recent years, emotion analysis has become one of the most active research directions in NLP, and has been widely used in information retrieval and text mining. Because the Internet is an important social platform for expression and sharing, the Internet brings rich topics containing emotional tendencies to users. The text sentiment analysis is a process of analyzing, processing, inducing and reasoning the texts with sentiment tendency. Where the aspect-based sentiment analysis subtask helps merchants and businesses obtain valuable feedback information to improve their products. To date, analysis of emotion based on aspects has been largely based on text data sets in the conventional field. Few people pay attention to texts containing human emotional tendencies in the professional field such as fish diseases, but the more and more adjectives and degree words which are used for expressing the viewpoints and ideas of professionals in the text description. These fish disease descriptors with a large number of emotional tendencies enable us to perform emotion extraction and analysis in different aspects.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provide a semantic analysis method for fish disease description by means of a neural network, and particularly focuses on emotional words in a text. The method carries out text vectorization based on a pre-training model, a sequence model processes text time sequence relation, and a classification model completes aspect and emotion polarity prediction. And in data processing, a special data iterator is arranged and conforms to a model input format. At the output, the output results are a predefined set of aspects and a set of emotion polarities. In this process, the data preparation and annotation process is performed manually, and the rest is performed automatically.

The invention provides a fish disease description emotion word extraction method based on a neural network, which comprises the following specific implementation processes:

the first step is as follows: and performing aspect and emotion polarity division based on an offline fish disease diagnosis process. Firstly, according to the practical basis, the clinical manifestations are the main and the spatio-temporal factors are the auxiliary in the diagnosis process. We therefore classify the description of fish diseases into two major categories: 1. clinical factors, 2. spatiotemporal factors. And then, specifically analyzing the characteristics of the collected text data and subdividing the characteristics, wherein clinical factors comprise five aspects of body surface, body, posture, physique and gill, and space-time factors comprise two parts of environment and time. Then, dividing emotion polarities, after referring to text data, we find that since the real problem is fish disease diagnosis, the text is neutral or negative description, so that in combination with a specific problem, we divide emotion polarities into four types of positive, neutral, negative and negative. In summary, the fish disease description consisted of 7 aspects, and emotional polarity consisted of four degrees.

Environment: the fish is in the surrounding environment, water body environment, geographical environment and the like.

Time-saving: refers to the season, month and morning and evening of the fish.

Body surface: refers to the manifestation of disorders on the skin and oral characteristics.

In vivo: refers to the internal features of fish, such as intestines and stomach, viscera.

The posture is as follows: refers to the external manifestations of fish, such as: food intake manifestation, active manifestation, etc

Physical constitution: refers to the body length, weight, fat and thin of fish

Fish gill: refers to the state of the gill of a fish.

The second step is that: the method is characterized by processing a data set, preprocessing collected fish disease descriptions, and removing blank spaces and non-Chinese characters. And then manually marking the category and the emotion polarity, wherein the method adopts the method that three persons mark the same data set, and the marking result is determined by the number of votes obtained. After that, the data set is processed in three aspects of data distribution, data label distribution and correlation coefficient in the data group. Finally, the whole data set is processed according to the following steps of 6: 2: and 2, dividing the training set, the verification set and the test set, and completing the test.

The third step: and combining a neural network to carry out text vectorization, text time sequence processing and classification prediction.

A. Firstly, three data sets are integrated into three data iterators, the configuration of the iterators is transmitted according to the specification of the BERT model (the sentence length required by the Chinese BERT model is 32 at most), and meanwhile, the calculation power provided by the equipment is considered, the size of the Batch is set to be 8, namely, only 8 sentences are transmitted at a time. After passing through the BERT model, the text becomes converted to a text vector.

B. Because of the text vector, which contains a large amount of time sequence information, we need to transfer the converted text vector into a sequence model (BilSTM + Attention) for processing, and meanwhile, the parameters of the model are continuously optimized. In the device algorithm considered, our bar BilSTM hidden layer size was set to 512.

C. Finally, since the problem itself is a task of emotion analysis, belonging to the category of classification task, we select a common layer after the sequence model as the classification layer.

The fourth step: in the specific application process, the whole model reaches a state with optimal parameters after being trained. At the moment, the specific description of the diseased fish can be transmitted into the model according to sentences, the model can output the aspect and emotional color of the description of the diseased fish, and the semantic information which is just the description of the diseased fish is obtained and used for assisting the diagnosis of fish diseases.

The invention has the beneficial effects that: the existing remote fish disease diagnosis method depends on the prior knowledge of experts and the system rule establishment. The process ignores the data information, and the invention aims to extract the emotional information part in the data information. The combination of a neural network on the basis of the existing system enables automatic recognition of aspects and emotional polarities in the fish disease description. Compared with the existing method, the invention optimizes the existing fish disease diagnosis expert system and fills the blank of fish disease diagnosis based on semantic information. Meanwhile, the invention also reduces the manual participation and improves the efficiency. We found that 81% -84% of the effect can be achieved when we train the classifier by using a simple BERT + feature extraction mode, which proves that the data set proposed by us has high quality.

Drawings

FIG. 1 is a flow chart of a method for extracting fish disease description emotion words based on a neural network.

FIG. 2 is a model structure diagram of a fish disease description emotion word extraction method based on a neural network.

Detailed Description

The invention is described in further detail below with reference to the figures and specific examples.

As shown in fig. 1, the invention provides a method for describing emotional words in fish diseases based on a neural network, which comprises the following steps:

the first step is as follows: and performing aspect and emotion polarity division based on the offline fish disease diagnosis process. Firstly, according to the practical basis, the clinical manifestations are the main and the space-time factors are the auxiliary in the diagnosis process. We therefore classify the description of fish diseases into two major categories: 1. clinical factors, 2. spatiotemporal factors. And then, specifically analyzing the characteristics of the collected text data and subdividing the text data, wherein clinical factors comprise five aspects of body surface, body, posture, physique and gill, and space-time factors comprise two parts of environment and time. Then, dividing emotion polarities, after referring to text data, we find that since the real problem is fish disease diagnosis, the text is neutral or negative description, so that in combination with a specific problem, we divide emotion polarities into four types of positive, neutral, negative and negative. In summary, the fish disease description consists of 7 aspects, and the emotional polarity consists of four degrees, and the specific data are shown in table three.

The second step is that: the method is characterized by processing a data set, preprocessing collected fish disease descriptions, and removing blank spaces and non-Chinese characters. And then manually marking the category and the emotion polarity, wherein the method adopts the method that three persons mark the same data set, and the marking result is determined by the number of votes obtained. After that, data analysis is carried out on the data set from three aspects of data distribution, data label distribution and correlation coefficient in the data set. Finally, the whole data set is processed according to the following steps of 6: 2: and 2, dividing the training set, the verification set and the test set, and completing all manual labeling work at the moment.

The third step: a fish disease description emotion word method model based on a neural network is composed of three parts: firstly, a semantic embedding layer is used for obtaining vectorized text representation; a semantic decision layer, which obtains deep semantic information through a sequence model; and thirdly, a classification layer is used for predicting the emotion category and the emotion polarity. The method specifically comprises the following steps:

A. firstly, three data sets are integrated into three data iterators, the configuration of the iterators is transmitted according to the specification of a BERT model (the sentence length required by the Chinese BERT model is 32 at most), and the size of Batch is set to be the size of Batch according to the calculation power provided by equipment8, i.e. only 8 sentences are incoming at a time. Then, the data in the iterator is transmitted into the BERT model to be converted into a text vector. The specific operation steps are as follows: first, a text sequence T ═ T { T } of the size of Batch is introduced₀,t₁,t₂,t₃,t₄,t₅,t₆,t₇W for each t ═ w₁,w₂,w3,w₄,., wn consists of n words. Then each T in the text sequence T is mapped to a vector space with fixed dimension by using BERT (bidirectional Encoder registration from transformations), so as to obtain the initial word embedding, then each w in the T obtains a segment embedding and a position embedding according to which sentence the T belongs to and the position in the sentence, and finally the three vectors are added to obtain the input vector x of each w. There are a total of 12 transforms in Bert, with an Encoder and Decoder in each transform respectively encoding and decoding the incoming word vector. The encoding and decoding process is continuously passed through a feedforward neural network and normalization. Finally, we get a 768-dimensional word vector at the output.

hi＝Bert(xi) i∈(1,N) (1)

Wherein h is_iIs a word vector representation at the output, Hi ═ h_i1,h_i2,...,h_inDenotes the feature vector of the sentence. N denotes the sentence length.

B. The layer is a semantic decision layer, and because the layer is a text vector which contains a large amount of time sequence information, the text vector after conversion needs to be transmitted into a sequence model (BilSTM + Attention) for processing, and meanwhile, parameters of the model are continuously optimized. At the facility of consideration, we set the hidden layer size of BilSTM to 512. It contains a forward LSTM and a backward LSTM, which respectively convert the text from w₁To w_nAnd w_nTo w₁Obtaining two LSTM feature representations

Forward direction Li ═ forward direction lstm (hi) i ∈ (1, N) (2)

Reverse Li ═ reverse lstm (hi) i ∈ (N,1) (3)

Finally at the output we obtain a BilSTM signature for each w:

bi ═ LSTM in the positive direction, LSTM in the negative direction i ∈ (1, N) (4)

Wherein Bi is the characteristic representation of the text after being subjected to BilSTM, and N represents the sentence length.

Then the whole sentence is transmitted into the Attention module, and the weight distribution is automatically carried out.

A＝∑Bi*Hi i∈(1,N) (5)

Where Bi represents a feature representation of the text, H_iA representation of the attention for each word in the text is calculated.

C. Finally, since the problem itself is a task of emotion analysis, belonging to the category of classification task, we select a common layer after the sequence model as the classification layer. This layer predicts the likelihood of aspect categories and emotional polarity through the fully connected layer.

out1，out2＝Linear(A) out1∈U，out2∈Q (6)

Where out1 and out2 are the aspect and emotion outputs, respectively, U is the aspect class and Q is the emotion class.

The fourth step: and (3) training a model, namely calculating the cross entropy of yR and T, and then completing model training by optimizing the cross entropy. Let the BERT parameter matrix W_bertBiLSTM + Attention parameter matrix W_biAnd W_atAnd parameter matrix W of Linear_LAnd (4) optimizing. After that, the whole model can automatically label the aspect and emotion polarity in the fish disease description for assisting fish disease diagnosis.

Fifth step, experimental verification:

1) experimental Environment and data

TABLE 1 extraction method data set of fish disease description emotion words based on neural network

TABLE 2 comparison of sentiment analysis data sets based on aspects

	Year of year	Data volume	Aspect(s)	FIELD	Professional field
						SemEval-2014	2014	7686	5	2	×
SemEval-2015	2015	4766	26	3	×
						SemEval-2016	2016	5984	28	2	×
SentiHood	2016	5215	2	1	×
						Cricket review	2018	3034	5	1	×
Our	2021	2300	5	1	√

We constructed a new aspect level sentiment analysis data set for fish disease description. Table 2 shows a comparison of the disclosed aspect-level sentiment analysis with our proposed data set. One problem with these existing datasets is that they all belong to the general domain, without data information in the professional domain. Another problem is that the existing domains do not guarantee the generalization capability required in the description of fish diseases. To address the previous shortcomings, we propose this data set, which consists of 1 domain (fish disease) and 7 aspects. Our data set is the only data set that contains knowledge of the fish disease domain.

To evaluate the model and experimental results, we used two indices of accuracy (Acc) and F1.

Where Rec is recall, All is All and correct is correct-predicted.

The hyper-parameters for the model are as follows: the position embedding and weight matrices are initialized with a uniform distribution U (-0.1,0.1) and the offset is initialized to zero. drop _ out is set to 0.3 and the learning rate is 10^-5The size of the Bert hidden layer is 768, the number of the BilSTM hidden layers is 256, and the number of the layers is 1. The model adopts a BertAdam optimizer, and the learning rate of the BertAdam optimizer is also 10^-5. And stopping when the maximum training times of the model are 1000 times and the model is lifted.

2) Comparison model

In order to evaluate the performance of our proposed model. In the example, the experimental results are compared with 3 baseline models, and meanwhile, the baseline models are briefly introduced.

First are three baseline models:

bert + Linear: the sentence is coded only by Bert to obtain a word vector, and the word vector is used as a characteristic representation of the sentence and is transmitted into a full connection layer for training. The advantage is that the model is simple, and the disadvantage is that the model is difficult to fit to the ideal state.

And (5) Bert + BilSTM, namely, coding the sentence through Bert to obtain a word vector, then transmitting the word vector into the BilSTM to obtain a feature representation of the sentence, and then transmitting the word vector into the full-connection layer. The model can be optimized simply subsequently, and the BilSTM processing sequence task is more reasonable. The disadvantage is that the feature representation cannot solve the problem of dependency between sentence words.

And Bert + CNN, namely, coding the sentence through Bert to obtain a word vector, then transmitting the word vector into CNN to obtain a feature representation of the sentence, and then transmitting the feature representation into a full-link layer to train. The model has the advantages that internal shared parameters can be realized by means of convolution and pooling, and training is simplified. The disadvantage is limited by the convolution kernel size, which makes it difficult to capture long range features. The experimental results are as follows:

TABLE 3 AE tasks ACC and F1 for each model

Model	ACC	F1
			Bert+Linear	81.96％	82.08％
Bert+BiLSTM	82.49％	82.55％
			Bert+CNN	83.00％	83.04％
Bert+BiLSTM+ATT	83.55％	83.64％

3) Analysis of results

Firstly, it is obvious that the feature representation of BiLSTM or CNN is better than that of the word vector after Bert coding.

(2) Even when dealing with sequential tasks, CNN's performance on certain specific texts is better than BiLSTM.

(3) The effect of our proposed model is entirely superior to the other three, which illustrates the effectiveness of our introduction of Attention.

(4) From table 3, we found that we can achieve 81% -84% of the effect when we train the classifier using simple BERT + feature extraction, which proves that our proposed data set is of high quality.

The above description is only exemplary of the preferred embodiments of the present invention, and is not intended to limit the present invention, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A fish disease description emotion word extraction method based on a neural network is characterized by mainly comprising the following steps:

s1, dividing aspect categories and emotion polarities based on an offline fish disease diagnosis process;

s11, aspect category division

According to the premise that clinical manifestations are dominant and space-time factors are subordinate in the diagnosis process, description of fish diseases is divided into two main categories: clinical and spatiotemporal factors;

analyzing the collected text data characteristics specifically and subdividing;

the clinical factors comprise five aspects of body surface, body, posture, physique and gill, and the space-time factors comprise two parts of environment and time;

s13, sentiment polarity division

Combining the reference text with specific questions, and dividing the emotion polarities into positive, neutral, partial negative and negative; therefore, the described aspects of fish diseases comprise 7 aspects of body surface, body state, physique, gill, environment and time, and the emotional polarity comprises four categories of positive, neutral, partial negative and negative;

s2, processing the data set:

s21, preprocessing the collected fish disease description, and removing blank spaces and non-Chinese characters;

s22, manually marking the category and the emotion polarity in the aspect, marking the same data set by three persons, and determining the marking result by the number of votes;

s23, data analysis is carried out on the data set in the aspects of data distribution, data annotation distribution and correlation coefficients in the data set, and the data set is processed according to the following steps of 6: 2: 2, dividing the training set, the verification set and the test set;

s3, the fish disease description emotion word method model based on the neural network comprises three parts: firstly, a semantic embedding layer is used for obtaining vectorized text representation; a semantic decision layer, which obtains deep semantic information through a sequence model; thirdly, a classification layer is used for predicting emotion types and emotion polarities; the method specifically comprises the following steps:

s31, semantic embedding layer: integrating three data sets of a training set, a verification set and a test set into three data iterators, transmitting the configuration of the iterators according to the specification of a BERT model, and setting Batch to be 8, namely transmitting 8 sentences at a time; transmitting the data in the iterator into a BERT model to be converted into a text vector, and specifically comprising the following operation steps:

(1) a text sequence T ═ T { T } of the size of Batch is transmitted into₀，t₁，t₂，t₃，t₄，t₅，t₆，t₇W for each t ═ w₁，w₂，w₃，w₄,., wn consists of n words;

(2) each text sequence T in the text sequence T can be mapped to a vector space with fixed dimension by using BERT to obtain initial word embedding of the text sequence T, and each phrase w in the text sequence T can obtain segment embedding and position embedding according to the sentence to which the phrase w belongs and the position in the sentence;

(3) adding the three vectors to obtain an input vector x of each phrase w; there are 12 layers of transformers in total in Bert, there are Encoder and Decode in each transformer to encode and decode the vector of the incoming word separately; continuously passing through a feedforward neural network and normalization in the encoding and decoding processes; finally, a 768-dimensional word vector is obtained at the output end;

hi＝Bert(xi)i∈(1，N) (1)

wherein h is_iIs a word vector representation at the output, N represents the sentence length;

s32, semantic decision layer: because the text vector contains time sequence information, the converted text vector is transmitted into a sequence model BilSTM + Attention for processing, and meanwhile, the parameters of the model are continuously optimized;

the hidden layer size of BilSTM is set to 512, which comprises a forward LSTM and a backward LSTM, and text is respectively read from w₁To w_nAnd w_nTo w₁Obtaining two LSTM feature representations

Forward direction Li ═ forward direction lstm (hi) i ∈ (1, N) (2)

Reverse Li ═ reverse lstm (hi) i ∈ (N,1) (3)

Finally, the BilSTM feature representation of each w is obtained at the output end:

Wherein Bi is the characteristic representation of the text after being subjected to the BilSTM, and N represents the sentence length;

then the whole sentence is transmitted into an Attention module, and weight distribution is automatically carried out;

A＝∑Bi*Hii∈(1，N) (5)

where Bi represents a feature representation of the text, H_iCalculating a representation of the attention of each word in the text, Hi ═ { Hi1, Hi 2.., hin } represents a feature vector of the sentence;

s33, classification layer: because the problem is an emotion analysis task and belongs to the category of classification tasks, a layer is selected as a classification layer behind a sequence model; the layer predicts the possibility of aspect category and emotion polarity through the full connection layer;

out1，out2＝Linear(A) out1∈U，out2∈Q (6)

wherein, linear (a) is layer, out1 and out2 are aspect and emotion output, respectively, U is aspect category, and Q is emotion category;

s4, training a model: calculating y^RAnd the cross entropy of the T, and completing model training by optimizing the cross entropy; let the BERT parameter matrix W_bertBiLSTM + Attention parameter matrix W_biAnd W_atAnd parameter matrix W of Linear_LOptimizing;

wherein, y^RAnd T is respectively expressed as a prediction label and a real label, and L is a parameter matrix of the whole model;

and S5, specific descriptions of the diseased fish are transmitted into the trained model according to sentences, the model can output the aspect categories and emotional colors of the descriptions of the diseased fish, and semantic information of the descriptions of the diseased fish is obtained and used for assisting fish disease diagnosis.