CN103345525B - File classification method, device and processor - Google Patents

Abstract

The present invention relates to natural language processing and mode identification technology, disclose a kind of file classification method, device and processor.In the method, first the probability of each emotional category and each emotion classification it is belonging respectively to according to grader acquisition emotion test sample and mood test sample, then emotion test sample and mood test sample are chosen as common sample, and obtain the joint probability of described common sample, described joint probability and probability are weighted summation, and obtain the emotional category belonging to text to be sorted, and/or emotion classification according to the result of summation.During this, emotion and the emotion of text to be sorted are combined, to realize the emotional semantic classification to text, and/or emotion classification.Owing to there being close contact between emotion and the emotion of one text, by the emotion of text to be sorted and emotion associating, with treat classifying text classify time, the precision of text classification can be effectively improved.

Description

File classification method, device and processor

Technical field

The present invention relates to natural language processing and mode identification technology, particularly relate to a kind of text and divide Class method, device and processor.

Background technology

In the activity of the mankind, people is to the attitude of things or viewpoint, often by emotion and the shadow of emotion Ring.Wherein, emotion is whether people meets oneself needs to objective things and a kind of attitude of producing is experienced, Including positive, intermediate or passive etc., and emotion refer in people psychoreaction and impression, such as happiness, Anger, sorrow and pleasure etc..Emotional semantic classification contributes to the pluses and minuses of consumption habit and the product understanding user, in order to Product review is analyzed and decision-making；Contribute to understanding satisfaction and the demand of the common people, find society in time Can hot issue；Contribute to analyzing the focus public feelings information of current social, to user, enterprise or government's machine Structures etc. provide the foundation of decision references.And emotion has potential field, theme and the spy of independence in period By the emotion of people, point, is able to observe that the viewpoint of things is inclined to by people.

In view of this, it is often necessary to text is carried out emotional semantic classification or emotion classification.Existing emotional semantic classification Method and emotion sorting technique, mainly after obtaining text to be sorted, carry out feelings to emotion test sample Sense mark, carries out emotion mark, and treats respectively according to the test sample after mark mood test sample The text of classification carries out emotional semantic classification or emotion classification respectively.

But, inventor finds in the research process of the application, uses aforesaid way to treat classifying text When carrying out emotional semantic classification or emotion classification, the classification results precision got is relatively low.

Summary of the invention

In view of this, it is an object of the invention to provide a kind of file classification method, device and processor, Specific embodiments is as follows:

A kind of file classification method, including:

Obtaining grader, described grader includes: emotion classifiers and emotion grader；

Obtain emotion test sample and the mood test sample of text to be sorted, according to described grader, right Described emotion test sample and mood test sample are classified, and obtain described emotion test sample and feelings Thread test sample is belonging respectively to the probability of each emotional category and each emotion classification；

Choose emotion test sample and mood test sample as common sample, and obtain described common sample Joint probability, described joint probability includes: emotion joint probability, and/or emotion joint probability, wherein, Described emotion joint probability p (s_i|e_j) representing: the emotion classification of certain sample is e_jIn the case of, its emotion Classification is s_iProbability, described emotion joint probability p (e_i|s_j) representing: the emotional category of certain sample is s_j's In the case of, its emotion classification is e_iProbability；

Described probability and joint probability are weighted summation, and obtain according to the result of weighted sum Emotional category belonging to text to be sorted, and/or emotion classification.

Preferably, described probability and joint probability are weighted summation, and according to weighted sum Result obtains the emotional category belonging to text to be sorted, including:

Probability according to described common sample and emotion joint probability, obtain the feelings of each common sample Sense transition probability, wherein, the algorithm obtaining described transference probability is:

$p_{\mathrm{reansfer}}\left(s_i\right|X)=\underset{j=1}{\overset{N_e}{\mathrm{\Σ}}}p\left(s_i\right|e_i)*p\left(e_i\right|X);$

Wherein, p (e_j| X) it is the common sample X probability that belongs to all kinds of emotion classification, p (s_i|e_j) it is Emotion joint probability, the emotion classification being certain common sample is e_jTime, emotional category is s_iProbability, N_eFor the emotion classification number of emotion classification, p_transfer(s_i| X) it is the transference probability of sample X；

According to described transference probability, obtain the final emotion probability of each common sample, wherein, obtain The algorithm taking described final emotion probability is:

p_joint(s_i|X)=(1-λ)p(s_i|X)+λp_transfer(s_i|X)；

Wherein, p (s_i| X) it is the sample X probability that belongs to all kinds of emotional category, p_transfer(s_i| X) it is sample The transference probability of X, λ is the weight parameter set, p_joint(s_i| X) it is final emotion probability；

Obtaining in described common sample, the common sample that the value of final emotion probability is maximum, as feelings Sample is demarcated in sense, and determines that described emotion demarcation emotional category belonging to sample is described text institute to be sorted The emotional category belonged to.

Preferably, described probability and joint probability are weighted summation, and according to weighted sum Result obtains the emotion classification belonging to text to be sorted, including:

Probability according to described common sample and joint probability, the emotion obtaining each common sample turns Moving probability, wherein, the algorithm obtaining described emotion transition probability is:

$p_{\mathrm{reansfer}}\left(e_i\right|X)=\underset{j=1}{\overset{N_s}{\mathrm{\Σ}}}p\left(e_i\right|s_j)*p\left(s_j\right|X)$

Wherein, p (s_j| X) it is the common sample X probability that belongs to all kinds of emotional category, p (e_i|s_j) it is feelings Thread joint probability, the emotional category being certain common sample is s_jTime, emotion classification is e_iProbability, N_s For the emotional category number of emotional semantic classification, p_transfer(e_i| X) it is the emotion transition probability of sample X；

According to described emotion transition probability, obtain the final emotion probability of each common sample, wherein, obtain The algorithm taking described final emotion probability is:

p_joint(e_i|X)=(1-λ)p(e_i|X)+λp_transfer(e_i|X)；

Wherein, p (e_i| X) it is the sample X probability that belongs to all kinds of emotion classification, p_transfer(e_i| X) be The emotion transition probability of sample X, λ is the weight parameter set, p_joint(e_i| X) it is final emotion probability；

Obtaining in described common sample, the common sample that the value of final emotion probability is maximum, as feelings Thread demarcates sample, and determines that described emotion demarcation emotion classification belonging to sample is described text institute to be sorted The emotion classification belonged to.

Preferably,

The interval of described weight parameter λ is: (0.4,0.8).

Preferably, described acquisition grader includes:

Obtain respectively and there is Emotion tagging and the corpus of emotion mark；

Described emotion classifiers is obtained according to the described corpus with Emotion tagging, and according to described tool The corpus marked of being in a bad mood obtains described emotion grader.

Preferably,

Described emotional category includes: positive, neutral and passive；

Described emotion classification includes: glad, sad, worried, angry, surprised, angry, loss of emotion.

Accordingly, the invention also discloses a kind of document sorting apparatus, including:

Grader acquisition module, is used for obtaining grader, and described grader includes: emotion classifiers and feelings Thread grader；

Probability acquisition module, for obtaining emotion test sample and the mood test sample of text to be sorted This, according to described grader, classify to emotion test sample and mood test sample, and obtain institute State emotion test sample and mood test sample is belonging respectively at the beginning of each emotional category and each emotion classification Beginning probability；

Joint probability acquisition module, is used for choosing emotion test sample and mood test sample as common sample Basis, and obtain the joint probability of described common sample, described joint probability includes: emotion joint probability, And/or emotion joint probability, wherein, described emotion joint probability p (s_i|e_j) represent: the emotion of certain sample Classification is e_jIn the case of, its emotional category is s_iProbability, described emotion joint probability p (e_i|s_j) represent: The emotional category of certain sample is s_jIn the case of, its emotion classification is e_iProbability；

Text categories acquisition module, for described probability and joint probability being weighted summation, and Result according to weighted sum obtains the emotional category belonging to text to be sorted, and/or emotion classification.

Preferably, described text categories acquisition module includes: emotional category obtains submodule, described emotion Classification obtains submodule and includes:

Transference probability acquiring unit, for the probability according to described common sample and emotion associating Probability, obtains the transference probability of each common sample, wherein, obtains described transference probability Algorithm is:

$p_{\mathrm{reansfer}}\left(s_i\right|X)=\underset{j=1}{\overset{N_e}{\mathrm{\Σ}}}p\left(s_i\right|e_i)*p\left(e_j\right|X)$

Wherein, p (e_j| X) it is the common sample X probability that belongs to all kinds of emotion classification, p (s_i|e_j) it is Emotion joint probability, the emotion classification being certain common sample is e_jTime, emotional category is s_iProbability, N_eFor the emotion classification number of emotion classification, p_transfer(s_i| X) it is the transference probability of sample X；

Final emotion probability acquiring unit, for according to described transference probability, obtains each common sample This final emotion probability, wherein, the algorithm obtaining described final emotion probability is:

p_joint(s_i|X)=(1-λ)p(s_i|X)+λp_transfer(s_i|X)；

Wherein, p (s_i| X) it is the sample X probability that belongs to all kinds of emotional category, p_transfer(s_i| X) it is sample The transference probability of X, λ is the weight parameter set, p_joint(s_i| X) it is final emotion probability；

Emotional category determines unit, is used for obtaining in described common sample, and the value of final emotion probability is maximum Common sample, demarcate sample as emotion, and determine that described emotion demarcates emotion belonging to sample Classification is the emotional category belonging to described text to be sorted.

Preferably, described text categories acquisition module includes: emotion classification obtains submodule, described emotion Classification obtains submodule and includes:

Emotion transition probability acquiring unit, for the probability according to described common sample and joint probability, Obtaining the emotion transition probability of each common sample, wherein, the algorithm obtaining described emotion transition probability is:

$p_{\mathrm{transfer}}\left(e_i\right|X)=\underset{j=1}{\overset{N_s}{\mathrm{\Σ}}}p\left(e_i\right|s_j)*p\left(s_j\right|X)$

Wherein, p (s_j| X) it is the common sample X probability that belongs to all kinds of emotional category, p (e_i|s_j) it is feelings Thread joint probability, the emotional category being certain common sample is s_jTime, emotion classification is e_iProbability, N_s For the emotional category number of emotional semantic classification, p_transfer(e_i| X) it is the emotion transition probability of sample X；

Final emotion probability acquiring unit, for according to described emotion transition probability, obtains each common sample This final emotion probability, wherein, the algorithm obtaining described final emotion probability is:

p_joint(e_i|X)=(1-λ)p(e_i|X)+λp_transfer(e_i|X)；

Wherein, p (e_i| X) it is the sample X probability that belongs to all kinds of emotion classification, p_transfer(e_i| X) be The emotion transition probability of sample X, λ is the weight parameter set, p_joint(e_i| X) it is final emotion probability；

Emotion classification determination unit, is used for obtaining in described common sample, and the value of final emotion probability is maximum Common sample, demarcate sample as emotion, and determine that described emotion demarcates emotion belonging to sample Classification is the emotion classification belonging to described text to be sorted.

Preferably, described grader acquisition module includes:

Corpus acquiring unit, has Emotion tagging and the corpus of emotion mark for obtaining respectively；

Grader acquiring unit, obtains described emotion for having the corpus of Emotion tagging described in basis Grader, and obtain described emotion grader according to the described corpus with emotion mark.

Accordingly, the invention also discloses a kind of processor, the chip of described processor is integrated with as above Described document sorting apparatus.

File classification method disclosed in this invention, first obtains emotion test sample and feelings according to grader Thread test sample is belonging respectively to the probability of each emotional category and each emotion classification, then chooses feelings Sense test sample and mood test sample are as common sample, and obtain the joint probability of described common sample, Described joint probability and described probability are weighted summation, and obtain according to the result of weighted sum Emotional category belonging to text to be sorted, and/or emotion classification.

During this, emotion and the emotion of text to be sorted are combined, to realize the feelings to text Sense classification, and/or emotion classification.Owing to there being close contact between emotion and the emotion of one text, Emotion and the emotion of text to be sorted are being combined, when text is classified, is taking full advantage of test specimens In Ben between emotion and emotion substantial connection provide additional amount of information, with treat classifying text classify time, The precision of text classification can be effectively improved.

It addition, file classification method disclosed in the present application, carry out for emotion and two kinds of classification task of emotion Combination learning, simply carries out literary composition according to emotion test sample and mood test sample relative in prior art For the method for this classification, add the quantity of test sample, be also correspondingly improved the essence of text classification Degree.

Accompanying drawing explanation

In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, below will be to reality Execute the required accompanying drawing used in example or description of the prior art to be briefly described, it should be apparent that below, Accompanying drawing in description is only some embodiments of the present invention, for those of ordinary skill in the art, On the premise of not paying creative work, it is also possible to obtain other accompanying drawing according to these accompanying drawings.

Fig. 1 is the workflow schematic diagram of a kind of file classification method disclosed in the embodiment of the present invention；

Fig. 2 is the workflow schematic diagram of the embodiment of the present invention another file classification method disclosed；

Fig. 3 is the workflow schematic diagram of the embodiment of the present invention another file classification method disclosed；

Fig. 4 is the accuracy rate experiment knot that file classification method disclosed in the embodiment of the present invention carries out emotional semantic classification Really comparison diagram；

Fig. 5 is the accuracy rate experiment knot that file classification method disclosed in the embodiment of the present invention carries out emotion classification Really comparison diagram；

Fig. 6 is the structural representation of a kind of document sorting apparatus disclosed in the embodiment of the present invention.

Detailed description of the invention

Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out Clearly and completely describe, it is clear that described embodiment is only a part of embodiment of the present invention, and It is not all, of embodiment.Based on the embodiment in the present invention, those of ordinary skill in the art are not making Go out the every other embodiment obtained under creative work premise, broadly fall into the scope of protection of the invention.

In order to solve prior art when carrying out text classification, the problem that the nicety of grading that had is low, this Application provides a kind of file classification method, to improve the precision of text classification.

Workflow schematic diagram shown in Figure 1, file classification method disclosed in the present application includes:

Step S11, acquisition grader, described grader includes: emotion classifiers and emotion grader.

Pass through emotion classifiers, it is possible to obtaining the emotional category of text, wherein, described emotional category is general Including: positive, neutral and passive；By emotion grader, it is possible to obtain the emotion classification of text, its In, described emotion classification generally comprises: glad, sad, worried, angry, surprised, angry, merciless Thread.It is, of course, also possible to include other emotional category and emotion classification, this is not limited by the application.

Step S12, the emotion test sample obtaining text to be sorted and mood test sample, according to described point Class device, classifies to described emotion test sample and mood test sample, and obtains described emotion test Sample and mood test sample are belonging respectively to the probability of each emotional category and each emotion classification.

By described emotion classifiers, it is possible to obtain each emotion test sample and adhere to separately and each emotional category Probability；By described emotion grader, it is possible to obtain each mood test sample and adhere to separately and each The probability of emotion classification.

Step S13, choose emotion test sample and mood test sample as common sample, and obtain described The joint probability of common sample, described joint probability includes: emotion joint probability, and/or emotion associating is general Rate, wherein, described emotion joint probability p (s_i|e_j) representing: the emotion classification of certain sample is e_jSituation Under, its emotional category is s_iProbability, described emotion joint probability p (e_i|s_j) represent: the feelings of certain sample Sense classification is s_jIn the case of, its emotion classification is e_iProbability.

Described emotion test sample and mood test sample is being got by emotion classifiers and emotion grader After this probability, select a number of emotion test sample and mood test sample as common sample This.Wherein, emotion test sample and mood test sample as common sample typically have equal number Amount, such as, selects 400 emotion test samples and 400 mood test samples as jointly simultaneously Sample.It is of course also possible to select the whole emotion test samples in step S12 and mood test sample to make For common sample.

When needing to carry out emotional semantic classification for text to be sorted, after choosing common sample, need to obtain The emotion joint probability of described common sample；When needing to carry out emotion classification for text to be sorted, then Need to obtain the emotion joint probability of described common sample；When text to be sorted simultaneously need to carry out emotion and divide When class and emotion classification, then need to obtain emotion joint probability and the emotion joint probability of described common sample. Wherein, described emotion joint probability p (s_i|e_j) representing: the emotion classification of certain sample is e_jIn the case of, Its emotional category is s_iProbability；Described emotion joint probability p (e_i|s_j) represent: the emotion class of certain sample Wei s_jIn the case of, its emotion classification is e_iProbability.

Utilize common sample acquisition emotion joint probability p (s_i|e_j) and emotion joint probability p (e_i|s_j) algorithm For:

$p\left(s_i\right|e_j)=\frac{1+\underset{{X\∈L}_{\mathrm{Sharing}}}{\mathrm{\Σ}}I(Y_s\left(X\right)=s_i.Y_E\left(X\right)=e_j)}{N_s+\underset{{X\∈L}_{\mathrm{Sharing}}}{\mathrm{\Σ}}I(Y_E\left(X\right)=e_j)}$

$p\left(e_i\right|s_j)=\frac{1+\underset{{X\∈L}_{\mathrm{Sharing}}}{\mathrm{\Σ}}I(Y_E\left(X\right)=e_i,Y_s\left(X\right)=s_j)}{N_e+\underset{{X\∈L}_{\mathrm{Sharing}}}{\mathrm{\Σ}}I(Y_s\left(X\right)=s_j)}$

In above-mentioned algorithmic formula, L_SharingRepresent is common sample, say, that sample X belongs to common Sample；Y_S(X) emotional category belonging to sample X, Y are represented_E(X) the emotion classification belonging to sample X is represented, N_sFor the emotional category number of emotional semantic classification, N_eIt is the number of emotion classification in emotion classification, and, $I\left(x\right)=\left\{\begin{array}{cc}1,& x=\mathrm{True}\\ 0,& x=\mathrm{False}\end{array}\right..$ Wherein, I(x) it is an indicator function, when x is True, its value is 1, X be false duration be 0.

According to above-mentioned algorithm, when calculating the emotion joint probability of a certain common sample, need to know Described common sample is belonging respectively to the probability of each emotional category and each emotion classification.And in step In S12, according to emotion classifiers and emotion grader, obtain described emotion test sample and emotion has been surveyed Sample is originally belonging respectively to the probability of each emotional category and each emotion classification, then may utilize emotion Grader, obtains emotion test sample and belongs to the probability of each emotion classification, and utilize emotional semantic classification Device, obtains mood test sample and belongs to the probability of each emotional category, and then public according to above-mentioned algorithm Formula obtains the emotion joint probability of common sample, and/or emotion joint probability.Or, it would however also be possible to employ people The mode of work mark, utilizes manual type to carry out emotion in the common sample chosen from emotion test sample Mark, and utilize manual type to carry out Emotion tagging in the common sample chosen from mood test sample, Thus improve the described emotion joint probability of acquisition, and/or the precision of emotion joint probability.

Step S14, described probability and joint probability are weighted summation, and according to weighted sum Result obtains the emotional category belonging to text to be sorted, and/or emotion classification.

When classifying text only need to be treated carries out emotional semantic classification, need to obtain common sample probability and Emotion joint probability；When classifying text only need to be treated carries out emotion classification, need to obtain common sample Probability and emotion joint probability；Treat classifying text when needs simultaneously and carry out emotional semantic classification and emotion is divided During class, then need to obtain the probability of described common sample, emotion joint probability and emotion joint probability.

File classification method disclosed in this invention, first obtains emotion test sample and feelings according to grader Thread test sample is belonging respectively to the probability of each emotional category and each emotion classification, then chooses feelings Sense test sample and mood test sample are as common sample, and obtain the joint probability of described common sample, Described joint probability and described probability are weighted summation, and obtain according to the result of weighted sum Emotional category belonging to text to be sorted, and/or emotion classification.

During this, emotion and the emotion of text to be sorted are combined, to realize the feelings to text Sense classification, and/or emotion classification.Owing to there being close contact between emotion and the emotion of one text, Emotion and the emotion of text to be sorted are being combined, when text is classified, is taking full advantage of test specimens In Ben between emotion and emotion substantial connection provide additional amount of information, with treat classifying text classify time, The precision of text classification can be effectively improved.

It addition, file classification method disclosed in the present application, carry out for emotion and two kinds of classification task of emotion Combination learning, simply carries out literary composition according to emotion test sample and mood test sample relative in prior art For the method for this classification, add the quantity of test sample, be also correspondingly improved the essence of text classification Degree.

Further, workflow schematic diagram shown in Figure 2, in step S14, to described initially Probability and joint probability are weighted summation, and obtain belonging to text to be sorted according to the result of weighted sum Emotional category, including:

Step S141, according to the probability of described common sample and emotion joint probability, obtain each altogether With the transference probability of sample, wherein, the algorithm obtaining described transference probability is:

$P_{\mathrm{transfer}}\left(s_i\right|X)=\underset{j=1}{\overset{N_e}{\mathrm{\Σ}}}p\left(s_i\right|e_j)*p\left(e_j\right|X);$

Wherein, p (e_j| X) it is the common sample X probability that belongs to all kinds of emotion classification, p (s_i|e_j) it is Emotion joint probability, the emotion classification being certain common sample is e_jTime, emotional category is s_iProbability, N_eFor the emotion classification number of emotion classification, p_transfer(s_i| X) it is the transference probability of sample X.

In step s 12, according to emotion grader, common sample X can be obtained and belong to all kinds of emotion class Other probability p (e_j| X), and, the scheme provided according to step S13, emotion associating can be obtained Probability p (s_i|e_j), substituted in above-mentioned algorithmic formula, transference Probability p can be obtained_transfer(s_i|X)。

Step S142, according to described transference probability, obtain the final emotion probability of each common sample, Wherein, the algorithm obtaining described final emotion probability is:

p_joint(s_i|X)=(1-λ)p(s_i|X)+λp_transfer(s_i|X)；

Wherein, p (s_i| X) it is the sample X probability that belongs to all kinds of emotional category, p_transfer(s_i| X) it is sample The transference probability of X, λ is the weight parameter set, p_joint(s_i| X) it is final emotion probability.

In above-mentioned algorithmic formula, λ is weighting parameters, is any number between 0 to 1, is used for regulating Emotion probability and the importance of emotion probability, be embodied as the interval of middle λ usually: (0.4,0.8). When emotion probability is identical with the importance of emotion probability, the value of λ is 0.5.

Step S143, obtain in described common sample, the common sample that the value of final emotion probability is maximum, Demarcate sample as emotion, and determine that described emotion demarcation emotional category belonging to sample is treated described in being Emotional category belonging to classifying text.

The emotional category that text to be sorted finally determines is:Wherein, Y_sentiment(X) represent is the emotional category that finally determines of text to be sorted, p_joint(s_i| X) represent is sample The final emotion probability of X,Represent is the common of the value maximum of final emotion probability Emotional category belonging to sample.

According to the operation of step S141 to step S143, by the probability got in advance and emotion Transition probability, can obtain the emotional category of text to be sorted.Further, text is being carried out emotional semantic classification Time, take full advantage of the extraneous information that the substantial connection between emotional training sample and emotion and emotion provides Amount, improves the accuracy rate of emotional semantic classification.

Further, workflow schematic diagram shown in Figure 3, in step S14, to described initially Probability and joint probability are weighted summation, and obtain belonging to text to be sorted according to the result of weighted sum Emotion classification, including:

Step S144, according to the probability of described common sample and joint probability, obtain each common sample This emotion transition probability, wherein, the algorithm obtaining described emotion transition probability is:

$p_{\mathrm{transfer}}\left(e_i\right|X)=\underset{j=1}{\overset{N_s}{\mathrm{\Σ}}}p\left(e_i\right|s_j)*p\left(s_j\right|X).$

Wherein, p (s_j| X) it is the common sample X probability that belongs to all kinds of emotional category, p (e_i|s_j) it is emotion Joint probability, the emotional category being certain common sample is s_jTime, emotion classification is e_iProbability, N_sFor The emotional category number of emotional semantic classification, p_transfer(e_i| X) it is the emotion transition probability of sample X；

In step s 12, according to emotion classifiers, common sample X can be obtained and belong to all kinds of emotion class Other probability p (s_j| X), and, the scheme provided according to step S13, emotion associating can be obtained Probability p (e_i|s_j), substituted in above-mentioned algorithmic formula, emotion transition probability p can be obtained_transfer(e_i|X)。

Step S145, according to described emotion transition probability, obtain the final emotion probability of each common sample, Wherein, the algorithm obtaining described final emotion probability is:

p_joint(e_i|X)=(1-λ)p(e_i|X)+λp_transfer(e_i|X)；

Wherein, p (e_i| X) it is the sample X probability that belongs to all kinds of emotion classification, p_transfer(e_i| X) be The emotion transition probability of sample X, λ is the weight parameter set, p_joint(e_i| X) it is final emotion probability.

In above-mentioned algorithmic formula, λ is weighting parameters, can choose any number between 0 to 1, be used for Regulation emotion probability and the importance of emotion probability, be embodied as the interval of middle λ usually: (0.4, 0.8).When emotion probability is identical with the importance of emotion probability, the value of λ is 0.5.

Step S146, obtain in described common sample, the common sample that the value of final emotion probability is maximum, Demarcate sample as emotion, and determine that described emotion demarcation emotion classification belonging to sample is treated described in being Emotion classification belonging to classifying text.

The emotion classification that text to be sorted finally determines is:Wherein, Y_emotion(X) represent is the emotion classification that finally determines of text to be sorted, p_joint(e_i| X) represent is sample The final emotion probability of X,Represent is the common sample of the value maximum of final emotion probability Emotion classification belonging to Ben.

According to scheme disclosed in step S144 and step S146, it is possible to shifted by the emotion of common sample Probability and probability, determine the emotion classification belonging to text to be sorted, and, text is being carried out feelings During thread classification, make use of the extraneous information that the substantial connection between emotion training sample and emotion and emotion provides Amount, improves the accuracy rate of emotion classification.

Further, disclosed described acquisition grader in step s 11, including:

First, acquisition has Emotion tagging and the corpus of emotion mark respectively；

Then, according to the described corpus described emotion classifiers of acquisition with Emotion tagging, and according to The described corpus with emotion mark obtains described emotion grader.

Wherein, there is Emotion tagging and the corpus of emotion mark, by obtaining training text in advance, Then described training text is carried out artificial Emotion tagging and emotion mark and obtains.Manually marking During note, it is mainly based upon the unitary characteristic of word.

Described emotion classifiers and emotion grader utilize the algorithm of machine learning, such as maximum entropy algorithm, SVM (Support Vector Machine, support vector machine) algorithm, bayesian algorithm etc., and there is emotion The disaggregated model that the corpus of mark and emotion mark generates, for this disaggregated model, works as input During sample to be sorted, emotion classifiers then can export the probability of the text each emotional category corresponding, Emotion grader then can export the probability of the text each emotion classification corresponding.It is to say, pass through Described state emotion classifiers and emotion grader, emotion test sample and mood test sample can be obtained respectively Originally the probability of each emotional category and each emotion classification it is belonging respectively to.

File classification method disclosed in the present application, it is achieved that emotion test sample and mood test sample Combination learning, thus improve the precision of text classification.In order to prove the effectiveness of the method for the present invention, We utilize Ren-CECps language material to carry out emotion and the experiment of emotion classification.Described Ren-CECps language Material comprises 34,603 sentences.In specific experiment, have selected 3000,5000,8000 sentences respectively As emotion test sample, additionally select 3000,5000,8000 sentences as mood test sample, And according to scheme disclosed in the present application, text carried out emotional semantic classification and emotion classification.Wherein, weight λ is set It is set to 0.5, and respectively selects 400 sentences as altogether from described emotion test sample and mood test sample Same sample.

File classification method disclosed in the present application shown in Figure 4 emotion on Ren-CECps language material Classification accuracy experimental result, wherein, what axis of abscissas represented is the quantity of the emotion test sample chosen, What axis of ordinates represented is the precision of classification, and what " single task emotional semantic classification " represented is only to utilize to have feelings The method that the sample of sense mark carries out emotional semantic classification, what " combination learning emotional semantic classification " represented is according to this The disclosed mode of application carries out emotional semantic classification.

From the correction data shown in Fig. 4, it can be seen that the method for the present invention is in emotional semantic classification task Classifying quality is compared traditional emotional semantic classification and is significantly improved, and average percentage improvement is 2.4%.

File classification method disclosed in the present application shown in Figure 5 emotion on Ren-CECps language material Classification accuracy experimental result, wherein, what axis of abscissas represented is the quantity of the mood test sample chosen, What axis of ordinates represented is the precision of classification, and what " classification of single task emotion " represented is only to utilize to have feelings The method that the sample of thread mark carries out emotion classification, what " classification of combination learning emotion " represented is according to this The disclosed mode of application carries out emotion classification.

From the correction data shown in Fig. 5, it can be seen that the method for the present invention is in emotion classification task Classifying quality is compared traditional emotion classification and is significantly improved, and average percentage improvement is 2.9%.

According to above-mentioned experimental result, the file classification method that the application provides take full advantage of emotion and Being closely connected between emotion, thus improve the precision that text is classified.

Accordingly, disclosed herein as well is a kind of document sorting apparatus, structural representation shown in Figure 6 Figure, described document sorting apparatus includes: grader acquisition module 11, probability acquisition module 12, connection Conjunction probability acquisition module 13 and text categories acquisition module 14, wherein,

Described grader acquisition module 11, is used for obtaining grader, and described grader includes: emotional semantic classification Device and emotion grader；

Described probability acquisition module 12, for obtaining emotion test sample and the emotion of text to be sorted Test sample, according to described grader, classifies to emotion test sample and mood test sample, and Obtain described emotion test sample and mood test sample is belonging respectively to each emotional category and each emotion class Other probability；

Described joint probability acquisition module 13, is used for choosing emotion test sample and mood test sample conduct Common sample, and obtain the joint probability of described common sample, described joint probability includes: emotion is combined Probability, and/or emotion joint probability, wherein, described emotion joint probability p (s_i|e_j) represent: certain sample Emotion classification be e_jIn the case of, its emotional category is s_iProbability, described emotion joint probability p (e_i|s_j) Represent: the emotional category of certain sample is s_jIn the case of, its emotion classification is e_iProbability；

Described text categories acquisition module 14, for being weighted asking to described probability and joint probability With, and obtain the emotional category belonging to text to be sorted, and/or emotion classification according to the result of weighted sum.

Document sorting apparatus disclosed in the present application, first passes through grader acquisition module and obtains emotion classifiers With emotion grader；Then by probability acquisition module, by the described emotion classifiers that gets and Emotion grader, obtains emotion test sample and mood test sample is belonging respectively to each emotional category with each The probability of individual emotion classification；Joint probability acquisition module chooses common sample, and obtain described jointly The joint probability of sample；Finally, by text categories acquisition module according to described probability and joint probability It is weighted summation, and according to the emotional category belonging to the result acquisition text to be sorted of weighted sum, and/ Or emotion classification.

Document sorting apparatus disclosed in the present application, is emotion and the emotion of text to be sorted to be combined, To realize the emotional semantic classification to text, and/or emotion classification.Due between emotion and the emotion of one text There is close contact, by the emotion of text to be sorted and emotion associating, carry out treating classifying text During classification, the precision of text classification can be effectively improved.

It addition, document sorting apparatus disclosed in the present application, carry out for emotion and two kinds of classification task of emotion Combination learning, simply carries out literary composition according to emotion test sample and mood test sample relative in prior art For the method for this classification, add the quantity of sample, be also correspondingly improved the precision of text classification.

Further, text categories acquisition module 14 disclosed in the present application includes: emotional category obtains Submodule, described emotional category obtains submodule and includes: transference probability acquiring unit, final emotion Probability acquiring unit and emotional category determine unit, wherein,

Described transference probability acquiring unit, for the probability according to described common sample and emotion Joint probability, obtains the transference probability of each common sample, wherein, obtains described transference general The algorithm of rate is:

$p_{\mathrm{reansfer}}\left(s_i\right|X)=\underset{j=1}{\overset{N_e}{\mathrm{\Σ}}}p\left(s_i\right|e_j)*p\left(e_j\right|X);$

Wherein, p (e_j| X) it is the common sample X probability that belongs to all kinds of emotion classification, p (s_i|e_j) it is Emotion joint probability, the emotion classification being certain common sample is e_jTime, emotional category is s_iProbability, N_eFor the emotion classification number of emotion classification, p_transfer(s_i| X) it is the transference probability of sample X；

Described final emotion probability acquiring unit, for according to described transference probability, obtains each altogether With the final emotion probability of sample, wherein, the algorithm obtaining described final emotion probability is:

p_joint(s_i|X)=(1-λ)p(s_i|X)+λp_transfer(s_i|X)；

Wherein, p (s_i| X) it is the sample X probability that belongs to all kinds of emotional category, p_transfer(s_i| X) it is sample The transference probability of X, λ is the weight parameter set, p_joint(s_i| X) it is final emotion probability；

Described emotional category determines unit, is used for obtaining in described common sample, the value of final emotion probability Maximum common sample, demarcates sample as emotion, and determines that described emotion is demarcated belonging to sample Emotional category is the emotional category belonging to described text to be sorted.

True according to described transference probability acquiring unit, final emotion probability acquiring unit and emotional category Cell, it is possible to realize the emotional semantic classification to text.

Further, described text categories acquisition module includes: emotion classification obtains submodule, described feelings Thread classification obtain submodule include: emotion transition probability acquiring unit, final emotion probability acquiring unit and Emotion classification determination unit, wherein,

Described emotion transition probability acquiring unit, for the probability according to described common sample and associating Probability, obtains each common sample emotion transition probability, wherein, obtains the calculation of described emotion transition probability Method is:

$p_{\mathrm{transfer}}\left(e_i\right|X)=\underset{j=1}{\overset{N_s}{\mathrm{\Σ}}}p\left(e_i\right|s_j)*p\left(s_j\right|X)$

Wherein, p (s_j| X) it is the common sample X probability that belongs to all kinds of emotional category, p (e_i|s_j) it is feelings Thread joint probability, the emotional category being certain common sample is s_jTime, emotion classification is e_iProbability, N_s For the emotional category number of emotional semantic classification, p_transfer(e_i| X) it is the emotion transition probability of sample X；

Described final emotion probability acquiring unit, for according to described emotion transition probability, obtains each altogether With the final emotion probability of sample, wherein, the algorithm obtaining described final emotion probability is:

p_joint(e_i|X)=(1-λ)p(e_i|X)+λp_transfer(e_i|X)；

Wherein, p (e_i| X) it is the sample X probability that belongs to all kinds of emotion classification, p_transfer(e_i| X) be The emotion transition probability of sample X, λ is the weight parameter set, p_joint(e_i| X) it is final emotion probability；

Described emotion classification determination unit, is used for obtaining in described common sample, the value of final emotion probability Maximum common sample, demarcates sample as emotion, and determines that described emotion is demarcated belonging to sample Emotion classification is the emotion classification belonging to described text to be sorted.

True according to described emotion transition probability acquiring unit, final emotion probability acquiring unit and emotion classification Cell, it is possible to realize the emotion to text and classify.

Further, described grader acquisition module includes: corpus acquiring unit and grader obtain Unit, wherein,

Described corpus acquiring unit, has Emotion tagging and the training of emotion mark for obtaining respectively Language material；

Described grader acquiring unit, obtains described for having the corpus of Emotion tagging described in basis Emotion classifiers, and obtain described emotion grader according to the described corpus with emotion mark.

Accordingly, the invention also discloses a kind of processor, the chip of described processor is integrated with above-mentioned Document sorting apparatus.

Wherein, described document sorting apparatus includes: grader acquisition module 11, probability acquisition module 12, joint probability acquisition module 13 and text categories acquisition module 14.Further, described text categories Acquisition module 14 includes: emotional category obtains submodule, and/or emotion classification obtains submodule, described Emotional category obtains submodule and includes: transference probability acquiring unit, final emotion probability acquiring unit Determining unit with emotional category, described emotion classification obtains submodule and includes: emotion transition probability obtains single Emotion probability acquiring unit first, final and emotion classification determination unit.

It addition, described grader acquisition module 11 includes: corpus acquiring unit and grader obtain single Unit.

Utilize described processor that text is classified, it is possible to emotion and the emotion of text to be sorted to be carried out Associating, to realize the emotional semantic classification to text, and/or emotion classification.Emotion and feelings due to one text There is close contact between thread, emotion and the emotion of text to be sorted are being combined, with to literary composition to be sorted When originally classifying, the precision of text classification can be effectively improved.

File classification method disclosed in the present application and document sorting apparatus, it is possible to be applied to the emotion to text, And/or emotion classification carries out the technical field classified.Such as, when getting newsletter archive and microblogging text, By file classification method disclosed in the present application and document sorting apparatus, it is possible to obtain the emotion class of the text Not, contribute to understanding satisfaction and the demand of the common people, find the hot spot of society in time, and, pass through Get the emotion classification of the text, contribute to getting the common people and the viewpoint of things is inclined to.

Those of ordinary skill in the art are it is to be appreciated that combine each of the embodiments described herein description The unit of example and algorithm steps, it is possible to electronic hardware or computer software and the knot of electronic hardware Incompatible realization.These functions perform with hardware or software mode actually, depend on the spy of technical scheme Fixed application and design constraint.Professional and technical personnel can use not Tongfang to each specifically should being used for Method realizes described function, but this realization is it is not considered that beyond the scope of this invention.

Those skilled in the art is it can be understood that arrive, and for convenience and simplicity of description, above-mentioned retouches The specific works process of system, device and the unit stated, is referred to the correspondence in preceding method embodiment Process, does not repeats them here.

In several embodiments provided herein, it should be understood that disclosed system, device and Method, can realize by another way.Such as, device embodiment described above is only shown Meaning property, such as, the division of described unit, be only a kind of logic function and divide, actual can when realizing There to be other dividing mode, the most multiple unit or assembly can in conjunction with or be desirably integrated into another System, or some features can ignore, or do not perform.Another point, shown or discussed each other Coupling direct-coupling or communication connection can be the INDIRECT COUPLING by some interfaces, device or unit Or communication connection, can be electrical, machinery or other form.

The described unit illustrated as separating component can be or may not be physically separate, makees The parts shown for unit can be or may not be physical location, i.e. may be located at a place, Or can also be distributed on multiple NE.Can select according to the actual needs part therein or The whole unit of person realizes the purpose of the present embodiment scheme.

It addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, Can also be that unit is individually physically present, it is also possible to two or more unit are integrated in a list In unit.

If described function realizes and as independent production marketing or use using the form of SFU software functional unit Time, can be stored in a computer read/write memory medium.Based on such understanding, the present invention's Part or the part of this technical scheme that prior art is contributed by technical scheme the most in other words can Embodying with the form with software product, this computer software product is stored in a storage medium, Including some instructions with so that computer equipment (can be personal computer, server, or The network equipment etc.) perform all or part of step of method described in each embodiment of the present invention.And it is aforesaid Storage medium includes: USB flash disk, portable hard drive, read only memory (ROM, Read-Only Memory), Random access memory (RAM, Random Access Memory), magnetic disc or CD etc. are various can To store the medium of program code.

Described above to the disclosed embodiments, makes professional and technical personnel in the field be capable of or uses The present invention.Multiple amendment to these embodiments will be aobvious and easy for those skilled in the art See, generic principles defined herein can without departing from the spirit or scope of the present invention, Realize in other embodiments.Therefore, the present invention is not intended to be limited to the embodiments shown herein, And it is to fit to the widest scope consistent with principles disclosed herein and features of novelty.

Claims (11)

1. a file classification method, it is characterised in that including:

Obtaining grader, described grader includes: emotion classifiers and emotion grader；

Obtain emotion test sample and the mood test sample of text to be sorted, according to described grader, right Described emotion test sample and mood test sample are classified, and obtain described emotion test sample and feelings Thread test sample is belonging respectively to the probability of each emotional category and each emotion classification；

Choose emotion test sample and mood test sample as common sample, and obtain described common sample Joint probability, described joint probability includes: emotion joint probability, and/or emotion joint probability, wherein, Described emotion joint probability p (s_i|e_j) representing: the emotion classification of certain sample is e_jIn the case of, its emotion Classification is s_iProbability, described emotion joint probability p (e_i|s_j) representing: the emotional category of certain sample is s_j's In the case of, its emotion classification is e_iProbability；

Described probability and joint probability are weighted summation, and obtain according to the result of weighted sum Emotional category belonging to text to be sorted, and/or emotion classification.

Method the most according to claim 1, it is characterised in that general to described probability and associating Rate is weighted summation, and obtains the emotional category belonging to text to be sorted according to the result of weighted sum, Including:

Probability according to described common sample and emotion joint probability, obtain the feelings of each common sample Sense transition probability, wherein, the algorithm obtaining described transference probability is:

$p_{transfer}\left(s_i\right|X)=\underset{j=1}{\overset{N_e}{\Σ}}p\left(s_i\right|e_j)*p\left(e_j\right|X);$

Wherein, p (e_j| X) it is the common sample X probability that belongs to all kinds of emotion classification, p (s_i|e_j) it is Emotion joint probability, the emotion classification being certain common sample is e_jTime, emotional category is s_iProbability, N_eFor the emotion classification number of emotion classification, p_transfer(s_i| X) it is the transference probability of sample X；

According to described transference probability, obtain the final emotion probability of each common sample, wherein, obtain The algorithm taking described final emotion probability is:

p_joint(s_i| X)=(1-λ) p (s_i|X)+λp_transfer(s_i|X)；

Wherein, p (s_i| X) it is the sample X probability that belongs to all kinds of emotional category, p_transfer(s_i| X) it is sample The transference probability of X, λ is the weight parameter set, p_joint(s_i| X) it is final emotion probability；

Obtaining in described common sample, the common sample that the value of final emotion probability is maximum, as feelings Sample is demarcated in sense, and determines that described emotion demarcation emotional category belonging to sample is described text institute to be sorted The emotional category belonged to.

Method the most according to claim 1, it is characterised in that general to described probability and associating Rate is weighted summation, and obtains the emotion classification belonging to text to be sorted according to the result of weighted sum, Including:

Probability according to described common sample and joint probability, the emotion obtaining each common sample turns Moving probability, wherein, the algorithm obtaining described emotion transition probability is:

$p_{transfer}\left(e_i\right|X)=\underset{j=1}{\overset{N_s}{\Σ}}p\left(e_i\right|s_j)*p\left(s_j\right|X)$

Wherein, p (s_j| X) it is the common sample X probability that belongs to all kinds of emotional category, p (e_i|s_j) it is feelings Thread joint probability, the emotional category being certain common sample is s_jTime, emotion classification is e_iProbability, N_s For the emotional category number of emotional semantic classification, p_transfer(e_i| X) it is the emotion transition probability of sample X；

According to described emotion transition probability, obtain the final emotion probability of each common sample, wherein, obtain The algorithm taking described final emotion probability is:

p_joint(e_i| X)=(1-λ) p (e_i|X)+λp_transfer(e_i|X)；

Wherein, p (e_i| X) it is the sample X probability that belongs to all kinds of emotion classification, p_transfer(e_i| X) be The emotion transition probability of sample X, λ is the weight parameter set, p_joint(e_i| X) it is final emotion probability；

Obtaining in described common sample, the common sample that the value of final emotion probability is maximum, as feelings Thread demarcates sample, and determines that described emotion demarcation emotion classification belonging to sample is described text institute to be sorted The emotion classification belonged to.

The most according to the method in claim 2 or 3, it is characterised in that

The interval of described weight parameter λ is: (0.4,0.8).

Method the most according to claim 1, it is characterised in that described acquisition grader includes:

Obtain respectively and there is Emotion tagging and the corpus of emotion mark；

Described emotion classifiers is obtained according to the described corpus with Emotion tagging, and according to described tool The corpus marked of being in a bad mood obtains described emotion grader.

Method the most according to claim 1, it is characterised in that

Described emotional category includes: positive, neutral and passive；

Described emotion classification includes: glad, sad, worried, angry, surprised, angry, loss of emotion.

7. a document sorting apparatus, it is characterised in that including:

Grader acquisition module, is used for obtaining grader, and described grader includes: emotion classifiers and feelings Thread grader；

Probability acquisition module, for obtaining emotion test sample and the mood test sample of text to be sorted This, according to described grader, classify to emotion test sample and mood test sample, and obtain institute State emotion test sample and mood test sample is belonging respectively at the beginning of each emotional category and each emotion classification Beginning probability；

Joint probability acquisition module, is used for choosing emotion test sample and mood test sample as common sample Basis, and obtain the joint probability of described common sample, described joint probability includes: emotion joint probability, And/or emotion joint probability, wherein, described emotion joint probability p (s_i|e_j) represent: the emotion of certain sample Classification is e_jIn the case of, its emotional category is s_iProbability, described emotion joint probability p (e_i|s_j) represent: The emotional category of certain sample is s_jIn the case of, its emotion classification is e_iProbability；

Text categories acquisition module, for described probability and joint probability being weighted summation, and Result according to weighted sum obtains the emotional category belonging to text to be sorted, and/or emotion classification.

Device the most according to claim 7, it is characterised in that described text categories acquisition module bag Including: emotional category obtains submodule, described emotional category obtains submodule and includes:

Transference probability acquiring unit, for the probability according to described common sample and emotion associating Probability, obtains the transference probability of each common sample, wherein, obtains described transference probability Algorithm is:

$p_{transfer}\left(s_i\right|X)=\underset{j=1}{\overset{N_e}{\Σ}}p\left(s_i\right|e_j)*p\left(e_j\right|X);$

Wherein, p (e_j| X) it is the common sample X probability that belongs to all kinds of emotion classification, p (s_i|e_j) it is Emotion joint probability, the emotion classification being certain common sample is e_jTime, emotional category is s_iProbability, N_eFor the emotion classification number of emotion classification, p_transfer(s_i| X) it is the transference probability of sample X；

Final emotion probability acquiring unit, for according to described transference probability, obtains each common sample This final emotion probability, wherein, the algorithm obtaining described final emotion probability is:

p_joint(s_i| X)=(1-λ) p (s_i|X)+λp_transfer(s_i|X)；

Wherein, p (s_i| X) it is the sample X probability that belongs to all kinds of emotional category, p_transfer(s_i| X) it is sample The transference probability of X, λ is the weight parameter set, p_joint(s_i| X) it is final emotion probability；

Emotional category determines unit, is used for obtaining in described common sample, and the value of final emotion probability is maximum Common sample, demarcate sample as emotion, and determine that described emotion demarcates emotion belonging to sample Classification is the emotional category belonging to described text to be sorted.

Device the most according to claim 7, it is characterised in that described text categories acquisition module bag Including: emotion classification obtains submodule, described emotion classification obtains submodule and includes:

Emotion transition probability acquiring unit, for the probability according to described common sample and joint probability, Obtaining the emotion transition probability of each common sample, wherein, the algorithm obtaining described emotion transition probability is:

$p_{transfer}\left(e_i\right|X)=\underset{j=1}{\overset{N_s}{\Σ}}p\left(e_i\right|s_j)*p\left(s_j\right|X)$

Wherein, p (s_j| X) it is the common sample X probability that belongs to all kinds of emotional category, p (e_i|s_j) it is feelings Thread joint probability, the emotional category being certain common sample is s_jTime, emotion classification is e_iProbability, N_s For the emotional category number of emotional semantic classification, p_transfer(e_i| X) it is the emotion transition probability of sample X；

Final emotion probability acquiring unit, for according to described emotion transition probability, obtains each common sample This final emotion probability, wherein, the algorithm obtaining described final emotion probability is:

p_joint(e_i| X)=(1-λ) p (e_i|X)+λp_transfer(e_i|X)；

Wherein, p (e_i| X) it is the sample X probability that belongs to all kinds of emotion classification, p_transfer(e_i| X) be The emotion transition probability of sample X, λ is the weight parameter set, p_joint(e_i| X) it is final emotion probability；

Emotion classification determination unit, is used for obtaining in described common sample, and the value of final emotion probability is maximum Common sample, demarcate sample as emotion, and determine that described emotion demarcates emotion belonging to sample Classification is the emotion classification belonging to described text to be sorted.

Device the most according to claim 7, it is characterised in that described grader acquisition module bag Include:

Corpus acquiring unit, has Emotion tagging and the corpus of emotion mark for obtaining respectively；

Grader acquiring unit, obtains described emotion for having the corpus of Emotion tagging described in basis Grader, and obtain described emotion grader according to the described corpus with emotion mark.

11. 1 kinds of processors, it is characterised in that be integrated with claim 7 on the chip of described processor To the document sorting apparatus described in any one of claim 10.