CN101377769A - Method for representing multiple graininess of text message - Google Patents

Abstract

The invention relates to a text information representation method based on multi-granularity text features. In the method, multi-granularity text representation model training is used for generating a multi-granularity text model and integration, thus forming text information multi-granularity integration representation. The method solves the problem of the integration of multi-granularity text features based on overall weights and text local features. The text information representation method is very robust and stable in respect of corpus scale and sparse data. The semantic structures implied in the text can be accurately and fully represented through acquiring the multi-granularity semantic space mapping of the text. The advantages of fine granularity text representation and coarse granularity text representation can be used comprehensively based on the relativity between the multi-granularity text features. Under the condition of training corpora with different scales, the text representation performance of the text information representation method is better than the representation performance of the single-granularity text representation method. Although the multi-granularity text representation method is in a multilayer structure, and the relations between the layers are clear, and the complex operations of a plurality of text representation methods in parameter adjustment are eliminated.

Description

A kind of method for representing multiple graininess of text message

Technical field

The present invention relates to Intelligent Information Processing, information retrieval and natural language processing technique field, particularly relate to a kind of text message method for expressing, be used for the expression originally of information retrieval or other text-processing applicating Chinese, make text be handled and to analyze use, in the purposes of carrying out information retrieval, text classification and some other text-processing, show superior performance by computing machine.

Background technology

Text-processing is an important techniques in the information processing technology, has core status in fields such as information retrieval, text message analysis, natural language processings.The first step of text-processing is that text message is expressed as the analyzable form of computer program, and the good and bad direct effect and the efficient to text-processing of text message method for expressing exerts an influence.Especially in text-processings such as text retrieval, text classification cluster and text content analysis were used, the performance of text representation often played conclusive effect.

Classical text representation method based on speech bag (BOW---Bag of Words) model and vector space model (VSM) method with the feature of speech different in the text as text representation, according to the statistical information (as frequency etc. occur) of these speech in text, utilize methods such as TFIDF, IG to carry out processing such as feature selecting and weight calculation again, constitute the representation of text vector as text.Be used to generate vectorial speech and be called as characteristic item (term).This is that the text representation granularity of method of feature is minimum based on single speech, is subjected to the influence of speech ambiguity easily.Some researchers propose to adopt the more text representation method of coarseness, improve the accuracy of feature speech semantic expressiveness, as based on method of phrase and n-gram etc.Research shows that also reasonable use coarseness feature helps to improve the performance of text representation.Also have many researchers to use the LSI model, the speech feature space is mapped to latent semantic space to extract potential semantic structure, improve the performance of text representation, this has come down to take the process of a kind of feature selecting and conversion.

But above method all is text representation methods of a kind of single granularity.Varigrained text representation correspondence be the mapping of text at the different grain size semantic space, varigrained semantic space all has Special Significance separately for text content analysis.Simultaneously, there is the nonuniformity of optimum granularity in text on local semantic expressiveness.Therefore, the text representation of the single granularity optimum that can not meet text is represented.

Summary of the invention

For the text representation that solves the single granularity of prior art can not meet the problem that the optimum of text is represented, the objective of the invention is: study many granularities text representation method, improve the performance that text message is represented, thereby the Intelligent treatment to text messages such as information retrieval, text classification, text cluster, text content analysis produces impetus, for this reason, this bright text message method for expressing that provides a kind of based on many granularities text feature.

In order to reach described purpose, the method for representing multiple graininess of text message provided by the invention comprises that step is as follows:

Step S1: utilize the study of many granularities text representation model, training generates the text model of many granularities;

Step S2: integrated based on many granularities text feature model, form many granularities of text message integrated representation.

Described many granularities text representation model is made of a plurality of varigrained text models, forms the model of sandwich construction, and is interrelated between each layer granularity text model feature unit; This model is used for the representing multiple graininess of text message.

The learning process of described many granularities text representation model may further comprise the steps:

Step S11: varigrained text model is learnt respectively;

Step S12: analyze the relation between the different grain size text model feature unit, the different grain size text model is combined constitutes the text representation model of sandwich construction.

Described many granularities text feature is integrated, be integrated into varigrained text model based on many granularities text feature of overall weight and give separately weight, their text feature space is weighted to merge obtains new text feature space, thereby merged many granularities text feature at new feature space.

Described many granularities text feature is integrated, based on integrated the steps include: of many granularities text feature of text local feature

Step S21: the text feature that obtains a plurality of granularity text models is represented;

Step S22: the weights that calculate the local different grain size feature of text;

Step S23: integrated many grain size characteristics are represented text.

Described integrated many grain size characteristics are represented text, it is the relation of inclusion of utilizing between the local different grain size text feature of text, probability distribution parameters according to varigrained text feature unit, the weights of each granularity text feature of analytical calculation text part are represented many grain size characteristics of text integrated on this basis.

Described many granularities text feature is integrated to be taked based on many granularities text feature of overall weight integratedly, or takes based on many granularities text feature of text local feature integrated.

Described text feature, employed text feature is provided by a plurality of varigrained text models, and the study of text model realizes under the support of corpus; Wherein, varigrained text representation model is the text representation of same type, or dissimilar text representations.

If it uses integrated based on many granularities text feature of text local feature in the described integrated many granularities text feature of step S2, then in the learning process of each granularity text model of step S1, need the probability distribution parameters of study different grain size text feature unit.

The many grain size characteristics of described text based on overall weight are integrated, be in text representation, by adjusting the overall separately weight that varigrained text model is endowed, if wherein have only the overall weight of a text model non-vanishing, then the method for representing multiple graininess of text is degenerated and is become the simple grain kilsyth basalt and show.

The many grain size characteristics of described text based on the text local feature are integrated, be in text representation, by adjusting the overall separately weight that varigrained text model is endowed, if wherein have only the overall weight of a text model non-vanishing, then the representing multiple graininess of text is degenerated and is become the simple grain kilsyth basalt and show.

Good effect of the present invention: superiority of the present invention is superior performance, and corpus scale and sparse data problem are had higher robustness and stability.By obtaining the mapping of many granularities of text semantic space, the text method for representing multiple graininess can be more accurately and is portrayed the semantic structure that contains in the text fully.And, can fully utilize the advantage of fine granularity text representation and coarseness text representation based on the correlativity between many granularities text feature.Under the training corpus situation of different scales, the text representation performance all is better than simple grain degree text representation method.Secondly, method realizes simple relatively.Though many granularities text representation model has the structure of multilayer, the relation between each layer structure is clear, and method realizes simple, has avoided the complex operations of many text representation methods in parameter adjustment.Therefore, the present invention is particularly suitable for carrying out the text information processing field that extensive text calculates, as information retrieval, text classification, text cluster and text content analysis etc.

Principle of the present invention is: varigrained text feature has represented to represent the mapping of text at the different grain size semantic space, has meaning separately in text representation.But represent from the integral body of text, have the approximation on the semantic expressiveness.Simultaneously, varigrained text representation is being different to the robustness of the dependence of corpus and sparse data.The fine granularity text representation can be portrayed trickleer semantic difference, robustness to sparse data problem is higher, but the while also exists in the semanteme portrayal to text and lacks accuracy, is subjected to the weakness of ambiguity influence easily, is suitable for the application on the small-scale training corpus.The coarseness text representation is the semantic abstraction of higher level often, has higher accuracy on semantic expressiveness, and it is littler influenced by ambiguity, can portray text semantic more exactly, but to the sparse data problem sensitivity, needs large-scale corpus support.Though can improve robustness to sparse data to a certain extent by smoothing technique, parameter regulation complexity and effect may not be desirable.Property difference based between the different grain size combines them, and brings into play their advantage, can obtain more performance.By the fusion of different grain size feature space, varigrained text feature addition is got up to realize the integrated of many granularities text representation, but this is not optimum method.The coarseness feature has superiority on the determinacy of semantic expressiveness, but observes semantic expressiveness from the angle of text part, and the granularity that neither one is single is suitable, and different local optimum text granularities are to change.The distribution probability of different grain size feature in corpus shown the reliability of this feature, and high more this feature that shows of probability is used to represent that the reliability of text is high more, should give higher weights in text representation.If low this grain size characteristic that then shows of this grain size characteristic probability represents that the reliability of text is lower, text representation should more depend on the more grain size characteristic of low order.Therefore, utilize the character that has relation of inclusion between the different grain size text feature,, be the different weight of different grain size characteristic allocation by analyzing the relativity of the probability parameter between local each granularity text feature unit of text.In other words, it is integrated to be with the reliability of text feature that the basis is that different grain size characteristic allocation weight is carried out feature, comprehensively coarseness and the advantage of fine granularity feature in text representation, thereby when the more excellent text semantic of acquisition is represented, raising obtains better text representation combination property to the robustness of learning sample.

Description of drawings

Fig. 1 is the text-processing FB(flow block) that adopts many granularities text representation method

Fig. 2 is the process flow diagram of the text model learning process in many granularities text representation method

Fig. 3 adopts many granularities text model to carry out the processing flow chart of text representation

Fig. 4 is a text representing multiple graininess structure of models synoptic diagram

Fig. 5 is many granularities text feature incidence relation synoptic diagram

Embodiment

Below introduce the preferred embodiments of the present invention, this part only is to illustrate of the present invention, but not to the restriction of the present invention and application or purposes.Other embodiment that draws according to the present invention belongs to technological innovation scope of the present invention too.There is the setting of related parameter also not show to have only example value to use in the scheme.

Shown in the text-processing FB(flow block) that Fig. 1 adopts many granularities text representation method, the present invention is a kind of text message method for expressing based on many granularities text feature.At first, train varigrained text model, concern between the analysis different grain size text model feature unit, each text model is combined into many granularities text representation model by corpus; Then, use these text models that target text is generated varigrained text feature and represent, utilize many granularities text feature integrated approach, the many grain size characteristics that form text message are represented.For this reason, the invention allows for a kind of many granularities text feature integrated approach and a kind of many granularities text feature integrated approach, solved the integration problem of many granularities text feature based on the text local feature based on overall weight.

The method for representing multiple graininess of text message is described below:

The feature speech can be an also phrase of single word in the text representation, and the granularity of text representation is meant the length of institute's use characteristic speech in text representation.Therefore be that the text representation of characteristic item is the text representation of granularity minimum based on word.Text representation method based on many granularities then integrates varigrained text representation, forms the unified expression of text, obtains better text representation performance.

Method mainly is divided into two steps, at first is the study and the training of many granularities text model, by the study of text corpus, makes up varigrained text model and is combined into many granularities text model.Be to utilize these models and parameter thereof to carry out many grain size characteristics integrated representation of text then.

Step S1: the study of many granularities text model:

With the corpus is object, learns varigrained text representation model, then varigrained text representation feature is carried out feature selecting and weight calculation.The learning process of each granularity text model and the difference of common text model learning process are little, but require the distribution probability parameter of learning text feature unit (or claiming characteristic item) in corpus in this course, use for many granularities integrated representation process.The feature unit probability parameter is meant in training text, the probability of occurrence information of text feature unit and the generating probability between the different grain size characteristic item.Shown in the process flow diagram of the text model learning process in many granularities of Fig. 2 text representation method, its treatment step is:

Step S11: text pre-service.As required, text is carried out pre-service, comprise processing such as stop words removal, stemization, participle.

Step S12: many granularities text model study.The varigrained text model of training on corpus comprises the work such as distribution parameter study, characteristic item weight calculation, feature selecting of characteristic item in the model.Text model with a plurality of granularities is combined into many granularities text representation model then.

Many granularities text representation structure of models as shown in Figure 4.The different grain size model forms a bigger model with the structure of multilayer, associates by the mutual relation of inclusion in different grain size text feature unit (adopting the arrow ways of connecting to represent in Fig. 4) between each layer.Fig. 5 is the synoptic diagram of incidence relation between many granularities text feature, shows that the text feature of coarseness has comprised fine-grained text feature for concrete text sequence (the speech string that is composed of words), and this relation is used to describe the association between the different grain size feature.

Step S2: many granularities integrated representation of text:

Many granularities integrated representation of text is a core of the present invention, by analyze the restriction relation that the different grain size text feature exists between the text part, generates many granularities integrated representation of target text.Adopt many granularities text model to carry out shown in the processing flow chart of text representation as Fig. 3, its basic step is:

Step S21: text pre-service.Text is carried out pre-service, comprise processing such as stop words removal, stemization, participle, processing mode is with step S11 in the training process of text model.

Step S22: the text representation that calculates each different grain size text model of target text.According to text model text representation implementation method separately, the different grain size representation of text is calculated.

Step S23: integrated many granularities text representation.Calculate many granularities integrated representation of text by the integrated approach of many granularities text feature.

The integrated of many granularities text feature is by certain mode different grain size text feature space to be merged in essence, thereby many granularities text feature is integrated the expression text.Can take dual mode or method, a kind of integrated approach that is based on overall weight; Another kind is based on the integrated approach of text local feature.

Realization based on the integrated approach of overall weight is simple relatively, gives weight separately for the different grain size text model, and then the text feature unit for each granularity text model inside will obtain identical overall weight.In text representation, will be based on this overall situation weight, the text representation with the different grain size feature is weighted gets the fusion that union realizes many granularities text feature space, to obtain final text representation.

Integrated approach based on the text local feature is considered as one section text flow or word sequence with target text, calculate the weights of each grain size characteristic unit by the incidence relation between the different grain size feature unit of analyzing the text part, and based on this varigrained text representation is integrated.Incidence relation between the feature unit of different grain size text model mainly utilizes the character that structurally has relation of inclusion between coarseness text feature unit and the fine granularity text feature unit, and portrays therebetween correlativity with the probabilistic relation that exists between the different grain size text feature unit.Usually, the probability height of text feature shows that this grain size characteristic is used for representing that the reliability of text is higher, the weight that worth acquisition is higher shows that then the parametric reliability of this grain size characteristic is lower if this grain size characteristic probability is low, and text representation should depend on the more grain size characteristic of low order.By the probability relativity between the different grain size feature unit, local for calculating and distribute different weights in each grain size characteristic unit at text, realize different grain size feature integrated in the text part with this weight.Generally speaking, in this process, should follow the coarseness priority rule, on weight calculation, tilt to coarseness, as far as possible to obtain semantic expressiveness determinacy preferably.

Have the use of overall weight in based on the integrated approach of text local feature equally, method is identical with integrated approach based on overall weight.Therefore, in many granularities integrating process of text, by the adjusting of overall weights, we can select the center of gravity of text representation is tilted to the coarseness text representation, thereby obtain the semantic text representation result who determines.Also can tilt, thereby keep fine granularity in trickle semanteme of portrayal and the advantage on robustness thereof to fine granularity text representation model.Simultaneously, no matter the integrated approach that is based on overall weight also is based on the integrated approach of text local feature, if wherein only have the overall weight of a text model non-vanishing, then the method for representing multiple graininess of text is degenerated and is become the simple grain kilsyth basalt and show method.Therefore, method of the present invention has realized the simple grain kilsyth basalt is shown the pardon of method in fact.

Example:

Text representation during this example is used with text classification is an example, adopts the n-gram method to implement the representing multiple graininess of text, and the difference of n value is represented different grain size.At many granularities text learning phase, need learn so, then different n-gram text models be integrated, form multi-layer structure model different n-gram models.Present embodiment is with by Unigram, and it is that example describes implementation process that three layers of n-gram model that Bigram and Trigram text model constitute carry out many granularities text representation, and the text feature of use is each rank language unit (gram).Specify as follows:

1. the study of many granularities text model

It is 1 to 3 text representation model that Unigram, Bigram, Trigram represent granularity respectively.It is carried out the study of n-gram respectively, and study is at separately language metamember's probability data.But be the probability of occurrence of n unit speech string here at the needed probability of n-gram text model.

1.1 the pre-service of text

The pre-service of text need be distinguished Chinese and English language and handle respectively.Usually need carry out the processing of participle for Chinese, then need to carry out the processing of stemization for English.The processing of removing stop words all is suitable for for Chinese and English, rejects semantic irrelevant word such as some tone auxiliary words or function word from text.Adopted the n-gram model in the present embodiment, can select not carry out word segmentation processing for the pre-service of Chinese.

1.2 Unigram text model study

Use w _iThe word that occurs in the expression corpus, total words is M, the appearance frequency of each word is designated as c (w in the statistics corpus _i), word w _iComputation process based on the distribution probability of maximal possibility estimation is shown below:

$P\left(w_i\right)=\frac{c\left(w_i\right)}{\underset{i=1}{\overset{M}{\mathrm{\Σ}}}c\left(w_i\right)}---\left(1\right)$

1.3 Bigram text model study

Binary speech string (w in the statistics Bigram text model _I1w _I2) appearance frequency in corpus is designated as c (w _I1w _I2), its distribution probability P (w _I1w _I2) the computation process following expression shown in:

$P\left(w_{i1}{w}_{i2}\right)=\frac{c\left(w_{i1}{w}_{i2}\right)}{c\left(w_{i1}\right)}P\left(w_{i1}\right)---\left(2\right)$

1.4 Trigram text model study

Ternary speech string (w in the statistics Trigram text model _I1w _I2w _I3) appearance frequency in corpus is designated as c (w _I1w _I2w _I3), its distribution probability P (w _I1w _I2w _I3) computation process be shown below:

$P\left(w_{i1}{w}_{i2}{w}_{i3}\right)=\frac{c\left(w_{i1}{w}_{i2}{w}_{i3}\right)}{c\left(w_{i1}{w}_{i2}\right)}P\left(w_{i1}{w}_{i2}\right)---\left(3\right)$

1.5 feature selecting and characteristic item weights calculate

Use characteristic is selected, and reduces the number of text feature item on the one hand, improves operational efficiency; On the other hand, select to help the characteristic item of text classification, filtering does not help even disadvantageous characteristic item text classification.In text classification, crossing high frequency or crossing the characteristics of low-frequency item not only increases calculated amount, and classification performance is had negative effect.Can take the word frequency method to filter these characteristic items, for example will be lower than 3 characteristic item in the highest 10 the characteristic item of word frequency in the training corpus study and word frequency here and give filtering, carry out feature selecting.

In the text vector that is generated by text model, component of a vector is corresponding with the weights of each characteristic item.Present embodiment is that each characteristic item calculates weights in the document with the TF-IDF method, represents with following formula at the component of a vector of TF-IDF method Chinese version:

a _jd＝f _jd·log(N/n _j) (4)

F wherein _JdRepresent the word frequency of j characteristic item in text d, the sum of N representative training storehouse Chinese version, n _jRepresent that text number at least once appears in j feature.

The weights of each characteristic item of j are the IDF value, are designated as:

tw(j)＝log(N/n _j) (5)

2. many granularities text representation is integrated

At first, text is carried out pre-service, processing procedure is with the learning phase of many granularities template.Then, utilize study gained different grain size model representation text, obtain the character representation vector of text.The feature space Φ of the text model of granularity k _kExpression, L _kSize for the space.To certain text sequence S=(w ₁w ₂..., w _T), its text vector V on granularity k _kExpression.Then the text feature of granularity k vector is represented with formula (4):

V _k＝{V _k1，V _k2，…，V _kLk} (6)

V _K1... wait each component of expression text vector.

The integrated of many granularities text representation in fact just carries out the text vector of different grain size feature integrated.Its key is many granularities text representation feature space merged and character representation process integrated.Mainly can adopt two kinds of methods: based on the integrated approach of overall weight, based on the integrated approach of text local feature.Exemplifying embodiment is as follows separately:

1) based on the integrated approach of overall weight

For each granularity text model is given separately overall weight r _k, varigrained text feature space merged obtaining new feature space, its process can be expressed as with relational expression (7):

$\mathrm{\Φ}=\underset{k=1}{\overset{N}{\∪}}{r}_k{\mathrm{\Φ}}_k---\left(7\right)$

In the enforcement for concrete text representation, can be implemented as the process that the text vector of different grain size feature is weighted fusion, each granularity text feature is endowed the overall weight r of place text model _k, the processing procedure of the representing multiple graininess of text sequence can be represented with relational expression (8):

$V=r_1{\mathrm{<figure-callout\; id="1"\; label="V"\; filenames="A200710121078E00161.png"\; state="\{\{state\}\}">V</figure-callout>}}_1\&CirclePlus;r_2{V}_2\&CirclePlus;\&CenterDot;\&CenterDot;\&CenterDot;\&CirclePlus;r_N{V}_N$

$=\{r_1{V}_{11},r_1{V}_{12},\&CenterDot;\&CenterDot;\&CenterDot;,r_1{V}_{1\mathrm{<figure-callout\; id="1"\; label="L"\; filenames="A200710121078E00161.png"\; state="\{\{state\}\}">L</figure-callout>}1},r_2{\mathrm{<figure-callout\; id="21"\; label="V"\; filenames="A200710121078E00181.png,A200710121078E00201.png"\; state="\{\{state\}\}">V</figure-callout>}}_{21},r_2{\mathrm{<figure-callout\; id="22"\; label="V"\; filenames="A200710121078E00181.png"\; state="\{\{state\}\}">V</figure-callout>}}_{22},\&CenterDot;\&CenterDot;\&CenterDot;,r_2{V}_{2L2},\&CenterDot;\&CenterDot;\&CenterDot;,r_{\mathrm{<figure-callout\; id="1"\; label="N\; V\; N"\; filenames="A200710121078E00161.png"\; state="\{\{state\}\}">N</figure-callout>}}{V}_N{1}_{},r_N{V}_{N2},\&CenterDot;\&CenterDot;\&CenterDot;,r_N{V}_{\mathrm{NLN}}\}$

(8)

2) based on the integrated approach of text local feature

Integrated approach based on the text local feature is complicated more, but can obtain better effect.

At first, the language of each layer unit unit is expressed as respectively in the N layer n-gram representation model of text sequence S:

L _k＝{(w _i-k+1…w _i)|i＝1，…T}，k＝1，…，N； (9)

The distribution probability of the unit i of language unit of layer k is represented with following formula:

p _k(i)＝P(w _i-k+1…w _i) (10)

Carry out weight calculation and setting by the relativity of its distribution probability parameter between the language unit characteristic item of each rank n-gram model, the weight that the grain size characteristic that probability is high more obtains is big more.For tilting to the coarseness feature, the weight of first feature of respectively speaking in the granularity model of high-order is set as 1.0, the weight of the language of other granularity models unit feature then can be made as the resulting surplus value of probability that itself distribution probability deducts the high-order grain size characteristic, is distributed in the weight q of text part for each grain size characteristic by the process of a recurrence _k(i).This process can be represented with following two steps:

At first, the weight of characteristic item all is made as 1.0 in the granularity model of high-order, is shown below:

q _k(i)＝1.0；k＝N (11)

Secondly, k asks for by the recursive fashion of k descending less than the weight of the characteristic item in each layer granularity model of N, and its process is represented with relational expression (12):

$\left\{\begin{array}{c}{q}_k\left(i\right)=(p_k\left(i\right)-(p_{k+1}\left(i\right)+p_{k+1}(i-1)-p_{k+2}(i-1)))/p_k\left(i\right);\\ k=1,\&CenterDot;\&CenterDot;\&CenterDot;,N-1;\mathrm{if\; k}=N-1\mathrm{then}{p}_{k+2}\left(i\right)=0;\\ q_k\left(i\right)=0;\mathrm{if}{q}_k\left(i\right)<0\end{array}\right.---\left(12\right)$

Make that text sequence S is tn in the corresponding k-gram characteristic item sequence number of k layer model i language unit _kAnd the attribute weights of k layer model j characteristic item are tw (i), _k(j), then the k-gram characteristic item weights sequence of text sequence can be represented with following form:

S′＝(q _k(1)·tw _k(tn _k(1))，q _k(2)·tw _k(tn _k(2))，…，q _k(T)·tw _k(tn _k(T)))

(13)

The text vector that obtains each text model correspondence is on this basis used Expression.Be similar to integrated approach, for each granularity text model is given different overall weight r based on overall weight _kThe representing multiple graininess gained text vector of text sequence can be expressed as with relational expression (14):

$V\&prime;=r_1{\mathrm{<figure-callout\; id="1"\; label="V"\; filenames="A200710121078E00161.png"\; state="\{\{state\}\}">V</figure-callout>}}_1^{\&prime;}\&CirclePlus;r_2{V}_2^{\&prime;}\&CirclePlus;\&CenterDot;\&CenterDot;\&CenterDot;\&CirclePlus;r_N{V}_N^{\&prime;}$

$=\{r_1{V}_{11}^{\&prime;},r_1{V}_{12}^{\&prime;},\&CenterDot;\&CenterDot;\&CenterDot;,r_1{V}_{1\mathrm{<figure-callout\; id="1"\; label="L"\; filenames="A200710121078E00161.png"\; state="\{\{state\}\}">L</figure-callout>}1}^{\&prime;},r_2{\mathrm{<figure-callout\; id="21"\; label="V"\; filenames="A200710121078E00181.png,A200710121078E00201.png"\; state="\{\{state\}\}">V</figure-callout>}}_{21}^{\&prime;},r_2{\mathrm{<figure-callout\; id="22"\; label="V"\; filenames="A200710121078E00181.png"\; state="\{\{state\}\}">V</figure-callout>}}_{22}^{\&prime;},\&CenterDot;\&CenterDot;\&CenterDot;,r_2{V}_{2L2}^{\&prime;},\&CenterDot;\&CenterDot;\&CenterDot;,r_{\mathrm{<figure-callout\; id="1"\; label="N\; V\; N"\; filenames="A200710121078E00161.png"\; state="\{\{state\}\}">N</figure-callout>}}{V}_N^{}{1}_{\&prime;}^{},r_N{V}_{N2}^{\&prime;},\&CenterDot;\&CenterDot;\&CenterDot;,r_N{V}_{\mathrm{NLN}}^{\&prime;}\}$

(14)

By distributing different overall weight r _k, we can give the importance of different grain size text model in many granularities text representation.Defaultly can all be made as 1.0.As if the weight of only giving 1 to certain one deck, it is 0 that other layer is all composed, and then multilayer n-gram model deteriorates to common single n-gram model, and in fact we can unite multilayer n-gram model and common n-gram model like this.For example, r _m=1 and r _k=0 (during k ≠ m), multilayer n-gram model is degenerated becomes common m rank n-gram.This has realized a Unified frame of many granularities text representation.

After having obtained the vectorization representation of text, we can utilize the text to represent to carry out the related application of text information processing, in text classification, can utilize text vector to train all kinds of sorters and use sorter to the target text processing of classifying.Test shows in text classification research on the widely used Reuters-21578 data set, under the situation of using identical feature extracting method and sorter, the inventive method is compared simple grain degree method can improve 2～3 percentage points of classification accuracies.

Claims (11)

1. the method for representing multiple graininess of a text message is characterized in that, may further comprise the steps:

Step S1: utilize the study of many granularities text representation model, training generates the text model of many granularities;

Step S2: integrated based on many granularities text feature model, form many granularities of text message integrated representation.

2. according to the method for representing multiple graininess of the described text message of claim 1, it is characterized in that: described many granularities text representation model is made of a plurality of varigrained text models, form the model of sandwich construction, interrelated between each layer granularity text model feature unit; This model is used for the representing multiple graininess of text message.

3. the method for representing multiple graininess of text message as claimed in claim 1, it is characterized in that: the learning process of many granularities text representation model may further comprise the steps:

Step S11: varigrained text model is learnt respectively;

Step S12: analyze the relation between the different grain size text model feature unit, the different grain size text model is combined constitutes the text representation model of sandwich construction.

4. according to the method for representing multiple graininess of the described text message of claim 1, it is characterized in that: described many granularities text feature is integrated, be integrated into varigrained text model based on many granularities text feature of overall weight and give separately weight, their text feature space is weighted to merge obtains new text feature space, thereby merged many granularities text feature at new feature space.

5. according to the method for representing multiple graininess of the described text message of claim 1, it is characterized in that: described many granularities text feature is integrated, based on integrated the steps include: of many granularities text feature of text local feature

Step S21: the text feature that obtains a plurality of granularity text models is represented;

Step S22: the weights that calculate the local different grain size feature of text;

Step S23: integrated many grain size characteristics are represented text.

6. according to the method for representing multiple graininess of the described text message of claim 5, it is characterized in that, described integrated many grain size characteristics are represented text, it is the relation of inclusion of utilizing between the local different grain size text feature of text, probability distribution parameters according to varigrained text feature unit, the weights of each granularity text feature of analytical calculation text part are represented many grain size characteristics of text integrated on this basis.

7. as the method for representing multiple graininess of claim 4 and 5 described text messages, it is characterized in that: described many granularities text feature is integrated to be taked based on many granularities text feature of overall weight integratedly, or takes based on many granularities text feature of text local feature integrated.

8. the method for representing multiple graininess of text message as claimed in claim 1 is characterized in that: described text feature, and employed text feature is provided by a plurality of varigrained text models, and the study of text model realizes under the support of corpus; Wherein, varigrained text representation model is the text representation of same type, or dissimilar text representations.

9. as the method for representing multiple graininess of claim 1 and 5 described text messages, it is characterized in that: if it uses integrated based on many granularities text feature of text local feature in integrated many granularities text feature of step S2, then in the learning process of each granularity text model of step S1, need the probability distribution parameters of study different grain size text feature unit.

10. the method for representing multiple graininess of text message as claimed in claim 4, it is characterized in that: the many grain size characteristics of described text based on overall weight are integrated, be in text representation, by adjusting the overall separately weight that varigrained text model is endowed, if wherein have only the overall weight of a text model non-vanishing, then the method for representing multiple graininess of text is degenerated and is become the simple grain kilsyth basalt and show.

11. the method for representing multiple graininess of text message as claimed in claim 5, it is characterized in that: the many grain size characteristics of described text based on the text local feature are integrated, be in text representation, by adjusting the overall separately weight that varigrained text model is endowed, if wherein have only the overall weight of a text model non-vanishing, then the representing multiple graininess of text is degenerated and is become the simple grain kilsyth basalt and show.

Images