CN104166644A - Term translation mining method based on cloud computing - Google Patents

Abstract

The invention discloses a term translation mining method based on cloud computing. The term translation mining method based on cloud computing comprises construction of a cloud computing hardware and software system platform and a term translation mining technology for extracting translations from the cloud computing hardware and software system platform. The term translation mining technology comprises the steps of effective summary resource acquisition, automatic extraction of candidate translation units and translation selection. Construction of the cloud computing hardware and software system platform comprises the steps of server group construction, parallel computing system establishment, distributed database system establishment and cloud computing system establishment and further comprises the steps of conducting interaction with a client-side in the mode of internet. Through the mode, the term translation mining method based on cloud computing can be applied to the relevant fields such as terminology dictionary compiling, machine translation, information retrieval, question-answering systems and subject content analyzing; the requirements for quickly obtaining terminologies in a field on the internet of people can be met.

Description

A kind of term translation method for digging based on cloud computing

Technical field

The present invention relates to machine translation technical field, particularly relate to a kind of term translation method for digging based on cloud computing.

Background technology

Along with mechanical translation, cross-language information retrieval, Web Research on Mining deeply and the fast development of Internet resources, across the Web Research on Mining field of linguistry and resource, started new climax.On internet, comprise a large amount of bilingual alignments, bilingual comparable or bilingual mixing webpage, from these webpages, excavate study various fine-grained across linguistry, as dictionary for translation, translation template, bilingual term, bilingualism corpora etc., can both provide important basic resource to improving traditional machine translation mothod and cross-language information retrieval techniques.

Meanwhile, increased along with international exchange, spoken and written languages communication disorder how to eliminate people becomes one of problem with strongest influence power.Structure and translation knowledge automatic acquisition technology across language resource storehouse has great importance for the practical of problem such as propel machine translation, cross-language information retrievals on a large scale.

Along with scientific and technical development, proper noun and neologisms constantly emerge in large numbers, and are promoting natural language and are constantly developing.Although there is MRD and online dictionary, still can not cover emerging various translation, more can not obtain in time the neologisms of each subject, each field appearance.Foundation and a dictionary of renewal need to expend a large amount of human and material resources and financial resources, and consuming time huge, affect the use value of dictionary.In network term translation method for digging, also exist some shortcomings at present, require further study and explore.First, from the depth & wideth of knowledge excavation, existing research can only be obtained the bilingual text in same website, can not obtain the bilingual text that is distributed in different web sites and in the bilingual resource of mixing webpage.Secondly, from search engine input term carries out translation excavation, knowledge used and result are not preserved at every turn, even if input same term next time, also will carry out the work of repetition.Further investigation for these aspects has important value to the practical application of the bilingual resource automatic acquisition research based on Web undoubtedly.

Summary of the invention

The technical matters that the present invention mainly solves is to provide a kind of term translation method for digging based on cloud computing, can be applied to the association area such as writing, mechanical translation, information retrieval, question answering system, subject content analysis of Terminology Dictionary; Can meet the needs of the technical term in Shang Mou field, people's quick obtaining internet, for researchist reads professional offering of materials translation information, also writing and the renewal for terminological dictionary provides resource guarantee.Meanwhile, explore cloud computing in the application of different field, for enterprise, build inner cloud, capture the gordian technique in cloud computing, make enterprise provide the translation service of term translation to become possibility to society.When ensureing business economic interests for society provides service.

For solving the problems of the technologies described above, the technical scheme that the present invention adopts is: a kind of term translation method for digging based on cloud computing is provided, comprises: the term translation digging technology of building and extract translation from described cloud computing hardware and software system platform of cloud computing hardware and software system platform; Described term translation digging technology comprises the obtaining of effective digest resources, the Automatic Extraction of candidate's translation unit and the selection of translation; The building of described cloud computing hardware and software system platform comprises builds server zone, sets up concurrent computational system, builds distributed data base system, mutual with mode and the client of network, structure cloud computing system; Described cloud computing system forms storage and arithmetic system by a plurality of servers and disk array, in each server, distributed data base is installed, and sets up concurrent computational system and searches the Terminology Translation knowledge in database; Terminology Translation knowledge is set up to multistage label, first by searching algorithm, in labels at different levels, inquire about, corresponding Terminology Translation information is provided after finding, satisfy condition in Shi Congku translation is returned to user, if cannot find the translation of term in inner groups, by the interface of inner cloud and outside cloud, to outside cloud, search the translation of term.

Preferably, described distributed data base system is comprised of one group of data unifying in logic, be physically scattered on the some websites of computer network, adopt SQL Server (a kind of relational database management system) to using XML (a kind of extend markup language) as intermediary, realize the data collaborative query processing between distributed data base.

Preferably, obtaining of described effective digest resources utilizes a kind of enquiry expanding method based on co-occurrence information, first submit source query word to search engine, obtain the source language summary info that comprises source inquiry, then utilize TF-IDF (a kind of information retrieval and information are prospected weighting technique) to inquire about the theme vocabulary of co-occurrence from the extraction of source language summary info and the source that obtain; Obtain after theme vocabulary, from bilingual dictionary, search the translation of theme vocabulary, the translation of source inquiry and these theme vocabulary is carried out across language extension, again submit to search engine to obtain bilingual digest resources the inquiry after expansion.

Preferably, the Automatic Extraction of described candidate's translation unit adopts FCM clustering algorithm to extract candidate's translation unit from the bilingual digest resources obtaining, in conjunction with frequency measure of variation and adjacency information; FCM formula is as follows:

$R\left(S\right)=\frac{f\left(S\right)}{1+\sqrt{\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{(x_i-\stackrel{\‾}{x})}^2}}$

Wherein, S is a Chinese character string, and f (S) is the frequency of character string S, x _ithe frequency of each character in S, it is the average frequency of all characters in S.

Preferably, the selection of described translation is extracted translation by comprehensive employing frequency-distance model, top layer template matches and transliteration model from the set of candidate's translation unit.

Preferably, the formula of described frequency-distance model is as follows:

$F\_Q(s,t)=\frac{\underset{J}{\mathrm{\Σ}}\underset{k}{\mathrm{\Σ}}\frac{1}{d_k(s,t)}}{{\max }_{\mathrm{fre}-\mathrm{dis}}}$

Wherein, s is source inquiry, and t is one of them candidate unit, the sum that J is all summaries, and K is s in a summary, the number of times of t co-occurrence, d _k(s, t) is s, the distance of the k time co-occurrence of t in a summary, max _fre-dismaximal value reciprocal for all candidate unit middle distances.

Preferably, the contribution margin of described top layer template matches adopts following formula to calculate:

$\mathrm{SP}(s,t)=\frac{N_{\mathrm{mathing}}}{{\max }_{\mathrm{num}}}$

Wherein, s is source inquiry, and t is a candidate unit, and molecule is s, the total degree of the template of t coupling, and denominator is the maximal value of matching times in all candidates.

Preferably, described transliteration model splits into english syllable sequence by source English language query, then calculates the matching probability of Chinese character in english syllable and candidate's Chinese unit, and then the probability of translation each other between calculating source inquiry and candidate unit; The score of described transliteration model is calculated by following formula:

$\mathrm{Trl}(s,t)=\frac{P(s,t)}{D(s,t)}$

Wherein molecule is s, the probability of t co-occurrence, and denominator is the number of different syllables in s and t, P (s, t) is defined as follows:

$P(s,t)\≈\underset{i=1}{\overset{\min (m,n)}{\mathrm{\Π}}}(1-{\mathrm{\γ}}_1)\mathrm{prob}(e_i,c_i)$

Wherein, γ ₁for smoothing factor, prob (e _i, c _i) be english syllable e _iwith Chinese character c _ithe probability of coupling, D (s, t) is defined as:

D(s,t)＝ε+|m-n|

Wherein, ε is attenuation parameter, and m is the english syllable sum of source inquiry, and n is the sum of Chinese character in a candidate unit.

The invention has the beneficial effects as follows: the present invention can be applied to the association areas such as the writing, mechanical translation, information retrieval, question answering system, subject content analysis of Terminology Dictionary; Can meet the needs of the technical term in Shang Mou field, people's quick obtaining internet.

Accompanying drawing explanation

Fig. 1 is a kind of term translation method for digging medium cloud computing hardware based on cloud computing of the present invention and the integral frame schematic diagram of soft ware platform;

The logical architecture figure of distributed data base system in a kind of term translation method for digging based on cloud computing of Fig. 2 shown in being.

Embodiment

Below in conjunction with accompanying drawing, preferred embodiment of the present invention is described in detail, thereby so that advantages and features of the invention can be easier to be it will be appreciated by those skilled in the art that, protection scope of the present invention is made to more explicit defining.

Refer to Fig. 1 and Fig. 2, the embodiment of the present invention comprises:

A term translation method for digging based on cloud computing, comprising: the term translation digging technology of building and extract translation from described cloud computing hardware and software system platform of cloud computing hardware and software system platform; Described term translation digging technology comprises the obtaining of effective digest resources, the Automatic Extraction of candidate's translation unit and the selection of translation; The building of described cloud computing hardware and software system platform comprises builds server zone, sets up concurrent computational system, builds distributed data base system, mutual with mode and the client of network, structure cloud computing system; Described cloud computing system forms storage and arithmetic system by a plurality of servers and disk array, in each server, distributed data base is installed, and sets up concurrent computational system and searches the Terminology Translation knowledge in database; Terminology Translation knowledge is set up to multistage label, first by searching algorithm, in labels at different levels, inquire about, corresponding Terminology Translation information is provided after finding, satisfy condition in Shi Congku translation is returned to user, if cannot find the translation of term in inner groups, by the interface of inner cloud and outside cloud, to outside cloud, search the translation of term.

As shown in Figure 2, described distributed data base system is comprised of one group of data unifying in logic, be physically scattered on the some websites of computer network, adopt SQL Server to using XML as intermediary, realize the data collaborative query processing between distributed data base.

In term translation digging technology, effectively obtaining of digest resources utilized a kind of enquiry expanding method based on co-occurrence information, first submit source query word to search engine, obtain the source language summary info that comprises source inquiry, then utilize TF-IDF to inquire about the theme vocabulary of co-occurrence from the extraction of source language summary info and the source that obtain; Obtain after theme vocabulary, from bilingual dictionary, search the translation of theme vocabulary, the translation of source inquiry and these theme vocabulary is carried out across language extension, again submit to search engine to obtain bilingual digest resources the inquiry after expansion.

The bilingual digest resources that in term translation digging technology, the Automatic Extraction of candidate's translation unit adopts FCM clustering algorithm to obtain from previous step, extract candidate's translation unit, in conjunction with frequency measure of variation and adjacency information, FCM is based on following two observations, one is that the frequency of each character in candidate's translation unit is similar, two is that after legal candidate's translation unit is expanded with an extra character, the unit frequency after expansion can obviously reduce; FCM formula is as follows:

$R\left(S\right)=\frac{f\left(S\right)}{1+\sqrt{\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{(x_i-\stackrel{\‾}{x})}^2}}$

Wherein, S is a Chinese character string, and f (S) is the frequency of character string S, x _ithe frequency of each character in S, it is the average frequency of all characters in S.

In term translation digging technology, the selection of translation is extracted translation by comprehensive employing frequency-distance model, top layer template matches and transliteration model from the set of previous step candidate translation unit.

Frequency-distance model: real translation frequency common and source inquiry co-occurrence is higher, and secondly, candidate unit is nearer with source inquiry in summary, and its probability that is correct translation is just larger, and the formula of described frequency-distance model is as follows:

$F\_Q(s,t)=\frac{\underset{J}{\mathrm{\Σ}}\underset{k}{\mathrm{\Σ}}\frac{1}{d_k(s,t)}}{{\max }_{\mathrm{fre}-\mathrm{dis}}}$

Wherein, s is source inquiry, and t is one of them candidate unit, the sum that J is all summaries, and K is s in a summary, the number of times of t co-occurrence, d _k(s, t) is s, the distance of the k time co-occurrence of t in a summary, max _fre-dismaximal value reciprocal for all candidate unit middle distances.

Top layer template matches: some asian language users customs are labeled in the translation of some terms in bracket.Punctuation mark in the middle of source inquiry and corresponding translation is important information.We can utilize these information to improve the quality of the translation of extraction.First we carry out the study of top layer template by some English-Chinese words to submission search engine.The template that we obtain is as follows:

If candidate's translation unit and source match query most templates, its probability as correct translation is larger so.The contribution margin of described top layer template matches adopts following formula to calculate:

$\mathrm{SP}(s,t)=\frac{N_{\mathrm{mathing}}}{{\max }_{\mathrm{num}}}$

Wherein, s is source inquiry, and t is a candidate unit, and molecule is s, the total degree of the template of t coupling, and denominator is the maximal value of matching times in all candidates.

Transliteration model: described transliteration model splits into english syllable sequence by source English language query, then calculates the matching probability of Chinese character in english syllable and candidate's Chinese unit, and then the probability of translation each other between the inquiry of calculating source and candidate unit; The score of described transliteration model is calculated by following formula:

$\mathrm{Trl}(s,t)=\frac{P(s,t)}{D(s,t)}$

Wherein molecule is s, the probability of t co-occurrence, and denominator is the number of different syllables in s and t, P (s, t) is defined as follows:

$P(s,t)\≈\underset{i=1}{\overset{\min (m,n)}{\mathrm{\Π}}}(1-{\mathrm{\γ}}_1)\mathrm{prob}(e_i,c_i)$

Wherein, γ ₁for smoothing factor, prob (e _i, c _i) be english syllable e _iwith Chinese character c _ithe probability of coupling, D (s, t) is defined as:

D(s,t)＝ε+|m-n|

Wherein, ε is attenuation parameter, and m is the english syllable sum of source inquiry, and n is the sum of Chinese character in a candidate unit.

The translation extracting by said method finally returns to client, completes translation.

The present invention can be applied to the association areas such as the writing, mechanical translation, information retrieval, question answering system, subject content analysis of Terminology Dictionary; Can meet the needs of the technical term in Shang Mou field, people's quick obtaining internet, for researchist reads professional offering of materials translation information, also writing and the renewal for terminological dictionary provides resource guarantee.Meanwhile, explore cloud computing in the application of different field, for enterprise, build inner cloud, capture the gordian technique in cloud computing, make enterprise provide the translation service of term translation to become possibility to society.When ensureing business economic interests for society provides service.

The foregoing is only embodiments of the invention; not thereby limit the scope of the claims of the present invention; every equivalent structure or conversion of equivalent flow process that utilizes instructions of the present invention and accompanying drawing content to do; or be directly or indirectly used in other relevant technical fields, be all in like manner included in scope of patent protection of the present invention.

Claims (8)

1. the term translation method for digging based on cloud computing, is characterized in that, comprising: the term translation digging technology of building and extract translation from described cloud computing hardware and software system platform of cloud computing hardware and software system platform; Described term translation digging technology comprises the obtaining of effective digest resources, the Automatic Extraction of candidate's translation unit and the selection of translation; The building of described cloud computing hardware and software system platform comprises builds server zone, sets up concurrent computational system, builds distributed data base system, mutual with mode and the client of network, structure cloud computing system; Described cloud computing system forms storage and arithmetic system by a plurality of servers and disk array, in each server, distributed data base is installed, and sets up concurrent computational system and searches the Terminology Translation knowledge in database; Terminology Translation knowledge is set up to multistage label, first by searching algorithm, in labels at different levels, inquire about, corresponding Terminology Translation information is provided after finding, satisfy condition in Shi Congku translation is returned to user, if cannot find the translation of term in inner groups, by the interface of inner cloud and outside cloud, to outside cloud, search the translation of term.

2. a kind of term translation method for digging based on cloud computing according to claim 1, it is characterized in that: described distributed data base system is comprised of one group of data unifying in logic, be physically scattered on the some websites of computer network, adopt SQL Server to using XML as intermediary, realize the data collaborative query processing between distributed data base.

3. a kind of term translation method for digging based on cloud computing according to claim 1, it is characterized in that: obtaining of described effective digest resources utilizes a kind of enquiry expanding method based on co-occurrence information, first submit source query word to search engine, obtain the source language summary info that comprises source inquiry, then utilize TF-IDF to inquire about the theme vocabulary of co-occurrence from the extraction of source language summary info and the source that obtain; Obtain after theme vocabulary, from bilingual dictionary, search the translation of theme vocabulary, the translation of source inquiry and these theme vocabulary is carried out across language extension, again submit to search engine to obtain bilingual digest resources the inquiry after expansion.

4. a kind of term translation method for digging based on cloud computing according to claim 1, it is characterized in that: the Automatic Extraction of described candidate's translation unit adopts FCM clustering algorithm to extract candidate's translation unit from the bilingual digest resources obtaining, in conjunction with frequency measure of variation and adjacency information; FCM formula is as follows:

$R\left(S\right)=\frac{f\left(S\right)}{1+\sqrt{\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{(x_i-\stackrel{\‾}{x})}^2}}$

Wherein, S is a Chinese character string, and f (S) is the frequency of character string S, x _ithe frequency of each character in S, it is the average frequency of all characters in S.

5. a kind of term translation method for digging based on cloud computing according to claim 1, is characterized in that: the selection of described translation is extracted translation by comprehensive employing frequency-distance model, top layer template matches and transliteration model from the set of candidate's translation unit.

6. a kind of term translation method for digging based on cloud computing according to claim 5, is characterized in that: the formula of described frequency-distance model is as follows:

$F\_Q(s,t)=\frac{\underset{J}{\mathrm{\Σ}}\underset{k}{\mathrm{\Σ}}\frac{1}{d_k(s,t)}}{{\max }_{\mathrm{fre}-\mathrm{dis}}}$

Wherein, s is source inquiry, and t is one of them candidate unit, the sum that J is all summaries, and K is s in a summary, the number of times of t co-occurrence, d _k(s, t) is s, the distance of the k time co-occurrence of t in a summary, max _fre-dismaximal value reciprocal for all candidate unit middle distances.

7. a kind of term translation method for digging based on cloud computing according to claim 5, is characterized in that: the contribution margin of described top layer template matches adopts following formula to calculate:

$\mathrm{SP}(s,t)=\frac{N_{\mathrm{mathing}}}{{\max }_{\mathrm{num}}}$

Wherein, s is source inquiry, and t is a candidate unit, and molecule is s, the total degree of the template of t coupling, and denominator is the maximal value of matching times in all candidates.

8. a kind of term translation method for digging based on cloud computing according to claim 5, it is characterized in that: described transliteration model splits into english syllable sequence by source English language query, then calculate the matching probability of Chinese character in english syllable and candidate's Chinese unit, and then the probability of translation each other between calculating source inquiry and candidate unit; The score of described transliteration model is calculated by following formula:

$\mathrm{Trl}(s,t)=\frac{P(s,t)}{D(s,t)}$

Wherein molecule is s, the probability of t co-occurrence, and denominator is the number of different syllables in s and t, P (s, t) is defined as follows:

$P(s,t)\≈\underset{i=1}{\overset{\min (m,n)}{\mathrm{\Π}}}(1-{\mathrm{\γ}}_1)\mathrm{prob}(e_i,c_i)$

Wherein, γ ₁for smoothing factor, prob (e _i, c _i) be english syllable e _iwith Chinese character c _ithe probability of coupling, D (s, t) is defined as:

D(s,t)＝ε+|m-n|

Wherein, ε is attenuation parameter, and m is the english syllable sum of source inquiry, and n is the sum of Chinese character in a candidate unit.