CN1354422A - Computer new word learning method and system - Google Patents
Computer new word learning method and system Download PDFInfo
- Publication number
- CN1354422A CN1354422A CN 00132955 CN00132955A CN1354422A CN 1354422 A CN1354422 A CN 1354422A CN 00132955 CN00132955 CN 00132955 CN 00132955 A CN00132955 A CN 00132955A CN 1354422 A CN1354422 A CN 1354422A
- Authority
- CN
- China
- Prior art keywords
- speech
- sub
- word set
- word
- neologisms
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Machine Translation (AREA)
Abstract
The invention relates to a method for learning new word by computer. It consists of child word recording program, first deleting program and secondary deleting program. The function of the child word recording program is to decompose at least one no word part recorded on the no word set into at least one child word that is recorded to the child word set. The no word part means that in a computer accessible file, any adjacent single character can not form a part of glossary recognizable by computer. The first delecting program calculates number of presence of each child word and deletes the child word whose number of presence is less than predetermined value from child word set. The second detecting program chooses first child word and second word from the child word set. When the first child word is contained in the second word and the number of presence of first child word is not largeer than the number of presence of second child word, then the first child word is deleted from the set. Thus computer recognizable new word is produced from child word set.
Description
The present invention relates to a kind of computer new word learning method and system, analyze but be particularly related to a kind of part that all can't constitute computing machine identification vocabulary, but use quantity Calculation machine new word learning method and the system that obtains neologisms and increase computing machine identification vocabulary at any adjacent individual character in the file.
In the file of alphabetic writings such as English, French or German,, the problem that could understand its real meaning do not cut apart in sentence so do not exist owing to all have blank to be separated between each word.Yet, in Chinese, Japanese or Korean etc., there is no between each word in the blank file of being separated, if can't the content of file be cut, why can't understand its real implication, and cause the mistake in the interpretation.
So-called " disconnected speech " is meant the sentence of being made up of the word of bunchiness Chinese, Japanese or Korean etc. cut apart, and makes sentence be cut into many significant vocabulary.In the application of many Language Processing, as correction, translation or speech recognition etc., all must want earlier file is broken behind the speech, can further process.
Along with the development of computing machine science and technology, had with the break method and system of speech of computing machine to occur.With compare with the speech that manually breaks, the disconnected speech of computing machine can significantly reduce the required time.Yet one of difficulty place of the disconnected speech of computing machine is when it meets the vocabulary that it can't identification, and as if not with the new glossary of artificial input, it can't do suitable processing.
At the problems referred to above, purpose of the present invention is for providing a kind of computer new word learning method and system, and it is the new term in the learning files automatically.
Another object of the present invention is for providing a kind of computer new word learning method and system, and it can be as the basis of compuword database update master maintenance.
For reaching above-mentioned purpose, comprise a sub-speech logging program, one first delete program and one second delete program according to computer new word learning method of the present invention.Sub-speech logging program is that at least one no speech that will be recorded in a no word set partly resolves at least one sub-speech, and resulting sub-speech is recorded in the sub-word set, wherein do not have speech and be meant partly that in the file of an embodied on computer readable any adjacent individual character all can't constitute the part of the cognizable vocabulary of computing machine.First delete program is an occurrence number of calculating each sub-speech respectively, and occurrence number is deleted in sub-word set less than the sub-speech of a predetermined value.Second delete program is to choose one first sub-speech inequality and one second sub-speech in sub-word set in regular turn, when the first sub-speech is contained in the second sub-speech, and when the occurrence number of the first sub-speech is not more than the second sub-speech, the first sub-speech is deleted from sub-word set, used by producing the cognizable neologisms of computing machine in this sub-word set.
The present invention also discloses a kind of computer new word learning system, and it comprises a sub-speech logging modle, one first removing module and one second removing module.Sub-speech logging modle is that at least one no speech that will be recorded in a no word set partly resolves at least one sub-speech, and should be recorded in the sub-word set by sub-speech, wherein do not have speech and be meant partly that in the file of an embodied on computer readable any adjacent individual character all can't constitute the part of the cognizable vocabulary of computing machine.First removing module is an occurrence number of calculating each these sub-speech respectively, and occurrence number is deleted in sub-word set less than the sub-speech of a predetermined value.Second removing module is then chosen one first sub-speech inequality and one second sub-speech in regular turn in sub-word set, when the first sub-speech is comprised in the second sub-speech, and when the occurrence number of the first sub-speech is not more than the second sub-speech, the first sub-speech is deleted from sub-word set, used by producing the cognizable neologisms of computing machine in the sub-word set.
Hereinafter with reference to relevant drawings, computer new word learning method and system according to preferred embodiment of the present invention are described, wherein components identical will be illustrated with identical reference marks.
Fig. 1 is a process flow diagram, shows the flow process according to the computer new word learning method of preferred embodiment of the present invention.
Fig. 2 is a synoptic diagram, shows the structure according to the computer new word learning method of preferred embodiment of the present invention.
With reference to Fig. 1, be to carry out a speech identification program 11 earlier according to the computer new word learning method 1 of preferred embodiment of the present invention, handle with the speech that breaks of the file 51 to an embodied on computer readable.As previously mentioned, so-called " disconnected speech " is meant the sentence of being made up of the word of bunchiness Chinese, Japanese or Korean etc. cut apart, and makes sentence be cut into many significant vocabulary.In the present embodiment, be to use a kind of known " dictionary statistics formula break morphology " to come to the file speech that breaks, that is, utilize vocabulary to form probability and vocabulary length etc. the file that is cut is assessed, in the hope of the disconnected speech result of the best.Yet the person of noting is familiar with this operator and can adopts other disconnected morphology to come the file speech that breaks easily, and do not break away from spirit of the present invention and category.
Then, do not have speech partial record program 12, with the no speech partial record in the file 51 in a no word set 52.In the present invention, so-called " no speech part " is meant that in file 51 any adjacent individual character all can't constitute the part of the cognizable vocabulary of computing machine.For example, if having following sentence in the file 51:
" Wang Ming yesterday visit Li Xiaohua " is in this sentence, because in " Wang Ming " and " Li Xiaohua " these two parts, the various combinations of individual character (as, " Li Xiaohua " can have three kinds of combinations such as " Lee is little ", " little China ", " Li Xiaohua ") all can't therefore can be become two " no speech parts " by System Discrimination.That is this disconnected speech result can become:
" Wang Ming yesterday visit Li Xiaohua "
These two no speech parts of " Wang Ming " and " Li Xiaohua " will become the combination of individual character.
Then, in sub-speech logging program 13, no word set 52 does not respectively have speech part can be broken down at least one sub-speech, and the sub-speech that will decompose gained is recorded in the sub-word set 53." Wang Ming yesterday visit Li Xiaohua " this sentence with the front be an example, and in sub-speech logging program 13, no speech partly " Li Xiaohua " can be broken down into " Lee is little ", " little China " and " Li Xiaohua " three sub-speech.In other words, but sub-speech logging program 13 each no speech energon speech partly all can be decomposed out.
Then, first delete program 14 calculates the occurrence number of each sub-speech respectively, and occurrence number is deleted in this sub-word set less than the sub-speech of a predetermined value.In this program, " occurrence number " of so-called each sub-speech can refer to that each sub-speech in the occurrence number of originally not having in the word set 52, also can refer to the occurrence number of each sub-speech in sub-word set 53.The visual actual state of those skilled in the art is selected any computing method.
When the occurrence number of certain sub-speech in whole part of file 51 is very few, represent that it is to occur occasionally in file 51, in sub-word set 53 so it can be deleted.For example, if " Wang Ming " only occurs once in whole part of file 51, and " Li Xiaohua " occurred tens times in whole part of file, then clearly, " Wang Ming " is not the neologisms with recorded value, and it only is a sub-speech that occurs occasionally, and " Li Xiaohua " this sub-speech is for file 51, because occurrence number is numerous, so be one to have the neologisms of recorded value.
As for the size of predetermined value, then visual actual state is set.For example, the mode that can manually import is changed preset value, or according to the number of words of file 51, comes maneuverability to adjust the size of predetermined value.So can set different neologisms study standards at different files.
In second delete program 15, be to choose one first sub-speech inequality and one second sub-speech in regular turn from this sub-word set 53 earlier.Then, when the first sub-speech is contained in the second sub-speech, and the occurrence number of the first sub-speech is when being not more than the second sub-speech, with the first sub-speech from sub-word set 53 deletions.For example, be the first sub-speech when choosing " Lee is little ", when " Li Xiaohua " is the second sub-speech, because " Lee is little " is to be contained in " Li Xiaohua ", so the occurrence number of " Lee is little " this sub-speech can equal the occurrence number of " Li Xiaohua " this sub-speech.In this case, be about to " Lee is little ", only stay " Li Xiaohua " this sub-speech from sub-word set 53 deletions.Identical reason is owing to " little China " this sub-speech also is contained in " Li Xiaohua ", so they also can be from sub-word set 53 deletions.So, can delete the short sub-speech that is contained in than eldest son's speech, keep the long sub-speech of length.
Then, in determining program 16, if sub-word set 53 is empty set, that is, in first delete program 14 and second delete program 15, all sub-speech are all deleted, then finish the flow process of whole computer new word learning method 1 according to preferred embodiment of the present invention.If in the sub-word set 53 sub-speech is arranged still, then carry out the 3rd delete program 17, only keep the maximum sub-speech of occurrence number, delete the less sub-speech of all other occurrence numbers.So, once will only can produce neologisms.
After producing neologisms, promptly there is not speech part segmentation procedure 18, partly delete with the no speech that will comprise neologisms, and will comprise in the no speech part of neologisms the new no speech part of the independent formation of part beyond the neologisms.That is, in the no speech part that has comprised neologisms, be positioned at individual character quantity before the neologisms and be two when above, be considered as another no speech part with being positioned at neologisms part before in the no speech part, and be added in the no word set 52.On the other hand, in the no speech part that has comprised neologisms, be positioned at individual character quantity after the neologisms and be two when above, then will not have and be positioned at neologisms part afterwards in the speech part and be considered as another no speech part, and it is added to no word set 52.
For example, if having one in addition in the file 51: " he is sitting in the front of Li Xiaohua ", because whole sentence does not all have the cognizable vocabulary of computing machine, so nine words of full sentence have constituted a no speech part, and comprising the neologisms of firm generation " Li Xiaohua ".At this no speech part, in no speech part segmentation procedure 18, this no speech part will be deleted, and be divided into two new no speech parts, be three words of neologisms " Li Xiaohua " " he is sitting in " before, and three words of neologisms " Li Xiaohua " " front " afterwards.
After no speech part segmentation procedure 18, promptly carry out sub-word set and empty program 19, so that sub-word set 53 is emptied, and get back to sub-speech logging program 13 and carry out the action that sub-speech decomposes again.
Via aforesaid flow process, can find out all possible neologisms in the file 51, and can not have influence on computing machine cognizable speech originally, and the speech of existing existence in the file 51.Therefore, it can be effectively carries out more suitable disconnected speech to the file of embodied on computer readable and handles.
As for the neologisms that produced, but visual actual state makes it become new computing machine identification vocabulary.For example, if the neologisms that produce are " ecommerce ", promptly consider this new term that just produces is in recent years added the lexical data base of computing machine, but make it become new computing machine identification vocabulary.So, will help the renewal and the maintenance of compuword database.
With reference to Fig. 2, comprise that according to the computer new word learning system 2 of preferred embodiment of the present invention a speech recognition module 21, one no speech partial record module 22, one sub-speech logging modle 23, one first removing module 24, one second removing module 25, one the 3rd removing module 26 and a no speech partly cut apart module 27.In the present embodiment, each module is the program module that is stored in the computer installation, it is to be recorded in a memory storage, in storer, Winchester disk drive or CD player etc., one CPU (central processing unit) is read after each module, can carry out the flow process of foregoing computer new word learning method 1, to find out the neologisms in the file 51.Yet, the person of noting, be familiar with this operator and also can carry out the modification of equivalence and further application it, for example each module making is become integrated circuit, with with The built-in in as electronic installations such as electronic dictionary or personal digital assistants, with disconnected speech that file 51 is carried out computer new word learning method 1 as previously mentioned and the work that produces neologisms, and do not exceed spirit of the present invention and category.
Computer new word learning system 2 can read file 51 in a memory storage (as storer) or recording medium (as disk sheet or discs), or reads file 51 via world-wide web from another network servicer.Lexical data base 30 required when carrying out the speech identification also can be stored in the memory storage or recording medium of an embodied on computer readable, so that the access of computer new word learning system 2.The neologisms that computer new word learning system 2 is produced also can be added in the lexical data base 30, with the action of it being safeguarded Yu upgrade.
According to computer new word learning method of the present invention and system is to utilize computer technology to come to the embodied on computer readable file speech that breaks, so that the sentence in the file is correctly cut into significant vocabulary.It helps the application of many Language Processing, as further developing of science and technology such as correction, translation or speech recognition.
According to computer new word learning method of the present invention and the system new term in the learning files automatically, file is done suitable disconnected speech processing.
Can be according to computer new word learning method of the present invention and system as the basis of compuword database update and maintenance.
The above only is an illustrative, but not is restricted person.Anyly do not break away from spirit of the present invention and category, and, all should be included in claims institute restricted portion its equivalent modifications of carrying out or change.
Claims (19)
1. computer new word learning method comprises following program:
One sub-speech logging program, be that at least one no speech that will be recorded in a no word set partly resolves at least one sub-speech, and should be recorded in the sub-word set by sub-speech, wherein this no speech partly is meant in the file of an embodied on computer readable, and any adjacent individual character all can't constitute the part of the cognizable vocabulary of computing machine;
One first delete program is an occurrence number of calculating each these sub-speech respectively, and occurrence number is deleted in this sub-word set less than the sub-speech of a predetermined value; And
One second delete program, be to choose one first sub-speech inequality and one second sub-speech in this sub-word set in regular turn, when this first sub-speech is contained in this second sub-speech, and the occurrence number of this first sub-speech is when being not more than this second sub-speech, this first sub-speech is deleted from this sub-word set
Use by producing the cognizable neologisms of computing machine in this sub-word set.
2. computer new word learning method as claimed in claim 1 also comprises:
One speech identification program is a file to be carried out the speech identification handle; And
One no speech partial record program is when having at least one no speech part in this document, should no speech partial record in this no word set.
3. computer new word learning method as claimed in claim 1 also comprises:
One determining program is to judge whether this sub-word set is empty set, and when this sub-word set is empty set, finishes the flow process of this computer new word learning method.
4. computer new word learning method as claimed in claim 1 also comprises:
One the 3rd delete program is after this second delete program, also will this sub-word set in sub-speech deletion beyond the maximum sub-speech of occurrence number.
5. computer new word learning method as claimed in claim 1 also comprises:
One no speech part segmentation procedure comprises
The no speech part that will comprise these neologisms removes from this no word set;
In the no speech part that comprises these neologisms, be positioned at individual character quantity before these neologisms and be two when above, be considered as another no speech part with being positioned at these neologisms part before in this no speech part, and it is added in this no word set; And
In the no speech part that comprises these neologisms, be positioned at individual character quantity after these neologisms and be two when above, be considered as another no speech part with being positioned at these neologisms part afterwards in this no speech part, and it is added in this no word set.
6. computer new word learning method as claimed in claim 1 also comprises:
One sub-word set empties program, is to empty this sub-word set and get back to this sub-speech logging program.
7. computer new word learning method as claimed in claim 1, wherein
This predetermined value is 2.
8. computer new word learning system comprises:
One sub-speech logging modle, be that at least one no speech that will be recorded in a no word set partly resolves at least one sub-speech, and should be recorded in the sub-word set by sub-speech, wherein this no speech partly is meant in the file of an embodied on computer readable, and any adjacent individual character all can't constitute the part of the cognizable vocabulary of computing machine;
One first removing module is an occurrence number of calculating each these sub-speech respectively, and occurrence number is deleted in this sub-word set less than the sub-speech of a predetermined value; And
One second removing module, be to choose one first sub-speech inequality and one second sub-speech in this sub-word set in regular turn, when this first sub-speech is contained in this second sub-speech, and the occurrence number of this first sub-speech is when being not more than this second sub-speech, this first sub-speech is deleted from this sub-word set
Use by producing the cognizable neologisms of computing machine in this sub-word set.
9. computer new word learning system as claimed in claim 8 also comprises:
One speech recognition module is a file to be carried out the speech identification handle; And
One no speech partial record module is when having at least one no speech part in this document, should no speech partial record in this no word set.
10. computer new word learning system as claimed in claim 8 also comprises:
One the 3rd removing module is with the sub-speech deletion beyond the sub-speech that occurrence number is maximum in this sub-word set.
11. computer new word learning system as claimed in claim 8 also comprises:
Module partly cut apart in one no speech, wherein
The no speech part that will comprise these neologisms removes from this no word set;
In the no speech part that comprises these neologisms, be positioned at individual character quantity before these neologisms and be two when above, be considered as another no speech part with being positioned at these neologisms part before in this no speech part, and it is added in this no word set; And
In the no speech part that comprises these neologisms, be positioned at individual character quantity after these neologisms and be two when above, be considered as another no speech part with being positioned at these neologisms part afterwards in this no speech part, and it is added in this no word set.
12. computer new word learning system as claimed in claim 8, wherein
This predetermined value is 2.
13. a computer new word learning system comprises:
One CPU (central processing unit); And
One memory storage, it is at least one procedure code of storage, makes this CPU (central processing unit) after reading this procedure code, can carry out following program:
One sub-speech logging program, be that at least one no speech that will be recorded in a no word set partly resolves at least one sub-speech, and should be recorded in the sub-word set by sub-speech, wherein this no speech partly is meant in the file of an embodied on computer readable, and any adjacent individual character all can't constitute the part of the cognizable vocabulary of computing machine;
One first delete program is an occurrence number of calculating each these sub-speech respectively, and occurrence number is deleted in this sub-word set less than the sub-speech of a predetermined value; And
One second delete program, be to choose one first sub-speech inequality and one second sub-speech in this sub-word set in regular turn, when this first sub-speech is contained in this second sub-speech, and the occurrence number of this first sub-speech is when being less than this second sub-speech, this first sub-speech is deleted from this sub-word set
Use by producing the cognizable neologisms of computing machine in this sub-word set.
14. computer new word learning system as claimed in claim 13, wherein this CPU (central processing unit) is also carried out after reading this procedure code:
One speech identification program is a file to be carried out the speech identification handle; And
One no speech partial record module is when having at least one no speech part in this document, should no speech partial record in this no word set.
15. computer new word learning system as claimed in claim 13, wherein this CPU (central processing unit) is also carried out after reading this procedure code:
One determining program is to judge whether this sub-word set is empty set, and when this sub-word set is empty set, finishes the flow process of this computer new word learning method.
16. computer new word learning system as claimed in claim 13, wherein this CPU (central processing unit) is also carried out after reading this procedure code:
One the 3rd delete program is with the sub-speech deletion beyond the sub-speech that occurrence number is maximum in this sub-word set.
17. computer new word learning system as claimed in claim 13, wherein this CPU (central processing unit) is also carried out after reading this procedure code:
One no speech part segmentation procedure comprises
The no speech part that will comprise these neologisms removes from this no word set;
In the no speech part that comprises these neologisms, be positioned at individual character quantity before these neologisms and be two when above, be considered as another no speech part with being positioned at these neologisms part before in this no speech part, and it is added in this no word set; And
In the no speech part that comprises these neologisms, be positioned at individual character quantity after these neologisms and be two when above, be considered as another no speech part with being positioned at these neologisms part afterwards in this no speech part, and it is added in this no word set.
18. computer new word learning system as claimed in claim 13, wherein this CPU (central processing unit) is also carried out after reading this procedure code:
One sub-word set empties program, is to empty this sub-word set and get back to this sub-speech logging program.
19. computer new word learning system as claimed in claim 13, wherein
This predetermined value is 2.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 00132955 CN1128404C (en) | 2000-11-16 | 2000-11-16 | computer new word learning method and system |
| HK02109097.1A HK1047636B (en) | 2000-11-16 | 2002-12-16 | Method and system for learning new terms in a computer |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 00132955 CN1128404C (en) | 2000-11-16 | 2000-11-16 | computer new word learning method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1354422A true CN1354422A (en) | 2002-06-19 |
| CN1128404C CN1128404C (en) | 2003-11-19 |
Family
ID=4595465
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 00132955 Expired - Fee Related CN1128404C (en) | 2000-11-16 | 2000-11-16 | computer new word learning method and system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN1128404C (en) |
| HK (1) | HK1047636B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100405371C (en) * | 2006-07-25 | 2008-07-23 | 北京搜狗科技发展有限公司 | Method and system for abstracting new word |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100397392C (en) * | 2003-12-17 | 2008-06-25 | 北京大学 | Method and apparatus for learning Chinese new words |
-
2000
- 2000-11-16 CN CN 00132955 patent/CN1128404C/en not_active Expired - Fee Related
-
2002
- 2002-12-16 HK HK02109097.1A patent/HK1047636B/en not_active IP Right Cessation
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100405371C (en) * | 2006-07-25 | 2008-07-23 | 北京搜狗科技发展有限公司 | Method and system for abstracting new word |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1128404C (en) | 2003-11-19 |
| HK1047636B (en) | 2004-05-21 |
| HK1047636A1 (en) | 2003-02-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP0282721B1 (en) | Paradigm-based morphological text analysis for natural languages | |
| US5680628A (en) | Method and apparatus for automated search and retrieval process | |
| US5369577A (en) | Text searching system | |
| US5890103A (en) | Method and apparatus for improved tokenization of natural language text | |
| US5708829A (en) | Text indexing system | |
| US5384703A (en) | Method and apparatus for summarizing documents according to theme | |
| EP0423683B1 (en) | Apparatus for automatically generating index | |
| US8447588B2 (en) | Region-matching transducers for natural language processing | |
| JP2742115B2 (en) | Similar document search device | |
| EP0361464A2 (en) | Method and apparatus for producing an abstract of a document | |
| EP1214643B1 (en) | System and method for parsing a document | |
| US20130185060A1 (en) | Phrase based document clustering with automatic phrase extraction | |
| WO1997004405A9 (en) | Method and apparatus for automated search and retrieval processing | |
| JP2001034623A (en) | Information retrievel method and information reteraval device | |
| US8327265B1 (en) | System and method for parsing a document | |
| EP0524694B1 (en) | A method of inflecting words and a data processing unit for performing such method | |
| CN1667614A (en) | Compression of logs of language data | |
| CN1128404C (en) | computer new word learning method and system | |
| CN1280757C (en) | Method for automatically-searching key word from file and its system | |
| JP2001184352A (en) | Automatic summary preparing device and recording medium | |
| JP4206266B2 (en) | Full-text search device, processing method, processing program, and recording medium | |
| Kaszkiel et al. | TREC 7 Ad Hoc, Speech, and Interactive tracks at MDS/CSIRO | |
| Sedlácek et al. | Automatic Processing of Czech Inflectional and Derivative Morphology | |
| JP2812511B2 (en) | Keyword extraction device | |
| JP2005189955A (en) | Document processing method, document processor, control program, and recording medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C06 | Publication | ||
| PB01 | Publication | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20031119 Termination date: 20101116 |