JP2016164724A - Vocabulary knowledge acquisition device, vocabulary knowledge acquisition method, and vocabulary knowledge acquisition program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To present read candidates of a notation without being restricted within preset information as knowledge about notations to be added to a dictionary.SOLUTION: According to an embodiment, a vocabulary knowledge acquisition device has morphological analysis means, compound word extraction means, unknown word extraction means, unknown word related information imparting means, abbreviation estimation means, formal notation candidate imparting means, and result output means. The unknown word extraction means extracts unknown words unregistered with an established dictionary. The unknown word related information imparting means extracts candidates of read of unknown words from data acquired from outside and imparts them to the unknown words as unknown word related information. The formal notation candidate imparting means imparts compound words generating abbreviations to the unknown words as formal notation candidates when the abbreviations generated from the compound words match the unknown words. The result output means outputs the unknown word, the unknown word related information, and the formal notation candidates together in order of dictionary additional registration effect.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a vocabulary knowledge acquisition device, a vocabulary knowledge acquisition method, and a vocabulary knowledge acquisition program that support user work when a new vocabulary is added to a dictionary used for applications such as speech recognition.

  In recent years, a speech recognition system that recognizes a sound generated by a human and records it in a document or displays it on a screen has been used. In order to improve recognition accuracy in a speech recognition system, it is effective to add vocabulary readings, notations, parts of speech, etc. that cannot be recognized correctly to the dictionary. Here, correct speech recognition means assigning correct reading and notation to the input speech signal.

  Vocabulary that cannot be recognized correctly is not only determined by looking at the results of speech recognition, but also when creating a new speech recognition system, it is a vocabulary that is not in the built-in dictionary for speech recognition. In some cases, frequently used items are predicted and added to the speech recognition dictionary in advance.

  In this case, the speech recognition system selects a vocabulary that is not in the built dictionary, that is, a notation of an unknown word, from a document or term list related to the scene where speech recognition is used, and gives part-of-speech and reading information to the dictionary. sign up.

  Conventionally, in order to give correct reading information to an unknown word notation, there is a method of extracting readings for unknown words as follows. For example, there is a technique for estimating a reading by referring to a dictionary and rules for reading determination prepared in advance. For example, for unknown words, the unknown word reading dictionary is referred to, and the readings “E”, “B”, “S” corresponding to the registered “A”, “B”, “C” are registered in the notation “ABC”. "Is added as a reading. In addition, with reference to the two tables recorded in the reading probability storage unit, a reading candidate having the maximum conditional probability is generated, and the reading candidate is superior or inferior from the number of hits obtained as a result of searching the Web with a set of notation and reading. And select a reading candidate with a high priority.

  Furthermore, there is a technique for searching a combination of an abbreviation, which is a kind of synonym, and an original word (official name) from a Web search result. In this technique, when an abbreviation is input, a syntax that differs only in the abbreviation portion is extracted, and an expression used at the same position as the abbreviation is estimated as an original word (official name).

Japanese Patent No. 4941495 JP 2009-204732 A Japanese Patent No. 5355537

  As described above, in the conventional technique, reading is estimated based on information provided in the system (information registered in a dictionary or the like in advance), and therefore, special reading that cannot be estimated by the system or set in the system. Cannot add readings for no notation.

  Further, the technique for searching for combinations of abbreviations and original words (official names) from Web search results estimates and narrows down abbreviations from original word candidates only when a plurality of original word candidates are extracted. Therefore, when there are few expressions of the same syntax, there is a possibility that synonyms that are not synonymous with abbreviations are extracted as the only original word candidates.

  The problem to be solved by the present invention is to provide a vocabulary knowledge acquisition device and vocabulary knowledge that can present reading candidates for a notation within a preset information as knowledge about the notation to be added to a dictionary. An acquisition method and a vocabulary knowledge acquisition program are provided.

  According to the embodiment, the vocabulary knowledge acquisition device includes a morphological analysis unit, a compound word extraction unit, an unknown word extraction unit, an unknown word related information addition unit, an abbreviation estimation unit, a formal notation candidate addition unit, and a result output unit. . The morpheme analysis means divides the text included in the plaintext corpus into words and performs morpheme analysis that gives parts of speech to each word. The compound word extracting unit extracts a compound word based on the result of the morphological analysis. The unknown word extraction means compares the word obtained by the morphological analysis and the compound word obtained by the compound word extraction with a registered word of the constructed dictionary, and extracts an unknown word that is not registered in the constructed dictionary. . The unknown word related information adding means extracts a reading candidate for the unknown word from data acquired from the outside, and adds it to the unknown word as unknown word related information. The abbreviation estimation means generates an abbreviation from the compound word. The formal notation candidate assigning means assigns a compound word that is the generation source of the abbreviation to the unknown word as a formal notation candidate when the abbreviation generated by the abbreviation estimation means matches the unknown word. The result output means puts together the unknown word, the unknown word related information, and the formal notation candidates, and outputs them as a vocabulary list arranged in descending order of the dictionary additional registration effect.

The block diagram which shows the structure of the system using the vocabulary knowledge acquisition apparatus in this embodiment. The block diagram which shows the function implement | achieved based on the vocabulary knowledge acquisition program in this embodiment. The flowchart which shows about operation | movement of the vocabulary knowledge acquisition process of the vocabulary knowledge acquisition apparatus in this embodiment. The figure which shows an example of the data registered into the constructed dictionary in this embodiment. The figure which shows an example of the morphological analysis result in this embodiment. The figure which shows an example of the unknown word related information output by the unknown word related information provision function 44 in this embodiment. The figure which shows an example of the vocabulary list output from the result output function in this embodiment. The flowchart which shows the unknown word related information provision process in this embodiment. The figure which shows an example of the reliability evaluation list | wrist in this embodiment. The flowchart which shows the result output process in this embodiment. The flowchart which shows the compound word extraction process in this embodiment. The figure which shows an example of the list | wrist used for the compound word extraction process in this embodiment.

  Hereinafter, embodiments will be described with reference to the drawings.

  FIG. 1 is a block diagram showing the configuration of a system that uses a vocabulary knowledge acquisition apparatus 10 according to this embodiment. In the system shown in FIG. 1, the vocabulary knowledge acquisition device 10 communicates with Web servers 14-1, 14-2,..., 14 -n and various electronic devices through a network 12 such as the Internet, and transmits and receives various data. can do.

  The vocabulary knowledge acquisition apparatus 10 in the present embodiment is realized by a computer such as a personal computer. As shown in FIG. 1, the vocabulary knowledge acquisition device 10 includes a processor 20, a memory 21, a storage device 24, an input unit 25, a display unit 26, a voice input unit 27, a voice output unit 28, and a communication unit 29.

  The processor 20 implements various functions by executing various programs (software) read from the storage device 24 to the memory 21. For example, the processor 20 executes various programs (software) such as an OS (Operating System) and application programs stored in the memory 21 to realize various functions. For example, the processor 20 executes a vocabulary knowledge acquisition program 21a and realizes a function for supporting user work when adding a new vocabulary to a speech recognition dictionary (built dictionary 24e) used in the speech recognition system. To do. The functions realized based on the vocabulary knowledge acquisition program 21a are shown in FIG. Further, the processor 20 implements a voice recognition system by executing the voice recognition program 21b.

  The memory 21 stores programs and data executed by the processor 20.

  The storage device 24 stores various programs (software) such as an OS (Operating System) and application programs, data necessary for executing the programs, and the like in a nonvolatile storage medium. The data stored in the storage device 24 includes, for example, a plain text corpus 24a, a formal name list 24b, a Japanese-English machine translation dictionary 24c, a Web crawling data 24d, a built dictionary 24e, a temporary construction dictionary 24f, a vocabulary list 24g, and an audio final 24h. including. Details of each data will be described later.

  The input unit 25 controls input from an input device (for example, a keyboard, a mouse, a tablet, etc.) operated by the user under the control of the processor 20.

  The display unit 26 controls display on a display such as an LCD (Liquid Crystal Display) under the control of the processor 20.

  The voice input unit 27 controls voice input from the microphone under the control of the processor 20.

  The sound output unit 28 controls sound output from a speaker, headphones, or the like under the control of the processor 20.

  The communication unit 29 controls communication with the Web server 14 and the electronic device via the network 12.

  Note that the vocabulary knowledge acquisition device 10 can be implemented with either a hardware configuration or a combined configuration of hardware resources and software (programs). The software is installed in the computer from the network 12 or a non-transitory computer-readable storage medium in advance, and is executed by the processor 20 of the computer, thereby causing the computer to execute the function of each device.

  FIG. 2 is a block diagram showing a functional configuration of the vocabulary knowledge acquisition apparatus 10 in the present embodiment. The processor 20 implements each function included in the functional unit 30 by executing the vocabulary knowledge acquisition program 21a. Each function included in the function unit 30 executes processing for each data included in the storage unit 32.

  Based on the vocabulary knowledge acquisition program 21a, the vocabulary knowledge acquisition device 10 has a morphological analysis function 41, a compound word extraction function 42, an unknown word extraction function 43, an unknown word related information addition function 44, an abbreviation estimation function 45, and a formal notation candidate assignment. Processing by the function 46, the result output function 47, and the dictionary editing function 48 is executed.

  The voice recognition system 49 is a function realized by the processor 20 executing the voice recognition program 21b. The speech recognition system 49 is a system independent of the function of the vocabulary knowledge acquisition device 10 and is used in the processing by the dictionary editing function 48. However, the speech recognition system 49 may be a part of the function realized by the vocabulary knowledge acquisition program 21a.

  The storage unit 32 (storage device 24) includes a plaintext corpus 24a, a formal name list 24b, a Japanese-English machine translation dictionary 24c, Web crawling data 24d, and a built-up dictionary, which are resources necessary for processing of each function of the function unit 30. 24e, temporary construction dictionary 24f, and audio file 24h. In addition, the storage unit 32 stores a vocabulary list 24g as a processing result of each function.

  The built dictionary 24e is a dictionary used for voice recognition processing by the voice recognition system 49, for example. In the built dictionary 24e, for example, as shown in FIG. 4, a set of data indicating notation (headword), part of speech, and reading is registered for each of a plurality of headwords. A new vocabulary (part of speech, notation, reading) can be added to the constructed dictionary 24e by a user operation using the processing result of the vocabulary knowledge acquisition device 10.

  The plaintext corpus 24a is a set of documents (for example, text data) used to add a new vocabulary to the built dictionary 24e. For example, an unknown word that is not registered in the built dictionary 24e is extracted from the plain text corpus 24a, and this unknown word becomes a vocabulary candidate to be added to the built dictionary 24e. The plaintext corpus 24a uses documents related to the corresponding field in order to improve the quality of the speech recognition in the field where the speech recognition system 49 is used. For example, in the medical / pharmaceutical field, a package insert of a drug is applicable.

  The formal name list 24b is a term list in which notations (terms and the like) related to the scene where the speech recognition system 49 is used are registered. For example, in the medical / pharmaceutical field, a formal name list (medical term dictionary) such as a disease name, a drug list, and the like are applicable. When performing speech recognition processing for a person name, a person name list (not only general person names but also stage names etc. may be used) is used. Similarly, a place name list is used for the place name, and a trademark list is used for the product name.

  The Japanese-English machine translation dictionary 24c is a list in which Japanese notations and English translations for the notations are registered. For example, “lymph node”, which is a parallel translation of English, is registered for the Japanese expression “lymph node” (reading: Rinpetsu, part of speech: noun-general).

  The web crawling data 24d is data acquired from the outside through the network 12 (Internet) by web crawling. The web crawling data 24d is obtained by saving a web page published on a website (web server 14) as a static file. The web crawling data 24d is used to acquire reading information for an unknown word (notation) extracted from the plaintext corpus 24a. The file format of the web crawling data 24d may be the HTML (Hyper Text Markup Language) format that is the source of the Internet public page, or the HTML format converted to the general document format that is the same as the public page. Good. The web crawling data 24d may be collected from the web server 14 through the network 12 by the function of the vocabulary knowledge acquisition program 21a of the vocabulary knowledge acquisition device 10, or created by an electronic device different from the vocabulary knowledge acquisition device 10. May be entered. The web crawling data 24d is not data that is fixedly recorded in the vocabulary knowledge acquisition device 10, but data that is continuously updated. Therefore, by updating the Web page that is published through the Internet, it is possible to acquire new reading information for the notation from the Web crawling data 24d.

  The temporary construction dictionary 24f is a dictionary used for voice recognition processing by the voice recognition system 49 to which the built dictionary 24e is copied. The temporary construction dictionary 24f is used to add a notation candidate to be added to the constructed dictionary 24e and execute the speech recognition processing by the speech recognition system 49. The vocabulary knowledge acquisition device 10 extracts the difference between the result of the speech recognition process using the constructed dictionary 24e and the result of the speech recognition process (analysis result) using the temporary construction dictionary 24f, and adds it to the constructed dictionary 24e. This is extracted as knowledge about notation.

  The vocabulary list 24g is data presented to the user in order to support the user work when adding a new notation to the built dictionary 24e. The vocabulary list 24g presents data (knowledge) that serves as a reference when the user determines whether or not to add a notation to the built dictionary 24e for a notation (unknown word) candidate to be added to the built dictionary 24e. . Details will be described later (see FIG. 7).

  The voice file 24h is voice data as input voice to the voice recognition system 49 for causing the voice recognition system 49 to execute voice recognition processing using the built dictionary 24e and the temporary construction dictionary 24f. The voice file 24h is, for example, voice data associated with the text data of the plaintext corpus 24a on a one-to-one basis, that is, voice data of voice obtained by reading out the text of the plaintext corpus 24a. The voice file 24h may be a voice data file having a different content from the text of the plaintext corpus 24a prepared for the test by the user.

Next, the operation of the vocabulary knowledge acquisition process of the vocabulary knowledge acquisition apparatus 10 in this embodiment will be described with reference to the flowchart shown in FIG.
First, the morpheme analysis function 41 executes a morpheme analysis process for the plaintext corpus 24a (step A1). The morphological analysis function 41 divides the Japanese text data included in the plaintext corpus 24a into words by morphological analysis processing, and gives parts of speech for each word.

  For example, the morphological analysis function 41 executes the morphological analysis processing on the text data of the Japanese text in the plaintext corpus 24a “Prescription Kakkonto was prescribed because there was a complaint of the initial symptoms of a cold. No swelling of LN”. As a result, a morphological analysis result as shown in FIG. 5 is obtained.

  Next, the compound word extraction function 42 inputs the output of the morpheme analysis function 41 (morpheme analysis result), and executes a compound word extraction process for extracting a compound word based on the morpheme analysis result (step A2).

  The compound word extraction function 42 extracts a character string that can be estimated to constitute a compound word by connecting adjacent morphemes, and outputs it as a compound word. As the determination of the character string constituting the compound word, for example, a rule such as “guesses a continuous part of“ noun-general ”as a compound word (compound noun)” is used.

  In the morphological analysis result shown in FIG. 5, since both “initial” and “symptom” are part-of-speech “noun-general” and appear continuously, it is possible to infer “initial symptom” as a compound word (compound noun). Further, it is also possible to use a technique for inferring not only a single morpheme analysis result but also a high-frequency morpheme connection that appears adjacently as a compound word based on a large amount of morpheme analysis results. Here, without limiting to “noun-general”, if a continuous part of “noun” or a continuous part of the alphabet is assumed to be a compound word (compound noun), the result of morphological analysis shown in FIG. "Yu" and "LN" are extracted as compound words (compound nouns).

  Next, the unknown word extraction function 43 extracts an unknown word (vocabulary) that is not registered in the constructed dictionary 24e from the morphological analysis result of the morphological analysis function 41 and the compound word extracted by the compound word extraction function 42. The unknown word extraction process is executed (step A3).

  The unknown word extraction function 43 extracts a basic form to which a part of speech corresponding to an independent word is assigned based on the morphological analysis result output from the morphological analysis function 41. An independent word refers to a word that can constitute a phrase alone. The part of speech corresponding to an independent word is a noun, pronoun, verb, adjective, adjective verb, adverb, conjunction, conjunction, or impression verb.

  The basic forms (notation) extracted from the morphological analysis results shown in FIG. 5 are “cold (noun-general)”, “initial (noun-general)”, “symptom (noun-general)”, “appeal (noun-general)”, “ "There is (verb-independence)" "For (noun-non-independence-adverb possible)" "Kakone (noun-proper noun-region-general)" "Yu (noun-general)" "Prescription (noun-sa connection)" 11 (verb-independent) "swelling (noun-general)".

  Further, the unknown word extraction function 43 adds the output (composite word) of the compound word extraction function 42 to 11 words extracted from the morpheme analysis result of the morpheme analysis function 41. Here, the added notation (compound word) is 3 notations of “initial symptom (noun)”, “Kakkonto (noun)”, and “LN (noun)”, and the extracted notation is 14 words (14 notation). . Here, the extracted 14-word notation is a candidate for an unknown word extracted from the plaintext corpus 24a.

  Next, the unknown word extraction function 43 compares the list of unknown word candidates (notation) with the built dictionary 24e, and extracts unknown words that are not registered in the built dictionary 24e. That is, the unknown word extraction function 43 extracts and outputs a combination of notation and part of speech included in the unknown word candidate list that is not registered in the built dictionary 24e.

  Since the “cold (noun-general)”, “initial (noun-general)”, “symptom (noun-general)”, and “prescription (noun-variant connection)” are registered in the built dictionary 24e, unknown words are extracted. The functions 43 are “sue (noun-general)” “ar (verb-independent)” “for (noun-non-independent-adverb possible)” “katsune (noun-proprietary noun-region-general)” “yu (noun- Ten notations such as “general)” “do (verb—independence)”, “swelling (noun—general)”, “early symptom (noun)”, “Kakkonto (noun)”, “LN (noun)” are extracted as unknown words.

  Note that the unknown word extraction function 43 may delete one of the notations extracted as the unknown word if a duplicate notation is included. For example, in the example described above, “Kakkonto (noun)” is extracted as a compound word by the compound word extraction function 42. On the other hand, from the output of the morphological analysis function 41, “kazune (noun-proper noun-region-general)” and “yu (noun-general)”, which are components of “Kakkonyu”, are extracted. In this case, the unknown word extraction function 43 deletes “Kak root (noun-proper noun-region-general)” and “yu (noun-general)” extracted from the morphological analysis result.

  As a result, the unknown word extraction function 43 is “sue (noun-general)” “is (verb-independent)” “for (noun-non-independent-adverb possible)” “do (verb-independent)” “swelling (noun) -General 8) "Initial symptoms (nouns)", "Kakkonto (nouns)" and "LN (nouns)" are output.

  Further, the unknown word extraction function 43 restricts notation (vocabulary) candidates to be registered in the built dictionary 24e based on the part of speech. For example, the unknown word extraction function 43 sets only candidates for part-of-speech notation as non-independent, for example, as registration candidates.

  As a result, the unknown word extraction function 43 uses five notations of “sue (noun-general)”, “early symptom (noun)”, “swelling (noun-general)”, “Kakkonto (noun)”, and “LN (noun)”. Output. In the subsequent processing, the output of the unknown word extraction function 43 is “sue (noun-general)”, “initial symptom (noun)”, “swelling (noun-general)”, “Kakkonto (noun)”, “LN (noun)”. It will be described as 5 notation.

  Next, the unknown word related information adding function 44 executes unknown word related information adding processing, and each of the notations (notation (unknown word) candidates registered in the built dictionary 24e) output from the unknown word extracting function 43. Is obtained and given as reference data (unknown word related information) when the user determines whether or not to add to the constructed dictionary 24e (step A4).

  Here, the unknown word related information addition function 44 obtains and assigns unknown word related information for each of the five notations output by the unknown word extraction function 43.

  The unknown word related information includes, for example, an estimated part of speech (“estimated part of speech”), an appearance frequency when processing the plaintext corpus 24a (text data) (“appearance frequency”), and an unknown word extracted from the web crawling data 24d. Readings (“reading”), snippets / information sources (“snippets / information sources”) from which readings for unknown words are extracted, and notations (similar registrations) registered in the built-in dictionary 24e where the unknown words and readings / notations / parts of speech are similar Word) and the frequency of use thereof, and at least one of information such as a difference between speech recognition processing results (analysis results) before and after adding or deleting a notation (entry word) to the dictionary.

FIG. 6 is a diagram illustrating an example of unknown word related information output by the unknown word related information adding function 44 in the present embodiment.
Here, the plain text corpus 24a contains the Japanese text “Kekkonto was prescribed because there was a complaint of an early symptom of a cold. There is no swelling of LN.” An example in which the notation such as “initial symptom” and “Kakkonto” appears multiple times respectively.

  The “estimated part of speech” is a part of speech obtained by morphological analysis attached to the notation output by the unknown word extraction function 43.

  The “appearance frequency” is a number obtained by counting the number of appearances in the plaintext corpus 24 a written by the unknown word extraction function 43.

  The “reading” is a reading obtained by morphological analysis attached to the notation output by the unknown word extraction function 43 or a notation (unknown word) extracted from the Web crawling data 24 d. The unknown word related information adding function 44 extracts a character string corresponding to reading from the Web crawling data 24d based on the notation (unknown word) output by the unknown word extraction function 43.

  For example, the unknown word related information addition function 44 extracts a portion in which a combination of an unknown word and a reading from the web crawling data 24d is described. For example, when there is a description of “Hiragana” or “Katakana” surrounded by “()” immediately after the unknown word, it is extracted as a part in which the combination of the unknown word and the reading is described.

  Alternatively, the unknown word related information adding function 44 includes an unknown word described in one column and a description of “Hiragana” or “Katakana” in the other column as an unknown word in the tabular description part of the Web crawling data 24d. If it is associated, it is extracted as a combination of an unknown word and its reading.

  The “snippet / information source” is, for example, a snippet (may be a part) including an unknown word reading in the Web crawling data 24d and a URL (uniform resource locator) of a Web site (Web page) including the unknown word. is there. When a plurality of “snippet / information source” pairs including unknown word readings are extracted, the unknown word related information adding function 44 may extract all of the plurality of sets, or the number of times the same reading is given Only the most snippet may be used. Further, the unknown word related information adding function 44 may aggregate information as unknown word related information by preferentially adopting a Web site with high reliability given in advance by the user.

  The registered notation and the frequency of use thereof are the registered notation and the registered notation in which the unknown word (notation) extracted from the constructed dictionary 24e is similar in reading / notation / part of speech (at least the readings match). The number of appearances in the plaintext corpus 24a is counted.

  In the unknown word related information shown in FIG. 6, information of “clear (swelling, part of speech: noun—general, appearance frequency: 1)” is added to the unknown word “swelling”.

  “Difference” indicates the result of the speech recognition process using the temporary construction dictionary 24 f when the unknown word is registered in the temporary construction dictionary 24 f and the result of the speech recognition process using the built dictionary 24 e in which the unknown word is not registered. Information (difference in speech recognition result). The unknown word related information adding function 44 obtains “difference” information as follows.

  The unknown word related information adding function 44 causes the dictionary editing function 48 to add the combination of the notation, the part of speech, and the reading, which are set as unknown words, to the temporary construction dictionary 24 f that is a copy of the constructed dictionary 24 e. Next, the unknown word related information addition function 44 instructs the dictionary editing function 48 to execute a speech recognition process using the temporary construction dictionary 24f and the built dictionary 24e. The dictionary editing function 48 causes the voice recognition system 49 to execute voice recognition processing using the temporary construction dictionary 24f in which the unknown words are registered and the built dictionary 24e in response to an instruction from the unknown word related information adding function 44. . At this time, the dictionary editing function 48 inputs the voice file 24h as voice data to be subjected to voice recognition processing to the voice recognition system 49.

  The dictionary editing function 48 outputs the result of the speech recognition process using the temporary construction dictionary 24 f and the result of the speech recognition process using the built dictionary 24 e to the unknown word related information adding function 44. The unknown word related information adding function 44 creates information on the difference (difference in the voice recognition result) based on the voice recognition result using the temporary construction dictionary 24f and the built dictionary 24e.

  The unknown word related information providing function 44 deletes the registered word from the temporary construction dictionary 24f when the registered word having similar reading / notation / part of speech in the built dictionary 24e exists for the unknown word. It can also be added to unknown word related information as a new analysis result.

  In this way, by extracting information on the difference between the speech recognition processing results before and after adding or deleting unknown words to the dictionary, it is possible to check the effectiveness when the user registers the unknown words in the dictionary and perform dictionary editing Therefore, the efficiency of dictionary editing can be improved, and the adverse effects of dictionary editing can be confirmed and prevented in advance.

  In the unknown word related information shown in FIG. 6, “difference” information is added for the unknown words “swelling” and “Kakkonto” (shown in A and B in the figure).

  In the unknown word related information shown in FIG. 6, information that could not be extracted is blank. For example, if the snippet / information source is not extracted from the web crawling data 24d, or if a registered word with similar reading / notation / part of speech is not extracted from the built-in dictionary 24e, the field is left blank. Further, the “difference” information may be given only when the reading is different from the reading given by the morphological analysis function 41.

  Next, the abbreviation estimation function 45 assigns a formal notation to an unknown word representing an abbreviation included in the unknown word related information, and therefore abbreviation estimation for estimating an abbreviation that may be included in the unknown word related information. Processing is executed (step A5).

  The abbreviation estimation function 45 is based on notations registered in the formal name list 24b, notations obtained as a result of morpheme analysis by the morpheme analysis function 41, and notations extracted as part of the formal name by the compound word extraction function 42. Create an abbreviation for Here, the abbreviation estimation function 45 generates an abbreviation for a notation composed of a plurality of English words or a notation composed of a plurality of Japanese morphemes.

  For example, when the Japanese name “lymph node” is registered in the official name list 24b and the English translation “lymph node” is registered in the Japanese-English machine translation dictionary 24c, the abbreviation estimation function 45 is: As an abbreviation for “lymph node”, “LN” is generated by concatenating the English initials with capital letters.

  The abbreviation estimation function 45, for example, concatenates the first letter of the morpheme of the morphological analysis result “artery (noun-general) injection (noun-sa-variant connection)” to the Japanese official name “arterial injection”. Generates the abbreviation “articulation”.

  Next, the formal notation candidate assigning function 46 executes a formal candidate assigning process for assigning formal notation candidates and readings to unknown words corresponding to the abbreviations generated by the abbreviation estimation function 45 included in the unknown word related information. (Step A6).

  First, the formal notation candidate giving function 46 compares the notation (unknown word) in the unknown word related information output by the unknown word related information giving function 44 with the abbreviation generated by the abbreviation estimation function 45.

  When there is an abbreviation that matches the notation (unknown word) in the unknown word related information, the formal notation candidate giving function 46 is the source of the abbreviation for the corresponding notation (unknown word) in the unknown word related information. Give the official name and its reading and part of speech.

  For example, in the unknown word related information shown in FIG. 6, the unknown word “LN” matches the abbreviation “LN” estimated from the formal expression “lymph node” generated by the abbreviation estimation function 45. In this case, the formal notation candidate assigning function 46 assigns the formal notation candidate “lymph node”, the reading “Rinpasetsu”, and the part of speech “noun-general” to the unknown word “LN”. The reading and part of speech of the formal notation candidate are handled as the reading of the abbreviation “LN” and the part of speech candidate.

  Next, the result output function 47 executes a result output process in which the unknown word related information output from the formal notation candidate assignment function 46 is edited and output in a format presented to the user (step A7). The result output function 47 generates a vocabulary list 24g by arranging a plurality of unknown words included in the unknown word related information in descending order of the dictionary additional registration effect, and displays the vocabulary list 24g on the display unit 26.

  The result output function 47 may not only display the vocabulary list 24g as a list but also display the unknown word related information in order for each unknown word (notation).

  FIG. 7 is a diagram showing an example of the vocabulary list 24g output from the result output function 47 in the present embodiment. The vocabulary list 24g shown in FIG. 7 shows an example in which the unknown word arrangement is changed in descending order of appearance frequency with respect to the unknown word relation shown in FIG.

  It should be noted that the vocabulary list 24g shown in FIG. 7 is given a formal notation candidate “lymph node (reading: Rinpatsutsu, part of speech: noun-general)” with respect to the notation “LN” by the formal notation candidate giving function 46 (FIG. 7). And the reading “Rinpetsutsu” (shown in C in the figure) is added.

Further, in the above description, the unknown words (notation) of the unknown word related information are rearranged in the order of appearance frequency, but can be edited based on other conditions.
For example, a notation extracted as a compound word or a notation in which the reading information extracted from the web crawling data 24d is different from the analysis result of the morphological analysis function 41 is judged to have a high dictionary addition registration effect, and is higher in the lexical list 24g. It may be positioned. In addition, when there are a plurality of notations having the same determination result of the dictionary additional registration effect, such as a compound word having a high appearance frequency, reading information extracted from the Web crawling data 24d being different from the analysis result of the morphological analysis function 41, The notation may be rearranged on the basis of another criterion for determining additional dictionary registration effects.

  The vocabulary list 24g shown in FIG. 7 shows an output example of a tabular format consisting of a plurality of lines for each notation, but other formats are also possible. For example, for a single notation, the unknown word related information can be in a tabular form that is shown in one line. For information with a long text such as information on “snippet / information source”, only link information to the corresponding information may be presented in the vocabulary list 24g.

  In this way, the vocabulary knowledge acquisition apparatus 10 according to the present embodiment is limited to information preset in the vocabulary knowledge acquisition apparatus 10 by acquiring readings corresponding to unknown words from the web crawling data 24d. In addition, readings of unknown words that are not registered in the built dictionary 24e can be acquired even when morphological analysis or estimation rules cannot handle them. In addition, by presenting the correspondence between an abbreviation and an official name for an unknown word, an appropriate reading can be given to an abbreviation that has a high possibility of being applied as it is. In addition, since the correspondence between the abbreviated name and the official name is presented, it is possible to easily cope with the voice conversation in which the meaning of the recognized word needs to be grasped. The extraction of notations as candidates to be registered in the constructed dictionary 24e from the plaintext corpus 24a and the reading estimation are performed mechanically, thereby reducing the work time for manual implementation and registration in the constructed dictionary 24e. This can reduce the omission of unknown words that have not been extracted. The user determines whether or not to register the notation (unknown word) as a candidate for registration in the constructed dictionary 24e presented by the vocabulary list 24g based on the unknown word related information given to each. be able to.

  The “difference” information corresponding to the unknown word extracted by the unknown word related information adding function 44 is the notation (unknown word) selected by the user operation from the vocabulary list 24g after the vocabulary list 24g is presented to the user. You may make it perform only with respect to. The “difference” information extraction method is executed in the same manner as described above, and a detailed description thereof will be omitted.

  By generating “difference” information only for the notation (unknown word) selected by the user, the processing burden for presenting the vocabulary list 24g is reduced, and the vocabulary list 24g can be sent to the user in a short time. Can be presented.

  Next, an application example of the unknown word related information adding process by the unknown word related information adding function 44 in the present embodiment will be described. FIG. 8 is a flowchart showing the unknown word related information adding process in the present embodiment.

  Here, the unknown word related information addition function 44 creates unknown word related information using a reliability evaluation list indicating the reliability of information published by the Web site (Web server 14).

  There are a mixture of Web sites that disclose reliable information edited by experts and those that disclose low-reliability information edited by non-experts.

  FIG. 9 is a diagram showing an example of the reliability evaluation list in the present embodiment. In the example illustrated in FIG. 5, the reliability evaluation list can record evaluation values indicating reliability, for example, three-level evaluation values “◯”, “Δ”, and “×” for each Web site (URL). In addition, the reliability evaluation list indicates, for each website (URL), when the information extracted from the website (reading for unknown words) is presented to the user, whether or not the user has adopted the information is “reading adopted. It can be recorded as “number” and “number of reading failures”.

  When a plurality of “snippet / information source” pairs are extracted from the Web crawling data 24d (step B1, Yes), the unknown word related information adding function 44 refers to the reliability evaluation list shown in FIG. 9 and presents it to the user. The information of “snippet / information source” is selected (step B2).

  For example, the unknown word related information adding function 44 preferentially selects information of a site having a reliability of “◯”, a large number of reading adoptions, and a small number of reading unacceptances, and gives the information as unknown word related information.

  Further, the unknown word related information adding function 44 is reliable when the information added to the unknown word related information is adopted (step B3, Yes), that is, when the reading presented in the vocabulary list 24g is registered together with the unknown word. The “reading adoption number” of the Web site from which the corresponding information in the evaluation list is extracted is counted up (step B4). Note that the “reading rejection number” in the reliability evaluation list is not selected when, for example, the user explicitly specifies that the reading is not adopted or when a plurality of readings are presented for one notation. Count up when.

  The reliability evaluation value may be set in the reliability evaluation list by the user operation after the user confirms the content of the website, or according to the “reading adoption number” and “reading rejection number”. It may be automatically set according to a preset rule. For example, when the “reading adoption number” is equal to or higher than the reference value and the “reading rejection number” is “0”, the reliability is set to “○”, or “reading adoption number” and “reading rejection number” Or may be set based on the ratio.

  When using the reliability evaluation list, it is possible to use only the website with the reliability “O”, and the reliability is obtained when necessary information cannot be extracted from the website with the reliability “O”. You may make it utilize the web site of "(triangle | delta)". Furthermore, other utilization methods can be used. Further, the evaluation value is not limited to three stages, and can be an arbitrary number of stages.

  In this manner, highly reliable information can be presented to the user by selecting information with reference to the reliability of the website evaluated in advance. Moreover, the reliability of the information presented to the user can be further improved by accumulating a history of whether or not the information presented by the user has been adopted and updating the evaluation.

  Next, an application example of the result output process by the result output function 47 in this embodiment will be described. FIG. 10 is a flowchart showing a result output process in the present embodiment.

  The result output function 47 presents the unknown word related information (unknown word list) created by the formal notation candidate assignment function 46 to the user by rearranging the unknown words in descending order of the dictionary additional registration effect.

The result output function 47 can use, for example, the following seven indices as the determination indices in descending order of the dictionary additional registration effect.
First index: High appearance frequency in the plaintext corpus 24a.
Second index: high appearance frequency in the formal name list 24b.
Third index: There are many vocabularies with the same part of speech as the notation registered in the built dictionary 24e.
Fourth index: Reading information extracted from the Web crawling data 24d is different from reading estimated from the morphological analysis result.
Fifth index: The number of different morphemes appearing immediately before and after the notation in the plaintext corpus 24a.
Sixth index: The value of the written weight evaluation value tf-idf is large.
Seventh notation: A high index (C-value, MC-value, etc.) for evaluating the independence of compound words.

  By using the first index, it is possible to preferentially present a notation with a high appearance frequency and present it as a registration candidate. By using the second index, priority is given to the correct notation included in the formal name list 24b, which is likely to appear in the target field even when the plaintext corpus 24a is not sufficient (for example, the amount of data is small). Can be presented. By using the third index, priority is given to the expression of parts of speech that are likely to be required in the constructed dictionary 24e (for example, adjectives effective for speech recognition, proper nouns effective for recognition of place names, person names, etc.). Can present. By using the fourth indicator, priority is given to new notations (new words, names of entertainers, etc.) that are likely to be difficult to read (uncommon) and that are likely to be effective to register. Can be presented. By using the fifth index, independent words can be preferentially presented. By using the sixth index, it is possible to preferentially present notations that are biased toward documents in a specific field and that are likely to be important words in that field. By using the seventh notation, it is possible to preferentially present the notation in the compound word for the notation in which the word included in the compound word is low independence (always used in the compound word).

  The notation weight evaluation value tf-idf is an index calculated by multiplying two indexes of “tf” (word appearance frequency) and “idf” (reverse document frequency). “Idf” has a lower value for words that appear in many documents, that is, general words, and has a higher value for words that appear only in a specific document. That is, “tf−idf” obtained by multiplying “idf” by “tf” is a high value for a notation that appears frequently only in a specific document. Therefore, it can be used as an index for determining an important word specific to a certain specialized field.

  In addition, an index C-value for evaluating the independence of compound words indicates the degree of coupling between words in a document.

C-value (w) = (length (w) -1) (n (w)-(t (w) / c (w)))
w: focused word length (w): length of w (number of words constituting w)
n (w): Number of occurrences of w t (w): Number of occurrences of longer compound words including w c (w): Number of different compound words including w When it is used only as a part, C-value becomes a value close to zero. A word with a large C-value is highly independent. Since C-value is always 0 when w is composed of only one word, it is possible to use a correction formula such as MC-value that gives an evaluation value other than 0 even for one word. it can.

  The result output function 47 determines the height of the dictionary additional registration effect using one or a combination of the seven indexes, and rearranges the results. Note that which index is used may be selectable by the user, or may be automatically set by the system. When the system automatically sets, it can be determined based on, for example, the contents (length, field) of the plaintext corpus 24a to be processed. In addition, when a plurality of indices are used, priority may be set for the indices.

  Further, conditions can be set for each index. For example, specification of a range of values to be presented to the user can be accepted and the output range of the result can be limited. For example, the output range can be limited by specifying “appearance frequency in plaintext corpus is 10 or more”, or can be limited to notation presumed to be a noun by specifying “estimated part of speech is a noun”. .

  The result output function 47 determines the level of the dictionary additional registration effect on the list of unknown words (notation to be registered candidates) output from the formal notation candidate giving function 46 based on a preset index. (Step C1), the order of unknown words is rearranged according to the determination result (Step C2).

  The result output function 47 outputs the vocabulary list 24g in which the notation order is rearranged based on the index (step C3).

  In this way, it is possible to select a combination flexibly by providing a plurality of evaluation indexes, and by limiting the output range, the accuracy is improved so that the content required by the user is included at the top of the output vocabulary list 24g. it can.

  Next, an application example of the compound word extraction process by the compound word extraction function 42 in this embodiment will be described. FIG. 11 is a flowchart showing compound word extraction processing in the present embodiment.

  In general, there is a technique for determining whether or not to constitute a compound word from the parts of speech of adjacent morphemes. For example, it is known that the sequence of “noun-general” is determined as a compound noun. In addition, "Prefix and noun connect", "Noun and suffix connect", "Noun connected with case particle" "is connected including case particle" "", etc. have high precision It is common to apply only rules. In such a technique, punctuation marks “,” “.” And “” (space) are often treated as delimiters and are not used as elements constituting a compound word.

  However, in recent years, in the proper nouns such as product names, names of various contents (books, movies, animations, etc.), names of people such as stage names, characters etc. (punctuation marks, Things including spaces, symbols, etc.) and parts of speech are used in many fields.

  Therefore, the compound word extraction function 42 in this embodiment extracts a compound word from the morpheme analysis result output by the morpheme analysis function 41 according to the procedure shown in FIG. All combinations of parts that appear adjacently are extracted as compound word candidates.

  That is, the compound word extraction function 42 determines whether or not the output (shown in FIG. 5) of the morphological analysis function 41 includes a character / part of speech preset as a reliable word break (step D1). When a preset character / part of speech is not included (step D2, No), the compound word extraction function 42 determines whether the result is a morpheme concatenation result and starts with a character / part of speech that does not start the compound word (step D3). . If it does not start with the corresponding character / part of speech (step D4, No), the compound word extraction function 42 determines whether it ends with a character / part of speech that does not end at the end of the compound word (step D5). If it does not end with the corresponding character / part of speech, the compound word extraction function 42 sets all the notations as compound word candidates (step D7).

  The compound word extraction function 42 can execute compound word extraction with reference to the list shown in FIG. 12, for example. The specification of each line of the list shown in FIG. 12 is that when both “part of speech” and “expression” are described, a morpheme whose both part of speech and expression match, and when only one is specified, the other is unconditional. Used for judgment. It should be noted that not only the maximum length character string that does not correspond to the list of FIG.

  When the compound word extraction function 42 extracts compound words based on the list shown in FIG. 12, the morphological analysis result shown in FIG. 5 indicates that “the complaint of cold, initial cold, initial symptoms of cold, initial symptoms of cold, Compound word candidates of “initial symptoms, complaints of initial symptoms, prescription kakkonto, kakkento” can be extracted.

  If the designations related to the punctuation marks and punctuation marks are deleted from the list of FIG. 12, for example, the original sentence “New Chu-Hi“ XXX ”is announced. ”Can extract compound word candidates“ new chu-hi ”,“ XXX ”, and“ announcement ”.

  The compound word extraction function 42 is not limited to the one that creates a morpheme concatenation from the output of the morpheme analysis function 41 as a premise. For example, by inputting the text of the plaintext corpus 24a or the original text of the formal name list 24b and cutting out word candidates by N-gram, the morphological analysis result and the delimiter position match, and a notation that does not correspond to the list of FIG. Good.

  In this way, by extracting compound word candidates flexibly, it is possible to reduce compound word extraction omissions as compared to the case where limited candidates are extracted by applying a rule with a high relevance ratio. .

  In the above description, the case where the speech recognition system 49 supports the addition of a vocabulary to a dictionary for speech recognition (built dictionary 24e) is taken as an example, but the vocabulary knowledge acquisition device 10 in the present embodiment is It can also be used when adding a notation to a dictionary used in a system other than voice recognition. For example, a kana-kanji conversion dictionary of a Japanese input system (word processor), a term dictionary for classifying information distributed on the Internet (blog, microblog, company announcement information) and the like according to contents can be targeted.

  Moreover, the vocabulary knowledge acquisition apparatus 10 can target not only Japanese notation but also other language notations.

  In the above description, the unknown word reading is extracted from the web crawling data 24d obtained from the website. However, it is also possible to target other data obtained from outside the vocabulary knowledge acquisition apparatus 10. It is. For example, a database system in which data is continuously updated, data recorded in a specific electronic device, or the like can be acquired through a recording medium or the network 12 and used for vocabulary knowledge acquisition processing.

  The method described in the embodiment is a program that can be executed by a computer, such as a magnetic disk (flexible disk, hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), a magneto-optical disk (MO), a semiconductor memory, etc. It can also be stored in a storage medium and distributed.

  In addition, as long as the storage medium can store a program and can be read by a computer, the storage format may be any form.

  In addition, an OS (operating system) running on a computer based on an instruction of a program installed in the computer from a storage medium, MW (middleware) such as database management software, network software, and the like realize the above-described embodiment. A part of each process may be executed.

  Furthermore, the storage medium in the embodiment is not limited to a medium independent of the computer, but also includes a storage medium in which a program transmitted via a LAN, the Internet, or the like is downloaded and stored or temporarily stored.

  Further, the number of storage media is not limited to one, and the case where the processing in each of the above embodiments is executed from a plurality of media is also included in the storage media in the present invention, and the media configuration may be any configuration.

  The computer in the embodiment executes each process in the embodiment based on a program stored in a storage medium. The computer includes a single device such as a personal computer, and a system in which a plurality of devices are connected to a network. Any configuration may be used.

  In addition, the computer in the embodiment includes an arithmetic processing device, a microcomputer, and the like included in the information processing device, and is a generic term for devices and devices that can realize the functions of the present invention by a program.

  In addition, although some embodiment of this invention was described, these embodiment is shown as an example and is not intending limiting the range of invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 10 ... Vocabulary knowledge acquisition device, 12 ... Network, 14 ... Web server, 20 ... Processor, 21 ... Memory, 21a ... Vocabulary knowledge acquisition program, 21b ... Speech recognition program, 24 ... Storage device, 24a ... Plain text corpus, 24b ... Formal Name list, 24c ... Japanese-English machine translation dictionary, 24d ... Web crawling, 24e ... Pre-built dictionary, 24f ... Temporary construction dictionary, 24g ... Vocabulary list, 25 ... Input unit, 26 ... Display unit, 27 ... Voice input unit, 28 ... voice output unit, 29 ... communication unit, 41 ... morphological analysis function, 42 ... compound word extraction function, 43 ... unknown word extraction function, 44 ... unknown word related information addition function, 45 ... abbreviation estimation function, 46 ... formal notation candidate Giving function, 47 ... result output function, 48 ... dictionary editing function, 48 ... voice recognition system.

Claims (8)

A morpheme analysis means for dividing a text contained in a plaintext corpus into words and adding a part of speech to each word;
Compound word extraction means for extracting a compound word based on the result of the morphological analysis;
An unknown word extraction means for comparing the word obtained by the morphological analysis and the compound word obtained by the compound word extraction with a registered word of the constructed dictionary and extracting an unknown word not registered in the constructed dictionary;
Extracting candidate readings for the unknown word from data acquired from the outside, and adding unknown word related information giving means to the unknown word as unknown word related information;
Abbreviation estimation means for generating abbreviations from compound words;
When the abbreviation generated by the abbreviation estimation means matches the unknown word, a formal notation candidate giving means for giving the unknown word as a formal notation candidate as a formal word candidate,
A vocabulary knowledge acquisition device comprising: a result output means for combining the unknown word, the unknown word related information, and the formal notation candidates, and arranging them in the order of high dictionary addition registration effect and outputting the result as a vocabulary list. The unknown word related information giving means is
As the unknown word related information, presumed part-of-speech, appearance frequency, reading / snippet / information source extracted from Web crawling data, constructed dictionary registered words with similar reading / notation / part of speech, similar to the unknown word The vocabulary knowledge acquisition apparatus according to claim 1, wherein at least one of information on the difference between the analysis results when the registered word usage frequency and the registered words in the dictionary are added and deleted is added and extracted. The unknown word related information giving means is
It has a website reliability evaluation list,
The vocabulary knowledge acquisition apparatus according to claim 2, wherein when adding reading / snippet information extracted from the Web crawling data, information is selected based on an evaluation value of a Web site set in the reliability evaluation list.   The lexical knowledge acquisition apparatus according to claim 1, wherein the result output means rearranges based on one or a plurality of combinations among a plurality of indices as a criterion for determining the dictionary additional registration effect. The compound word extraction means includes
The lexical knowledge acquisition apparatus according to claim 1, wherein from the result of the morphological analysis, all combinations of parts in which parts of speech that may constitute a compound word appear adjacently are extracted as compound word candidates. Further comprising dictionary editing means for acquiring information on the difference between analysis results using the constructed dictionary before and after adding unknown words included in the vocabulary list to the constructed dictionary;
The vocabulary knowledge acquisition apparatus according to claim 1, wherein the unknown word related information adding unit adds difference information of analysis results to the unknown word. Divide the text contained in the plaintext corpus into words, and perform morphological analysis to give parts of speech to each word,
Extract compound words based on the results of the morphological analysis,
Compare the word obtained by the morphological analysis and the compound word obtained by compound word extraction with the registered word of the constructed dictionary, and extract the unknown word that is not registered in the constructed dictionary,
Extracting candidate readings for the unknown word from data acquired from the outside, and giving the unknown word as unknown word related information,
Generate abbreviations from compound words,
When the abbreviation and the unknown word match, give the compound word as a formal notation candidate as the formal word candidate to the unknown word,
A vocabulary knowledge acquisition method of combining the unknown word, the unknown word related information, and the formal notation candidates, and arranging them in the order of high dictionary addition registration effect and outputting as a vocabulary list. Computer
A morpheme analysis means for dividing a text contained in a plaintext corpus into words and adding a part of speech to each word;
Compound word extraction means for extracting a compound word based on the result of the morphological analysis;
An unknown word extraction means for comparing the word obtained by the morphological analysis and the compound word obtained by the compound word extraction with a registered word of the constructed dictionary and extracting an unknown word not registered in the constructed dictionary;
Extracting candidate readings for the unknown word from data acquired from the outside, and adding unknown word related information giving means to the unknown word as unknown word related information;
Abbreviation estimation means for generating abbreviations from compound words;
When the abbreviation generated by the abbreviation estimation means matches the unknown word, a formal notation candidate giving means for giving the unknown word as a formal notation candidate as a formal word candidate,
A vocabulary knowledge acquisition program for functioning as a result output means for combining the unknown word, the unknown word related information, and the formal notation candidates and arranging them as a vocabulary list in order from the highest dictionary addition registration effect.

Images