JP6300596B2 - Dictionary device, morpheme analyzer, data structure, morpheme analysis method and program - Google Patents

Description

  The present invention relates to a dictionary device for morphological analysis, a morphological analysis device, a data structure, a morphological analysis method, and a program for storing character string data in a structure based on a trie.

  Text-to-speech conversion (Text-To-Speech, TTS) is a typical usage form of speech synthesis technology. Text-to-speech conversion is a process of synthesizing a speech waveform corresponding to input text. In the following, this series of processes is roughly divided into two processes: a process of analyzing input text to generate information on how to read the text, and a process of synthesizing a speech waveform from the information on how to read. Also, it is assumed that the input is a Japanese kanji kana mixed sentence.

  Hereinafter, a symbol used for expressing information on how to read is referred to as a speech synthesis symbol. There are various forms of symbols for speech synthesis. Here, we assume that the phonetic information that composes a series of speech and the prosodic information that is mainly expressed as a pose or voice pitch. . An example of such a symbol for speech synthesis is JEITA (Electronic Information Technology Industry Association) standard IT-4006 “Symbol for Japanese Text Speech Synthesis” (see Non-Patent Document 1). Although it is difficult to express even emotional expressions of speech with these symbols alone, it contains at least the information necessary to describe the linguistic features of normal speech.

  On the other hand, the process of synthesizing the speech waveform is performed so that the waveform according to the speech synthesis symbol is synthesized. Therefore, in order to realize accurate reading of the Japanese text, it is only necessary to create an accurate speech synthesis symbol corresponding to the Japanese kanji kana mixed text.

  The process of generating symbols for speech synthesis from arbitrary Japanese text is to divide Japanese Kanji kana mixed sentences into the smallest units that have meaning in linguistic expressions called morphemes, add readings for each morpheme, It can be realized by referring to the morpheme information appropriately, inserting prosodic boundaries such as poses as necessary, and connecting them. At this time, the reading of each morpheme is created and stored in advance as morpheme dictionary information.

  However, the morpheme does not have to be as defined in linguistic terms, and may be a unit appropriately divided for performing a series of processing. For example, in order to more appropriately process the arrangement of morphemes, a phrase composed of a plurality of morphemes (such as a compound noun phrase) may be regarded as one morpheme for convenience. Therefore, in the following, a morpheme refers to a sequence of characters (character string) that is appropriately set as a minimum unit for processing from the viewpoint of its use, and all sentences are concatenated with this character string. It can be configured with. A process of dividing a sentence into morphemes is generally called a morpheme analysis process, and is used not only for speech synthesis processing but also for extraction of sentence components.

  As a morphological analysis method, a method based on the minimum cost method will be described below. In the morpheme analysis by the minimum cost method, first, an occurrence cost function reflecting the appearance frequency of each morpheme and a concatenated cost function representing ease of connection of consecutive morpheme are defined in advance. Then, an appropriate morpheme sequence is obtained by searching the morpheme registered in the morpheme dictionary for a morpheme sequence that matches the input text and minimizes the cost of the entire sentence. Usually, the occurrence cost function is defined so that the morpheme having the higher appearance frequency and the concatenated cost function have the smaller value as the morpheme sequence that is more easily connected.

  That is, when the morpheme sequence is M = (m1,..., Mn), the generation cost function is Ct (m), and the concatenation cost function is Cc (m (i−k + 1),..., Mi), the total cost Σ Ct + Σ A morpheme analysis process is performed by obtaining a morpheme string M that minimizes Cc, that is, argmin (Σ Ct + Σ Cc). Here, it is assumed that the concatenated cost function is determined by an array of k morphemes.

  When the cost function is defined in this way, the partial series constituting the optimal whole series in terms of cost are optimal in terms of cost even when only the partial series is viewed. Therefore, the subsequence that is not optimal in terms of cost does not need to be considered in the search because it is not a component of the optimal overall sequence. As described above, the optimum sequence search method that does not consider a partial sequence that does not have the possibility of forming an optimum sequence is generally called dynamic programming, and the optimum sequence can be searched efficiently.

  Among the components of the cost function, information on the occurrence cost can be held as the contents of the morpheme dictionary. On the other hand, the connection cost can be obtained by creating a table called a connection table in advance and using the values in the table. However, since it is difficult to create a table of all morpheme sequence combinations, for example, a table that focuses only on the morpheme part-of-speech type is also used. In some cases, these functions are defined as having larger values. In that case, a morpheme sequence having the largest value of the entire sentence is searched.

  In the morpheme string search process in morpheme analysis, it is preferable to examine all possible morpheme sequences. Therefore, in normal morpheme analysis, in order to obtain a morpheme candidate, a partial character string starting from an arbitrary position in a sentence is used as a search key, and the morpheme registered in the morpheme dictionary is equal to the first partial character string of the key. The process of extracting all morphemes is repeated. Such a search is generally referred to as a common prefix search. As a data structure that expresses this relatively efficiently, trie and Patricia tree are known.

  Here, a trie is a multi-tree structure for storing a plurality of character strings, and here, it is assumed that each character is stored as a tree branch in order from the first character of each character string. In the trie, a common prefix between character strings is shared on the tree structure, and therefore, all registered words that are prefixes of the character strings to be searched are arranged on one path of the tree structure. That is, the common prefix search can be realized by tracing the tree in the leaf direction along the search key from the root of the trie.

  In addition, the Patricia tree means a node obtained by combining a node having only one child with the child node in the above-described trie. By this combination, not only one character but also a plurality of consecutive characters may be stored in one branch. In the following, a common prefix including a Patricia tree is shared on the data structure, and a tree structure as a whole is collectively called a trie.

"Symbol for Japanese text-to-speech synthesis" JEITA standard IT-4006, March 2010

  In many languages, not limited to Japanese, how to read them may not be determined regularly based on character notation. Or, even if there is a rule, it is complicated and it may not be practical to decide regularly. For example, whether or not to read a character string “I went” should be read as “I did” or “I did” is determined by the context before and after it, but in general it is automatic considering the context Processing is difficult. Therefore, for example, in Japanese text, additional information related to reading is given, for example, as ruby, and the reading / meaning of the information may be indicated.

  However, in many cases such information is only given in part. Furthermore, even when information such as ruby is given, information about accents is not usually given, so it is necessary to determine the accent type in some form in order to perform speech synthesis processing. .

  Based on this, in order to achieve high reading and accent accuracy in speech synthesis processing, for example, even if information such as ruby is given, it is not used directly as reading information, but instead of normal speech synthesis A method of performing the same morpheme analysis process as the process and determining the accent type or the like based on the reading information previously linked to the morpheme information is conceivable. The ruby information in this case is used as information for selecting a correct answer from a plurality of reading information given to the morphemes.

  However, more accurate morphological analysis can be realized by considering reading from the morphological analysis stage. Specifically, morphemes with the same written character string but different readings are treated as different morphemes, and by setting appropriate values for the generation cost and concatenation cost of each morpheme, morpheme analysis based on the minimum cost method is further improved. Can be done with precision. In this case, if there is information such as ruby, only morphemes that match ruby are listed as candidates as morpheme candidates, while if there is no ruby, all morphemes that match the written character string are listed as candidates, Those optimum sequences will be searched.

  In order to efficiently perform such processing, it is necessary to be able to efficiently acquire reading information about morphemes at the morpheme dictionary reference stage in morpheme analysis. In such a process, if a morpheme candidate that does not match the constraint on reading of a morpheme given through ruby or the like can be discarded as early as possible in the morpheme candidate enumeration process, the process can be made more efficient.

  However, for example, when a morpheme dictionary based on a tree structure is used, the reading information is linked only to the leaf of the tree, so the partial reading information up to the traced part is traced from the root to the leaf node. I can't even get. In other words, morpheme candidates that do not match the restrictions on reading must be traced to the leaf nodes of the tree structure dictionary.

  In the conventional trie based morpheme dictionary structure for common prefix search, when the tree is traced from the root node to the leaf node, the branch corresponding to the last character of the notation All the information about the morpheme, including the information about reading, was linked to the node immediately after. Alternatively, when a code indicating the end of a character string is defined on the morpheme dictionary, all information regarding the corresponding morpheme is linked to the leaf node immediately below the branch of the end character.

  In other words, the morpheme information was obtained only after tracing the tree to the node corresponding to the morpheme terminal. For this reason, when performing morpheme analysis processing with restrictions on reading, first extract morphemes registered in the dictionary by common prefix search using only the written character string information, and then select morpheme candidates that do not meet the reading restrictions from the hypothesis. It is necessary to perform the discarding operation.

  The present invention has been made in view of such circumstances, and a dictionary device and a morphological analysis that can discard candidates that do not satisfy the condition at each node during morphological analysis and can improve the processing efficiency. An object is to provide a device, a data structure, and a method and program for morphological analysis.

  (1) In order to achieve the above object, the dictionary device of the present invention is a morpheme analysis dictionary device that stores character string data in a structure based on a trie that branches from a root to a leaf, and constitutes a morpheme. A character or partial character string is stored in association with a branch, and at least information about the character or partial character string associated with the closest branch on the root side of the node connecting the branches is associated with all nodes. It is characterized by being stored.

  Thereby, the morpheme analyzer can refer to information related to reading from the beginning to the middle of the registered morpheme during the common prefix search. For example, if the reading of the first character does not satisfy the reading constraint, it is determined at that time that all of the registered morphemes starting from the character and the reading are not correct in the morphological analysis. Further, it is not necessary to follow the dictionary of the trie structure any more, and the processing can be made efficient. As a result, candidates that do not satisfy the condition at each node can be discarded at the time of morphological analysis, and the processing can be made more efficient.

  (2) Further, the dictionary device of the present invention provides the node with information relating to reading of a character or partial character string associated with the closest branch on the route side, identification information of the information relating to the reading, and information relating to the reading. On the other hand, the identification information of the reading information is stored in association with the node near the route. As a result, candidates that do not satisfy the condition can be discarded at a node where a plurality of readings among the readings continuing from the root occur.

  (3) Further, in the dictionary device of the present invention, the node is associated with information related to reading of a character string formed by sequentially connecting characters or partial character strings associated with a branch of a route from a route to the node. It is characterized by being attached and stored. As a result, candidates that do not satisfy the condition can be discarded at a node where a plurality of readings among the readings continuing from the root occur.

  (4) Moreover, the morpheme analyzer of this invention is a morpheme analyzer which outputs the reading information regarding this based on a character string, Comprising: The dictionary apparatus in any one of (1) to (3) And a collating unit that divides the inputted character string into morphemes, collates the inputted character string for each morpheme in the order of character strings with the dictionary device, and outputs information candidates regarding reading for each morpheme And whether or not the information related to reading satisfies the predetermined constraint for each node, and discards candidates that do not satisfy the predetermined constraint. And a constraint reference unit. Thereby, candidates that do not satisfy the condition can be discarded at an early stage in the process of collating morpheme readings, and the process can be made more efficient.

  (5) Further, the data structure of the present invention is a data structure of a dictionary for morpheme analysis configured based on a trie branching from a root to a leaf in a storage unit in a computer, and includes a character or The partial character string is stored in association with the branch, and at least information on the character or partial character string associated with the closest branch on the root side of the node connecting the branches is associated with all the nodes. It is characterized by being stored. As a result, candidates that do not satisfy the condition at each node can be discarded at the time of morphological analysis, and the processing can be made more efficient.

  (6) Further, the method of the present invention is a morpheme analysis method for outputting reading information related to a character string, dividing the input character string into morphemes, and following the order of the character strings. Collating the input character string for each morpheme with the dictionary device according to any one of (1) to (3), outputting candidate information on reading for each morpheme, and the input characters Using a computer, referring to a predetermined constraint attached to a column, determining whether information about reading satisfies the predetermined constraint for each node, and discarding candidates that do not satisfy the predetermined constraint It is characterized by being executed. Thereby, candidates that do not satisfy the condition can be discarded at an early stage in the process of collating morpheme readings, and the process can be made more efficient. Thereby, candidates that do not satisfy the condition can be discarded at an early stage in the process of collating morpheme readings, and the process can be made more efficient.

  (7) Further, the program of the present invention is a morpheme analysis program that outputs reading information relating to a character string, and divides the input character string into morphemes and follows the order of the character strings. A process of collating the input character string for each morpheme with the dictionary device according to any one of (1) to (3), and outputting a candidate for information about reading for each morpheme, and the input character A series of processing including: referring to a predetermined constraint attached to a column, determining whether information about reading satisfies the predetermined constraint for each node, and discarding candidates that do not satisfy the predetermined constraint It is characterized by causing a computer to execute processing. Thereby, candidates that do not satisfy the condition can be discarded at an early stage in the process of collating morpheme readings, and the process can be made more efficient.

  According to the present invention, candidates that do not satisfy the condition at each node can be discarded at the time of morphological analysis, and the processing can be made efficient.

It is a block diagram which shows the morphological analyzer of this invention. It is a flowchart which shows operation | movement of the morphological analyzer of this invention. It is a figure which shows an example of the conventional data structure. It is a figure which shows an example of the data structure of this invention. It is a figure which shows an example of the data structure of this invention.

  Next, embodiments of the present invention will be described with reference to the drawings. In the following description, morphemes with different readings even if the notation character strings are the same are treated as different morphemes.

[First Embodiment]
(Configuration of morphological analyzer)
FIG. 1 is a block diagram showing a morphological analyzer 100. As shown in FIG. 1, the morphological analysis apparatus 100 includes a collation unit 110, a dictionary device 120, a constraint reference unit 130, a determination unit 140, and a connection table storage unit 150, and relates to reading based on an input character string. Determine and output suitable information.

  The collation unit 110 divides the input character string (kanji-kana mixed text) into morphemes, collates the character strings input for each morpheme in the order of the character strings, and outputs candidate information about reading. To do. The data in the dictionary device 120 is collated in the order along the stored character string.

  The dictionary device 120 stores character string data and information related to character string data. The dictionary device 120 stores character string data in a structure based on a trie that branches from a root to a leaf. The dictionary device 120 preferably has data having a data structure in which pronunciation information is embedded in the index.

  For example, in the dictionary device 120, characters or partial character strings constituting morphemes are stored in association with branches. Information on characters or partial character strings associated with at least the branch closest to the root of the node connecting the branches is stored in association with all nodes. The node preferably stores information relating to reading of a character string formed by sequentially connecting characters or partial character strings associated with branches of a route from the root to the node. As a result, candidates that do not satisfy the condition can be discarded at a node where a plurality of readings among the readings continuing from the root occur.

  If a morpheme that matches the partial character string up to that node is registered in the dictionary, the flag is set to 1, otherwise it is set to 0. Further, the generation cost information of the morpheme corresponding to the information related to reading whose flag is 1 is set for each reading information. The morpheme generation cost information includes information that is a parameter of the generation cost function, such as a morpheme part-of-speech type.

  When performing a morpheme analysis process using a data structure constituted by a trie, the tree is traced from the root node of the trie toward the leaf node as a common prefix search, and the morpheme on the node is output as a candidate.

  The constraint reference unit 130 refers to a predetermined constraint attached to the input character string, determines whether or not information related to reading satisfies the predetermined constraint for each node, and discards candidates that do not satisfy the predetermined constraint. . Thereby, candidates that do not satisfy the condition can be discarded at an early stage in the process of collating morpheme readings, and the process can be made more efficient.

  The constraint reference unit 130 refers to information related to the reading accompanying the input character string as a constraint, and outputs candidates that satisfy the constraint among candidates output as a result of collation. By using the data structure of the dictionary device 120, it is possible to easily refer to information related to reading for each morpheme, so that it is possible to realize a morpheme analysis process in consideration of restrictions on reading information.

  The determination unit 140 determines an optimal arrangement of the morpheme candidates output from the collation reference unit 130 based on the cost, and outputs the result as a morpheme sequence of the morpheme analysis result. The concatenation table storage unit 150 stores a concatenation table representing costs corresponding to partial character strings and consecutive readings.

(Specific data structure based on trial)
In the dictionary device 120, in each node including a leaf node of a morphological dictionary based on a trie, information related to reading of a partial character string corresponding to the root to each node (information related to the partial character string) is stored. However, multiple types of information regarding reading may be registered. In addition, a code indicating the end of the morpheme is not provided on the notation character string side, and for each reading information, a code indicating whether or not there is a morpheme that directly corresponds to the reading information (herein referred to as an end flag) is set. To do.

(Operation of morphological analyzer)
An operation example of common prefix search by the morpheme analyzer 100 will be described. FIG. 2 is a flowchart showing the operation of the morphological analyzer. First, a variable i is set to 1, and a trie root node is set to a variable n representing a trie node (step S1). Next, it is determined whether or not there is a branch of a trie node n that matches the i-th character of the search character string s (step S2). If so, the process ends.

  In step S3, a leaf-side closest node of n branches that matches the i-th character of the search character string s is newly set as a variable n, and the process proceeds to step S4. In step S4, it is determined whether or not the reading information corresponding to the i-th character in the search character string s satisfies the reading constraint. If satisfied, the process proceeds to step S5, and if not satisfied, the process ends. In this way, it is possible to more efficiently realize the morphological analysis processing with restrictions required for speech synthesis or the like.

  In step S5, it is determined whether there is a morpheme that terminates at node n. If there is, the process proceeds to step S6, and if not, the process proceeds to step S7. In step S6, morpheme candidates satisfying the reading constraint are added to the morpheme candidate list among the morphemes that end at node n, and the process proceeds to step S7. In step S7, i is incremented by 1, and the process returns to step 2. The above operation can be realized by causing a computer to execute a program.

(Concrete example)
FIG. 3 is a diagram illustrating an example of a conventional data structure. In the conventional morpheme dictionary structure, the morpheme information corresponding to the morpheme that ends with the node of the trie is associated with the trie node. For this reason, when performing a constrained search for reading information, it is necessary to list all the morpheme candidates by a common prefix search and then examine each morpheme information to discard candidates that do not satisfy the reading constraint.

  On the other hand, when the morpheme dictionary structure is constructed with reading information from the root node to the character corresponding to the node, the pronunciation information does not simply skip the pronunciation information, and the pronunciation information satisfies the restrictions on the given reading information. It can be checked at the stage of the search for a try.

  In this way, morphemes that do not satisfy the restriction condition of the reading information can be excluded from the morpheme candidate enumeration targets at an earlier time point. As a result, the number of candidate morphemes to be enumerated decreases, and the amount of processing and memory usage can be reduced.

  In the following, an example regarding the notation character string “Mita” will be considered. First, in this example, four morphemes of “Mita”, “Sanda”, “Santa”, and “Mitsuda” with different readings about “Mita” are registered in the dictionary. Further, in order to uniquely determine the reading for “Mita Library”, it is assumed that “Mita Library” is registered as a morpheme with the reading “Mitatoshokan”. Further, it is assumed that morphemes such as “library”, “ni” (particle), and “go” are registered in the morpheme dictionary. Furthermore, for the sake of simplicity of explanation, it is assumed that no other morpheme is registered in the morpheme dictionary.

  Here, if the morphological analysis of “go to Mita Library” is performed, there are two possibilities of “Mita”, “Library”, “Ni”, “Go”, or “Mita Library”, “Ni”, “Go”. The reason why the morpheme “Mita Library” is registered as described above is to fix the reading of this character string, and in many cases, the generation cost and concatenation cost are set so that the latter is the morpheme analysis result. .

  However, as a result, “Go to Mita Library” is read as “Mitatoshokaniku”, and even if the correct answer is “Sandatoshokaniku”, it cannot be read as such. In addition, if “go to Sanda Library” is entered in the form of changing the place where reading is restricted to Kana notation, the “Sanda” part is not registered in the dictionary, so there is no candidate in the morphological analysis result. Therefore, in normal morphological analysis, the part of “Sanda” is temporarily recognized as an unknown word morpheme, and a morpheme string from the beginning to the end of the sentence is forcibly created.

  However, in such morphological analysis, an analysis error is likely to occur, and “Sanda” is not a word registered in the morpheme dictionary, so accurate information regarding accent type reading cannot be obtained. This causes unnatural reading such as accent type errors.

  On the other hand, if it is possible to accept the input “go to Mita <Sanda> library” (<> indicates reading constraint specification based on information such as ruby), “Mita” “library” “to” “go” ”,“ Mita Library ”,“ Ni ”, and“ Go ”, it can be seen that the latter is not the correct answer, and“ Sanda ”should be selected as the reading for“ Mita ”. However, in this method, it is necessary to first enumerate all possible combinations of morpheme strings and then select candidates that satisfy the reading constraints.

  By the way, in the morphological analysis process, a common prefix search is performed for each partial character string for the input character string such as “go to Mita Library”, “go to Ta Library”, “go to library”, etc. Each will be listed.

  If “Mita Library”, which does not satisfy the reading restrictions at the time of the common prefix search for the first “Go to Mita Library”, can be excluded from the candidates, the number of morpheme candidates to enumerate will be reduced, and the correct answer will be with less memory consumption and calculation amount A morpheme string can be obtained. Therefore, in the present invention, even in an intermediate node of a dictionary having a trie structure, morpheme candidates that do not satisfy the reading constraint can be prevented from being enumerated as candidates by including information related to reading so far.

(Application to specific examples)
The above example can be represented by the data structure of the first embodiment. FIG. 4 is a diagram illustrating an example of a data structure according to the first embodiment. In the example shown in FIG. 4, all the reading information corresponding to the notation character string up to the intermediate node is registered in the intermediate node. In a common prefix search with reading constraints, when tracing a trie from the root node, it is found that there are no morpheme candidates that satisfy the reading constraints when there are no more readings that satisfy the reading constraints. Thus, there is no need to perform processing for searching for morpheme candidates, and the processing amount can be reduced.

  In this case, for example, in a dictionary composed of five words “Mita” (Mita), “Mita” (Sanda), “Mita” (Santa), “Mita” (Mitsuda) and “Mita Library” (Mitashokan) The structure is as shown in FIG. In such an example, in the common prefix search for “go to Mita Library”, the morpheme whose reading of “Mita” is “Sanda” at the stage of tracing the letters “Figure” It turns out that it doesn't exist even if it traces any more.

[Second Embodiment]
In the above embodiment, information related to reading from the root to the node is stored in the node. However, difference information for each node may be stored in the node. That is, the node is associated with information related to reading of the character or partial character string associated with the closest branch on the root side and identification information of the information related to reading, and associated with the node closest to the root with respect to the information related to reading. The identification information of the reading information is stored. As a result, candidates that do not satisfy the condition can be discarded at a node where a plurality of readings among the readings continuing from the root occur.

  In the above embodiment, reading information is arranged for each node, but reading information in each node may be stored as additional difference information from the parent (root node side) node. In this case, considering the case where a plurality of reading information is stored in each node, information for determining which information is the difference is required.

  For this purpose, an ID number is set for each reading information of each node, and information indicating which ID of the reading information is different from the parent node may be added to each reading information. By this method, since the common part of the reading information is shared among the plurality of morphemes, the total size of the reading information in the morpheme dictionary can be reduced. Accordingly, each piece of information is composed of four items: a “reading ID” that identifies the reading at the node, a “previous reading ID” that indicates the reading ID used at the parent node, reading information, and a termination flag.

(Application to specific examples)
The above example can be represented by the data structure of the second embodiment. FIG. 5 is a diagram illustrating an example of a data structure according to the second embodiment. In the second embodiment, the difference information for the reading is stored in the intermediate node up to the previous intermediate node (on the root node side in the trie). By storing only the difference, the size of the data linked to each node is reduced, but multiple readings may be stored in each node, so it indicates the difference with respect to which reading of the previous node The identifier is also stored.

  In this case, in the common prefix search process, it is necessary to create a list of identifiers for readings that satisfy the reading constraint at each node. At a certain node, only the difference with respect to the reading satisfying the reading constraint at the preceding intermediate node is valid.

  In this case, for example, in a dictionary composed of five words “Mita” (Mita), “Mita” (Sanda), “Mita” (Santa), “Mita” (Mitsuda) and “Mita Library” (Mitashokan) The structure is as shown in FIG. With this structure, it is possible to determine whether or not there is a candidate that satisfies the reading constraint during the trie search in the common prefix search, and if there is no candidate, the common prefix search can be aborted as in the first embodiment. The process is the same.

(Variations, etc.)
In the description so far, the terminal code is not set on the morpheme character string side and the terminal flag is set on the reading information side. However, the terminal code character may be defined on the character string side. In this case, all the reading information of the node immediately below the branch corresponding to the terminal code character corresponds to the terminal flag in the above description being 1, while all the reading information of the other nodes is This corresponds to the end flag being 0.

  Note that the restriction of reading information in the common prefix search considering reading information is not limited to the perfect match condition. In ruby, stuttering is often described as, for example, “Kiya” instead of “Kya”, but in order to deal with such cases, it is possible to use a method for determining the ambiguity. In this case, for example, it is possible to use an editing distance and to allow replacement, insertion, and omission up to a certain number of characters regarding the reading information. Actually, it is not easy to describe accent type information and prosodic boundary information (accent phrase boundary information, etc.) as reading restrictions as defined in the KEITA IT-4006 kana level notation. It is effective to have

  Also, the reading information is not limited to direct notation such as JEITA IT-4006. For example, “additional substitution of up to two symbols on reading symbol notation” is added as additional information for reading fluctuations. A rule such as “acceptance of muddy sound” may be added to the reading information in the form of a code representing it, and this may be taken into account by a judgment criterion that recognizes ambiguity. As a result, fluctuations in the reading instruction caused by the influence of, for example, rendaku (a phenomenon in which reading is performed when two morphemes are connected and the beginning of the subsequent morpheme becomes muddy) can be taken into consideration in the processing.

  In the method of the present invention, information other than reading information may be stored, and the information may be used for the presence / absence of enumeration determination when enumerating candidate morphemes. For example, in a morphological analysis system in which the morpheme occurrence cost is defined as the sum of the occurrence costs of each character in the character notation, a symbol representing the character occurrence cost is defined, and a set of characters and this symbol is arranged to indicate a trie index. By configuring the information, the information about the morpheme generation cost can be directly embedded in the trie structure itself.

  In this structure, since the cost of occurrence is obtained in the process of following the trial, enumeration of candidates taking into account the cost of occurrence of the morpheme is performed, such as discarding candidates with high morpheme generation cost in the process of enumerating candidates for morpheme. be able to. The information to be stored may be a combination of various information such as a combination of reading information and a character generation cost.

  In the above description, the table linked to the node is configured with three types of information, but the configuration of the information included in the table is not limited to this. For example, information such as “permits replacement of up to two symbols on reading symbol notation” and “accepts muddying at the beginning of reading information” may be added as additional information for fluctuations in reading. As a result, fluctuations in the reading instruction caused by the influence of, for example, rendaku (a phenomenon in which reading is performed when two morphemes are connected and the beginning of the subsequent morpheme becomes muddy) can be taken into consideration in the processing. In this case, instead of outputting the reading information registered in the dictionary as morpheme information, it is also possible to regard the restriction information related to reading as a strong reading specification and output the specified reading as morpheme information as it is.

  Furthermore, it is possible to construct a dictionary in which all written character strings are arranged in reverse order. For example, a morpheme dictionary in which character strings are reversed in front and back is constructed, and in the morpheme analysis of a sentence, a method of enumerating morpheme candidates from the end of the sentence toward the head is conceivable. In this case, the optimal morpheme sequence finally obtained is determined from the beginning of the sentence toward the end of the sentence.However, in many devices, the morpheme analysis result is also output from the beginning of the sentence toward the end of the sentence. Unlike the case of determining from the end of the sentence to the beginning of the sentence, there is an advantage that it is not necessary to temporarily store the morphological analysis result.

100 Morphological Analysis Device 110 Collation Unit 120 Dictionary Device 130 Constraint Reference Unit 140 Determination Unit 150 Concatenated Table Storage Unit n Trie Nodes Search String

Claims (5)

A morphological analyzer that outputs information about readings based on character strings,
A dictionary device for morpheme analysis that stores character string data in a structure based on a trie branching from a root to a leaf, wherein a character or a partial character string constituting the morpheme is stored in association with a branch, and at least A dictionary device in which reading information related to characters or partial character strings associated with the closest branch on the root side of the node connecting the branches is stored in association with all the nodes ;
The input character string is divided into partial character strings , the input character strings are collated with the dictionary device for each partial character string in the order of character strings , and candidates for information related to reading are determined for each partial character string. A collation part to output,
Refers to the restriction on the reading information attached to the input character string, determines whether the information on reading satisfies the restriction on the reading information for each node, and does not satisfy the restriction on the reading information A morpheme analyzer comprising: a constraint reference unit that discards candidates. In the dictionary apparatus, the node includes information related to reading of a character or a partial character string associated with a branch closest to the root side, identification information of the information related to the reading, and information on the root side closest to the information related to the reading. The morpheme analyzer according to claim 1, wherein identification information of reading information is stored in association with a contact node. In the dictionary device, the node stores information related to reading of a character string formed by sequentially connecting characters or partial character strings associated with a branch of a route from a route to the node. The morpheme analyzer according to claim 1, wherein A morphological analysis method based on a character string that outputs reading information about this,
Separate the input character string into partial strings, morphemes that stores data of a string the string is the input for each of the partial character string along the order of the string from the root structure based on trie that branches to the leaf A dictionary device for analysis, wherein a character or a partial character string that constitutes a morpheme is stored in association with a branch, and at least a character or a character corresponding to a root side closest branch of a node connecting the branch Collating with the dictionary device in which the reading information about the partial character string is stored in association with all the nodes, and outputting candidate information about the reading for each partial character string ;
Refers to the restriction on the reading information attached to the input character string, determines whether the information on reading satisfies the restriction on the reading information for each node, and does not satisfy the restriction on the reading information And discarding the candidates, using a computer. A morphological analysis program that outputs information about readings based on character strings,
Separate the input character string into partial strings, morphemes that stores data of a string the string is the input for each of the partial character string along the order of the string from the root structure based on trie that branches to the leaf A dictionary device for analysis, wherein a character or a partial character string that constitutes a morpheme is stored in association with a branch, and at least a character or a character corresponding to a root side closest branch of a node connecting the branch A process of collating with a dictionary device in which reading information related to partial character strings is stored in association with all nodes, and outputting candidate reading information for each partial character string ;
Refers to the restriction on the reading information attached to the input character string, determines whether the information on reading satisfies the restriction on the reading information for each node, and does not satisfy the restriction on the reading information A program for causing a computer to execute a series of processes including a process of discarding candidates.

Images