JP5834772B2 - Information processing apparatus and program - Google Patents

Description

  The present invention relates to an information processing apparatus and a program.

  A translation device that translates a word string (original text) including a plurality of words into another language is known. When translating an original sentence, such a translation apparatus performs translation processing by estimating which word (word-to-word) the original sentence is divided.

In relation to a method for estimating how to separate a document or a word string, Patent Document 1 proposes a technique for inferring how to separate a document using a parser that has been programmed in advance with the grammar rules of the language to which the document belongs. Yes.
Patent Document 2 proposes a technique for dividing a character string that is not divided into words.

JP-A-6-309310 JP-A-10-254874

  In the technique of Patent Document 1, in order to infer which words are separated from each other in the original text, a syntax analyzer that is programmed with the grammar rules of the language to which the original text belongs is used. Therefore, it was necessary to create a parser for each language to which the original text belongs, spending a lot of development costs and days. Further, Patent Document 2 discloses a technique for dividing a character string that is not divided into words, but does not disclose a method for determining which word and word are separated by the character string. .

  Words from teacher data indicating how word strings in the same category as the original text are separated as a technique for inferring which words are separated from each other without using a parser There is a method for obtaining the probability of being divided between a word and a word.

  However, a sufficient number of all n-grams included in the original text is used to infer how the words are divided for n words (n-grams) included in the original text using the teacher data. Teacher data is required. For this reason, there is a problem that the amount of labor and calculation for collecting and processing a huge amount of teacher data becomes enormous.

  The present invention has been made in view of the above circumstances, and in the word string to be analyzed, information processing with a small amount of teacher data required to obtain the probability that the word string is divided between a word and a word An object is to provide an apparatus and a program.

In order to achieve the above object, an information processing apparatus according to the present invention provides:
A word string acquisition unit for acquiring a word string having a plurality of words;
An extraction unit for extracting a plurality of partial word strings including one or more words included in the word string acquired by the word string acquisition unit;
The partial word sequence extracted by the extraction unit with a delimiter pattern corresponding to each delimiter pattern between the case where the word sequence is delimited and the case where the word sequence is not delimited between each word between the words constituting the word sequence A word string belonging to the same category as the word string for each of the extracted division patterns is obtained for each of the division probability coefficients indicating the degree of probability that the partial word string is divided by the division method corresponding to the division pattern. A teacher word string having the same pattern as the delimiter pattern from a teacher word string storage unit storing a teacher word string defining whether or not the word string is delimited between words of the word string Based on the number of teacher word strings extracted and extracted, a probability coefficient acquisition unit to acquire,
A dividing unit that divides the word string acquired by the word string acquisition unit based on the division probability coefficient acquired by the probability coefficient acquisition unit;
It is characterized by providing.

  According to the present invention, it is possible to provide an information processing apparatus and a program that require a small amount of teacher data in order to obtain a probability that a word string is divided between certain words in a word string to be analyzed. it can.

It is a block diagram which shows the structure of the menu display apparatus which concerns on Embodiment 1 of this invention. (A) is a figure which shows the relationship between the character string which concerns on Embodiment 1, and teacher data, (b) is a figure which shows the relationship between a word string, a division | segmentation flag, a trigram, and a division | segmentation pattern. It is a block diagram which shows the structure of the probability coefficient output device which concerns on Embodiment 1, (a) shows a physical structure, (b) shows a functional structure, respectively. It is a figure which shows the example of the n-gram list which concerns on Embodiment 1, (a) shows a trigram list, (b) shows a bigram list, (c) shows a monogram list, respectively. It is a figure which shows the outline | summary of the probability coefficient calculation process which concerns on Embodiment 1, (a) is an example of the process which calculates the division | segmentation probability coefficient of a trigram from the division | segmentation probability coefficient of a bigram, (b) is a division | segmentation probability coefficient of a monogram. An example of processing for calculating a trigram segmentation probability coefficient from each is shown. 4 is a flowchart illustrating menu display processing according to the first embodiment. 4 is a flowchart illustrating menu division processing according to the first embodiment. 6 is a flowchart illustrating a probability coefficient acquisition process according to the first embodiment. 5 is a flowchart illustrating a probability coefficient calculation process according to the first embodiment. 6 is a flowchart illustrating calculation processing for each separation pattern according to the first embodiment. It is a figure which shows the outline | summary of the probability coefficient calculation process which concerns on Embodiment 2 of this invention, (a) is an example of the process which calculates the division | segmentation probability coefficient of a trigram from the division | segmentation probability coefficient of a bigram, (b) is a monogram and An example of a process for calculating a trigram segmentation probability coefficient from a bigram segmentation probability coefficient is shown in FIG. 10 is a flowchart illustrating a probability coefficient calculation process according to the second embodiment. 10 is a flowchart showing calculation processing for each separation pattern according to the second embodiment. It is a flowchart which shows the n-gram (trigram) pattern probability coefficient list which concerns on other embodiment of this invention.

  A menu display device and a probability coefficient output device (information processing device) according to an embodiment for carrying out the present invention will be described below with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals.

(Embodiment 1)
The probability coefficient output device 40 according to the first embodiment is mounted on the menu display device 1 shown in FIG. The menu display device 1 i) a photographing function for photographing paper or the like describing a character string (menu, menu, etc.) belonging to a specific category to be analyzed; ii) recognizing a character string to be analyzed from the photographed image Iii) A function to analyze the extracted character string and convert it to a word string, iv) Output a coefficient (separation probability coefficient) indicating the probability that the menu will be separated at a predetermined part of the character string (between words) V) a function for dividing a word string based on a dividing probability coefficient, vi) a function for translating each divided word string, vii) a function for displaying a translation result, and the like. The probability coefficient output device 40 is in charge of a function of outputting a coefficient (probability coefficient) indicating a probability that the menu is divided at a predetermined portion (between words) of the character string among these functions.

  The menu display device 1 includes an input unit 10, an information processing unit 70 including an OCR (Optical Character Reader) 20, a menu analysis unit 30, and a translation unit 50, a probability coefficient output device 40, and a display unit 60.

  The input unit 10 includes a camera and an image processing unit, and acquires an image obtained by photographing a menu with such a physical configuration. The input unit 10 transmits the acquired image to the OCR 20.

  The information processing unit 70 includes an image processing DSP (Digital Signal Processor), a RAM (Random Access Memory) used as a work area, an EPROM (Erasable Programmable Read Only Memory) that stores a character recognition program and a language processing program, and the like. Is done. The information processing unit 70 functions as the OCR 20, the menu analysis unit 30, and the translation unit 50 with such a physical configuration.

  The OCR 20 recognizes the characters of the image transmitted from the input unit 10 and acquires a character string (such as a dish name) written on the menu. The OCR 20 transmits the acquired character string to the menu analysis unit 30.

The menu analysis unit 30 divides the character string transmitted from the OCR 20 into words and converts them into word strings.
The menu analysis unit 30 extracts a partial word string (n-gram) composed of n words appearing in the word string. Furthermore, a delimiter pattern to be described later is generated from the n-gram, and a delimiter pattern for which a probability coefficient needs to be acquired is selected.

Here, the character string (menu) to be analyzed, the teacher data, the n-gram, and the partition pattern according to the present embodiment will be described with reference to FIG.
The character string to be analyzed in the present embodiment is a character string indicating a menu as shown in the upper part of FIG. The data shown in FIG. 2 is the teacher data (bottom of FIG. 2 (a)) with a tag attached to the menu "pork belly with red wine cooked with vegetables" and divided into words and chunks. In the example of Fig. 2 (a), the teacher data is "<m><c><s><w> pig </ w> rose meat </ w><w></w></s><s><w> Red wine </ w><w> boiled </ w></s><s><w></c> warm vegetables </ w><w> with </ w></s>< / m> ”.

  In this teacher data, the menu is divided into seven words “pork”, “rose meat”,..., “Attached” by tags <w> </ w> indicating words. Furthermore, the tag <s> </ s> that is divided into units such as ingredient name, cooking method, etc., is divided into three parts: "pork belly meat", "boiled red wine", "with warm vegetables" Yes. In addition, the tag <c> <c /> that divides into one cooking unit that includes the ingredient name, cooking method, and other modifiers (for example, “Provence style”, “Specialties”, etc.) It is divided into “red wine boiled” and “warm vegetables”. Tags <m> </ m> are tags that separate character strings into menus (menus). Here, in the teacher data, character strings are delimited by tags <w>, <s>, <c>, and <m>, but the format of the teacher data is not limited to this. The teacher data is an arbitrary character string that includes a unique mark (a single-byte space is allowed) that divides a character string included in a predetermined category into words, and a unique mark that is further separated by at least one separation method other than words. It may be. The teacher data is data in which character strings belonging to a specific category (here, menu or dish name) of a specific language (here, Japanese) are collected and tagged manually. Note that the method of tagging the teacher data is not limited to manual operation, and any known tagging method such as a syntax analyzer may be used.

  FIG. 2B shows the relationship between the teacher data, the n-gram, and the division pattern. An n-gram is obtained by extracting a set of word strings including n words such as the nth word from the first word, the (n + 1) th word from the second word,... Is a column. A word string including n words constituting the n-gram string is called an n-gram. Further, n-grams with n = 3 are called trigrams, n-grams with n = 2 are called bigrams, and n-grams with n = 1 are called monograms.

  You can obtain a trigram sequence consisting of trigram "pork rose meat", "red rose wine", ..., "boiled warm vegetables" from "pig rose meat with red wine stewed vegetables" (FIG. 2 (b)). As shown in the upper part of FIG. 2B, the menu word string is divided into a tree structure by the tag structure. Then, it is possible to determine where a word is divided between words at a predetermined height (corresponding to a predetermined tag of teacher data) determined by the system design.

In the example of the tree in FIG. 2 (b), the part with the tag <m> or </ m>, the part with the tags <s> and </ s>, the part with the tags <c> and </ c> The menu is separated by each (separation line). Information between each word in the word string is called a delimiter flag when it is delimited by 1 and when it is not delimited by 0.
Note that the criteria for determining which tag is delimited by a certain part can be freely set. For example, it is possible to make arbitrary settings such as a setting for determining that the <s></s> tag is separated only by a portion and arranging a separation flag.

For trigrams (n-grams), a pattern that defines whether or not teacher data is divided between words of the n-gram is defined by arranging words and a delimiter flag side by side is called a delimiter pattern.
For example, for the three words (word A, word B, word C) constituting the trigram, this corresponds to the case where the teacher data is not divided between any words including the word A before and the word C. The delimiter pattern is “0A0B0C0”, and the delimiter pattern corresponding to the case where all words are delimited is “1A1B1C1”.

  In the example of FIG. 2 (b), for the trigram “pig of pork”, in front of the word “pig”, between “pig” and “rose”, between “rose” and “no”, After “no”, it is possible to define 2 ^ 4 = 16 division patterns between the four words, where 1 is the case where the menu is divided and 0 is the case where the menu is not divided. The delimiter pattern corresponding to the teacher data is “1 pig 0 rose 1 0”.

  In the following, a sufficient amount of teacher data is created from menu / cook names belonging to the language (for example, Japanese). A coefficient indicating the probability that the teacher data includes a certain n-gram and is divided by a certain delimiter pattern is called a probability coefficient (delimiter probability coefficient) of the delimiter pattern. In addition, the probability coefficient of the division pattern corresponding to a certain n-gram is referred to as the probability coefficient of the n-gram.

  The menu analysis unit 30 uses the probability coefficient acquired from the probability coefficient output device 40 to estimate how the word string is divided, and outputs the word string to the translation unit 50 in association with the estimation result. Specific processing executed by the menu analysis unit 30 will be described later.

  The menu analysis unit 30 transmits information indicating n words (n-gram) and a delimiter pattern that requires a probability coefficient among n-gram delimiter patterns to the probability coefficient output device 40. Get the probability coefficient of the delimiter pattern from.

When the probability coefficient output device 40 receives n words (n-gram) and information indicating a delimiter pattern that requires a probability coefficient among the n-gram delimiter patterns from the menu analysis unit 30, the teacher data The probability coefficient indicating the probability that the menu is divided by the division pattern obtained from the above is transmitted to the menu analysis unit 30.
Specific processing executed by the probability coefficient output device 40 and its configuration will be described later.

The translation unit 50 divides the word string transmitted from the menu analysis unit 30 by the division method indicated by the division pattern of the word string and translates it into a language desired by the user.
The translation unit 50 may translate any known translation method, but here, the words included in the divided word string are sequentially translated by dictionary data.
The translation unit 50 transmits the translation result to the display unit 60.

  The display unit 60 includes a liquid crystal display or the like, and displays information transmitted from the translation unit 50.

Next, the configuration of the probability coefficient output device 40 will be described with reference to FIG.
The probability coefficient output device 40 physically includes an information processing unit 401, a data storage unit 402, a program storage unit 403, an input / output unit 404, a communication unit 405, as shown in FIG. And an internal bus 406.

  The information processing unit 401 includes a CPU (Central Processing Unit), a DSP (Digital Signal Processing), and the like, and is executed by a probability coefficient output device 40 described later according to a control program 407 stored in the program storage unit 403. Execute.

  The data storage unit 402 includes a RAM (Random-Access Memory) and the like, and is used as a work area for the information processing unit 401.

  The program storage unit 403 includes a nonvolatile memory such as a flash memory or a hard disk, and stores a control program 407 for controlling the operation of the information processing unit 401 and data for executing the following processing.

The communication unit 405 includes a LAN (Local Area Network) device, a modem, and the like, and transmits the processing result of the information processing unit 401 to an external device connected via a LAN line or a communication line. In addition, information is received from an external device and transmitted to the information processing unit 401.
Note that the information processing unit 401, the data storage unit 402, the program storage unit 403, and the input / output unit 404 are connected to each other via an internal bus 406, and can transmit information.

  The input / output unit 404 is an I / O unit that controls input / output of information to / from an external device connected to the probability coefficient output device 40 through a USB (Universal Serial Bus) or a serial port.

  As shown in FIG. 3B, the probability coefficient output device 40 has a word string acquisition unit 410, a determination unit 420, an (n-1) gram generation unit 430, a probability coefficient acquisition unit 440, a probability coefficient, as shown in FIG. It functions as a calculation unit 450, an output unit 460, and a storage unit 470.

  The word string acquisition unit 410 acquires information indicating an n-gram (word-of-interest string) included in the menu from the menu analysis unit 30 and a delimiter pattern generated from the n-gram (a delimiter pattern for which a probability coefficient needs to be acquired). To do. The word string acquisition unit 410 transmits the acquired n-gram and information to the determination unit 420.

The determination unit 420 determines whether or not the probability coefficient indicating the probability that the menu is divided according to the division method of the n-gram division pattern transmitted from the word string acquisition unit 410 can be acquired. The determination unit 420 refers to the n-gram list stored in the n-gram list storage unit 4710 of the storage unit 470 in the determination process. Specific contents of the discrimination process executed by the n-gram list and discrimination unit 420 will be described later.
In this embodiment, the probability that the learning will be divided by a certain division pattern is assumed, and the probability that the menu will be divided by the n-gram division pattern of the menu is assumed under the assumption that the word string (menu) to be analyzed is also divided. , (N-1) gram to monogram teacher data.

If the determination unit 420 determines that n-gram probability coefficients can be acquired, the determination unit 420 transmits n-grams to the probability coefficient acquisition unit 440.
On the other hand, if it is determined that n-gram probability coefficients cannot be acquired, n-grams are transmitted to the (n−1) -gram generation unit 430.

When (n-1) gram generation unit 430 receives n gram from discrimination unit 420, (n-1) gram (n-1) gram (n-1) from the first word constituting n gram (n-1) gram ( (N-1) gram) and n-1 gram (after (n-1) gram) composed of the nth word from the second word constituting.
The (n-1) gram generation unit 430 transmits the generated two (n-1) grams to the determination unit 420.

When two (n-1) grams are transmitted from the (n-1) gram generation unit 430, the determination unit 420 determines whether a probability coefficient can be acquired for each of the two (n-1) grams. . If the probability coefficient cannot be acquired for either of the two (n-1) grams, the (n-1) gram generation unit 430 generates three (n-2) grams, and the same determination is made until a monogram is obtained. Repeat the process. Specific contents of processing executed by the determination unit 420 and the (n-1) gram generation unit 430 will be described later.
When the determination unit 420 receives the monogram from the (n-1) gram generation unit 430, the determination unit 420 transmits the monogram to the probability coefficient acquisition unit 440 without executing the determination process.

When the n-gram acquired by the word string acquisition unit 410 is transmitted as the word string from the determination unit 420, the probability coefficient acquisition unit 440 indicates the probability coefficient of the partition pattern indicated by the information acquired by the word string acquisition unit 410 among the partition patterns. Is transmitted to the output unit 460.
On the other hand, when n-1 gram to monogram (here, j-gram) is transmitted as a word string, the probability coefficient of the division pattern indicated by the information acquired by the word string acquisition unit 410 for all the transmitted j-grams To generate a delimiter pattern necessary to acquire the probability coefficient and obtain a probability coefficient for each delimiter pattern.
Then, all the transmitted j-grams, the generated segmentation patterns, and the probability coefficients thereof are transmitted to the probability coefficient calculation unit 450.
The probability coefficient acquisition unit 440 refers to the teacher data stored in the teacher data storage unit 4730 in the process of acquiring the probability coefficient. How the probability coefficient acquisition unit 440 acquires the probability coefficient will be described later in detail.

The probability coefficient calculation unit 450 receives the delimiter pattern generated from all the transmitted j-grams from the probability coefficient acquisition unit 440, and the probability coefficient of the n-gram delimiter pattern from the transmitted information. Is calculated. How the probability coefficient calculation unit 450 calculates the probability coefficient of the n-gram segmentation pattern will be described later.
When the probability coefficient calculation unit 450 calculates the probability coefficient of the n-gram delimiter pattern, it extracts the delimiter pattern that requires the probability coefficient and the probability coefficient acquired by the word string acquisition unit 410 and transmits it to the output unit 460. To do.

  The output unit 460 outputs the division pattern and the probability coefficient transmitted from the probability coefficient calculation unit 450 to the menu analysis unit 30.

  The storage unit 470 receives information from each unit other than the storage unit 470 of the probability coefficient output device 40 and stores the transmitted information. Further, in response to a command from each unit other than the storage unit 470 of the probability coefficient output device 40, information to be stored is output.

  The storage unit 470 includes an n-gram list storage unit 4710 that stores an n-gram list, a setting storage unit 4720 that stores setting parameters for the probability coefficient output device 40 to execute processing described below, and a teacher data storage that stores teacher data Part 4730.

The n-gram list is a list in which all n-grams appearing in the teacher data stored in the teacher data storage unit 4730 are registered.
An example of an n-gram list stored in the n-gram list storage unit 4710 will be described with reference to FIG. In the example of FIG. 4, the n-gram storage unit 4710 stores a trigram list (FIG. 4A), a bigram list (FIG. 4B), and a monogram list (FIG. 4C).
The trigram list stores all the trigrams appearing in the teacher data in association with the number of data indicating how many teacher data includes the trigram. The same applies to the bigram list and the monogram list.

  Next, a process in which the probability coefficient calculation unit 450 calculates the probability coefficient of the n-gram partition pattern using the j-gram partition pattern and the probability coefficient will be described with reference to FIG.

  FIG. 5 shows a case where a certain trigram (here, “no boiled red wine”) is not registered in the trigram list (FIG. 4A) or a sufficient number (a predetermined threshold or more) is not registered. In addition, the probability coefficient calculation unit 450 calculates the probability coefficient of the trigram partition pattern from the probability coefficient of the bigram or monogram partition pattern.

  A calculation method in the case of calculating the attention partition pattern “0 1 red wine boiled 1” of the trigram “no red wine boiled” will be described with reference to FIG. Here, the first half of the trigram (front bigram) is “red wine”, and the second bigram (second bigram) is “red wine boiled”. The bigram delimiter patterns (corresponding delimiter patterns) corresponding to the target delimiter pattern are “0 red wine 0” and “1 red wine 0 boiled 1”, which have common delimiter flags between words.

  When the probability factor calculation unit 450 receives the corresponding bigram (j-gram) segmentation pattern and its probability factor from the probability coefficient acquisition unit 440, the probability factor calculation unit 450 compares the segmentation flags and extracts the corresponding segmentation pattern. Then, the probability coefficients of the corresponding partition patterns are added and averaged to obtain the probability coefficient of the target partition pattern (FIG. 5A).

  Similarly, when the corresponding monogram delimiter pattern and its probability coefficient are acquired from the probability coefficient acquisition unit 440, the corresponding delimiter patterns are extracted from the front, center, and tail monograms, and the probability coefficients of the corresponding delimiter patterns are added and averaged. Thus, the probability coefficient of the target separation pattern is set (FIG. 5B).

  Next, processing executed by the menu display device 1 will be described with reference to flowcharts (FIGS. 6 to 10). When the user performs an operation for photographing a menu, the information processing unit 70 of the menu display device 1 executes a menu display process shown in FIG.

In the menu display process, first, an image on which a menu is printed is acquired using the input unit 10 (step S101).
Then, from the acquired image, the OCR 20 recognizes a character and acquires a character string (step S102).

When the OCR 20 acquires a character string and transmits it to the menu analysis unit 30, the menu analysis unit 30 first executes a splitting process for dividing the character string into units of words to convert the character string into a word string. (Step S103).
Here, the menu analysis unit 30 may execute the above-described segmentation process using any known method for extracting a word from a character string, but here, the menu analysis unit 30 executes the segmentation process using a method exemplified in Patent Document 2. I will do it.
Note that if the menu to be analyzed is a language that is separated by a space for each word, such as English or French, the menu analysis unit 30 recognizes the space and executes the above-described splitting process.

  Then, the menu analysis unit 30 estimates at which part of the word string the menu is divided, and executes a process for dividing the menu (menu division process) (step S104).

The menu division process executed in step S104 will be described with reference to FIG.
When the menu division process is started, first, the menu analysis unit 30 generates an n-gram sequence from the word sequence (step S201). Each n-gram included in the n-gram string is a partial string of the word string.
Here, the value of n may be an arbitrarily determined default value, but here n = 3.

  Then, the counter variable i = 1 is set, and the first (first) n-gram (trigram) of the n-gram sequence is set as the target partial sequence (target trigram) (step S202).

  Then, the menu analysis unit 30 obtains the probability coefficient for all the delimiter patterns (eight) having the delimiter flag of 1 among the 16 types of probability coefficient output device 40 for the first trigram (target subsequence). The command is transmitted to the probability coefficient output device 40, and the probability coefficient acquisition process is started (step S203).

The probability coefficient acquisition process executed in step S203 will be described with reference to FIG.
In step S203, when the word string acquisition unit 410 receives an n-gram from the menu analysis unit 30 and a command regarding a delimiter pattern for which a probability coefficient is to be calculated, the probability coefficient output device 40 starts a probability coefficient acquisition process.
In the probability coefficient acquisition process, first, the determination unit 420 refers to the trigram list stored in the n-gram list storage unit 4710, and acquires the number of teacher data including the attention trigram (step S301).

  Then, whether or not there is a sufficient number of teacher data for obtaining the probability coefficient of the attention trigram, the threshold for the trigram stored in the setting storage unit 4720 and the number of data in the attention subsequence are compared. To determine.

  If the number of data in the target subsequence is equal to or greater than the threshold (step S302; YES), it can be determined that there is a sufficient number of teacher data, and therefore the delimiter pattern using the current target subsequence (n-gram) as it is. Are obtained (steps S303 to S304).

First, the probability coefficient acquisition unit 440 generates a break pattern transmitted from the menu analysis unit 30 (step S303), and acquires the probability coefficient (step S304). Specifically, the teacher data stored in the teacher data storage unit 4730 and including the target subsequence is extracted. The number of data extracted at this time is n1. The division flag of the corresponding part of the extracted teacher data is compared with the division flag of the division pattern, and the teacher data having the same division method is extracted. The number of data extracted at this time is n2. The probability coefficient p is obtained by the ratio of n1 and n2.
That is, p = n2 / n1.
Note that the method of obtaining p is not limited to this, and the value of p increases as n2 increases and decreases as n1 increases (for example, p = n2 ^ 2 / n1 ^ 2). You can ask.

  On the other hand, if the number of data in the subsequence of interest is smaller than the threshold value or not registered in the trigram list (step S302; NO), it can be determined that a sufficient number of teacher data is not stored, so n-1 gram Processing for calculating a probability coefficient using (bigram) to monogram (probability coefficient calculation processing, here, probability coefficient calculation processing 1) is executed (step S305).

The probability coefficient calculation process 1 executed in step S305 will be described with reference to FIG.
In the probability coefficient calculation process 1, first, the (n-1) gram generation unit 430 generates two (n-1) grams which are partial strings of the target character string (n gram) (here, the previous bigram in FIG. 5A). And later bigram).

  Then, the determination unit 420 can acquire probability coefficients for both of the two front bigrams and the rear bigrams, or stores them in the n-gram list storage unit 4710 as in step S302 of the probability coefficient acquisition processing (FIG. 8). The bigram list is compared and determined. Specifically, the number of corresponding bigram data is compared with the number of thresholds for a predetermined bigram (step S402). Here, the threshold for n-gram can be set arbitrarily, but preferably the larger the n, the larger the threshold. The number of delimiter patterns that can be defined from n-grams is 2 ^ n, and the number increases as n increases. This is because the number of teacher data sufficient to obtain the probability coefficients for each of a large number of delimiter patterns increases accordingly.

  If the number of data of all bigrams is equal to or greater than the threshold (step S402; YES), it can be determined that probability coefficients can be obtained for all of the (n-1) grams (bigrams), so the probability coefficients are calculated using bigrams. In order to do this (FIG. 5A), the process proceeds to step S406.

If the number of data of any bigram is smaller than the threshold (step S402; NO), it can be determined that the probability coefficient cannot be obtained for any bigram, so the probability coefficient is calculated using the monogram (FIG. 5B). ).
That is, it is determined whether the current n-1 is not 1 (step S403). If it is not 1 (step S403; NO), n is decremented by 1 to 2 (step S404), and the process returns to step S401 (n -1) A gram (here, a monogram) is generated.

  On the other hand, when n-1 is 1 (step S403; YES), since n cannot be further reduced, a delimiter pattern is generated for a monogram for which a probability coefficient cannot be obtained, and the probability coefficient is set to a default value ( Here, 0.5) is set (step S405).

  And if the value of (n-1) which can acquire all the probability coefficients is determined in step S401 to step S405, then for all of the (n-1) gram, the delimiter pattern transmitted from the menu analysis unit 30 and A delimiter pattern having a delimiter flag common to corresponding words is generated (step S406). Then, a probability coefficient is acquired for the generated delimiter pattern in the same manner as in step S304 of the probability coefficient acquisition process (FIG. 8) (step S407).

  Then, a process (calculation process 1 for each delimiter pattern) for calculating a probability coefficient for each delimiter pattern of the target subsequence (trigram) is executed (step S408), and the probability coefficient calculation process 1 is completed.

  The delimiter pattern calculation process 1 executed in step S408 will be described with reference to FIG. In the calculation process 1 for each delimiter pattern, first, among all delimiter patterns that can be generated from the target subsequence (target trigram), a delimiter pattern for which the probability coefficient requested from the menu analysis unit 30 is to be acquired is generated (step S501). .

  Then, paying attention to the kth delimiter pattern of the generated delimiter pattern using k as a counter variable (step S502). In the example of FIG. 5, “0 1 red wine 0 boiled 1” is the notable separation pattern.

  Next, for the (n-1) gram delimiter pattern (corresponding delimiter pattern and corresponding delimiter flag between corresponding words) corresponding to the delimiter pattern generated in step S406, the probability coefficient acquired in step S407 is noted. It adds to the probability coefficient of a division | segmentation pattern (step S503).

Next, it is determined whether all the delimiter patterns generated in step S501 have been processed (whether the probability coefficients of all delimiter patterns have been calculated and added) (step S504).
If there is an unprocessed delimiter pattern (step S504; NO), k is incremented (step S505), and the process from step S502 is repeated for the next delimiter pattern.

  On the other hand, when the processing has been completed for all the partition patterns (step S504; YES), the number obtained by adding the probability coefficients of the target partition pattern obtained by adding the probability coefficient of (n-1) grams in the above processing (current Divide by n) to make the value an addition average value, and the calculation process 1 for each separation pattern ends.

  Returning to FIG. 8, when the probability coefficient is acquired in step S304 or step S305, the probability coefficient acquired by the output unit 460 is output to the menu analysis unit 30 (step S306), and the probability coefficient acquisition process ends.

  Returning to FIG. 7, when the probability coefficient acquisition process (step S203) is completed, a pattern having the highest probability coefficient (maximum likelihood pattern) is selected from all the division patterns whose head division flag is 1 (step S204). ).

  Next, it is determined whether or not there is an unprocessed n-gram (step S205). If the i-th n-gram is not the last n-gram of the word string to be analyzed, it is determined that there is an unprocessed n-gram (step S205; YES), and i is incremented (step S206).

Then, returning to step S202, the next loop is started. In step S203 of the second and subsequent loops, among the 16 types of division patterns of the i-th n-gram, two division patterns whose division flags other than the last are the same as the pattern selected in step S204 of the previous loop. Send a command to get the probability coefficient.
That is, from the step S202 of the second and subsequent loops, it is determined whether or not the pattern is divided after the last word of the i-th n-gram among the two patterns having the same segmentation flag as the pattern selected up to the previous loop. Determined in step S204.

  On the other hand, if the i-th n-gram is the last n-gram of the word string to be analyzed, it is determined that there is no unprocessed n-gram (step S205; NO). Then, the word string is divided at the part where the delimiter flag in the delimiter pattern selected in step S204 of the previous loop is 1 (step S207), and the menu division process ends.

  Returning to FIG. 6, when the menu division process is completed, the translation unit 50 translates each word included in the divided word string using the translation dictionary (step S105).

  Then, the display unit 60 displays the translation result (step S106), and the menu display process ends.

As described above, according to the probability coefficient output device 40 according to the present embodiment, even when teacher data including the focused n-gram cannot be obtained sufficiently or not, a subsequence of the n-gram The data related to the separation method can be acquired from the above, and the probability coefficient of the separation pattern can be obtained.
That is, assuming that the word string (menu) to be analyzed is also divided by the probability of being divided by a certain division pattern, the probability that the menu will be divided by the n-gram division pattern of the menu is (n -1) It can be estimated from gram-monogram teacher data.
For this reason, the number of necessary teacher data is smaller than the probability of the partition pattern obtained from only the teacher data including the n-gram itself and estimating the menu partition position.

  In this embodiment, the probability coefficient of the n-gram partition pattern is calculated based on the probability coefficient of the n-1 gram to monogram partition pattern with the same partition flag. Therefore, the accuracy of the calculation result is higher than that in the case where the coefficient used for the calculation is extracted using only the word commonality.

  Further, a value of j that can be determined to be able to obtain a reliable probability coefficient for all of (n-1) grams to monograms (j gram), which are n-gram substrings, is determined, and the probability coefficient is calculated from the delimiter pattern of the number of words. To do. Therefore, when calculating the probability coefficient, there is no bias between the information amount of the previous j-gram and the information amount of the subsequent j-gram. Therefore, the probability coefficient of n-gram can be calculated without any bias such that the probability coefficient of either j-gram more strongly affects the calculation result.

Further, according to the probability coefficient output device 40 according to the present embodiment, since the teacher data is generated from a character string of a predetermined category (here, a menu), the teacher data of a wide category (for example, the entire Japanese language) is obtained. The probability coefficient that matches the category can be obtained as compared with the case where the probability coefficient of the segmentation pattern is obtained by using it.
Therefore, when the menu is divided using the menu display device 1 including the probability coefficient output device 40, the accuracy of dividing the menu is high.

In the above description, the probability coefficient of the corresponding pattern of the partial sequence (j-gram) from which the n-gram probability coefficient is extracted is obtained by averaging. However, the method for obtaining the probability coefficient of n-gram is not limited to this.
The probability coefficient of n-gram can be replaced by any calculation formula that increases the probability coefficient of n-gram as at least one of the corresponding patterns of j-gram increases. For example, among the probability coefficients of the corresponding pattern, the weighted addition is performed so that the influence of the probability coefficient of the corresponding pattern located at the earliest is increased, and the expression of averaging the probability coefficient of each corresponding pattern is a power average. be able to.
Further, the probability coefficient of n-gram has a predetermined maximum value (for example, 0.8), and if the calculated value is equal to or greater than the maximum value, the maximum value may be the calculation result.
Further, a table storing the correspondence pattern probability coefficient and the calculated value in association with each other may be stored in the storage unit 470, and the n-gram probability coefficient may be obtained by referring to this table regardless of the calculation formula.

(Embodiment 2)
Next, the menu display device 1 and the probability coefficient output device 40 according to Embodiment 2 of the present invention will be described.

  The menu display device 1 and the probability coefficient output device 40 of the present embodiment have the same configuration as the menu display device 1 and the probability coefficient output device 40 according to the first embodiment (FIGS. 1 and 3).

The probability coefficient output device 40 according to the present embodiment is different from the probability coefficient output device 40 according to the first embodiment in the probability coefficient calculation method.
Here, a calculation method of the probability coefficient of the probability coefficient output device 40 of the present embodiment will be described with reference to FIG.

  A method in which the probability coefficient output device 40 of the present embodiment calculates the trigram segmentation pattern using the probability coefficients of the front bigram and the rear bigram will be described with reference to FIG.

  At this time, in calculating the probability coefficient of the trigram division pattern (here, “0 1 red wine 0 boiled 1”), first, the previous bigram (“0 1 red wine 0”, probability coefficient p1 having a common division flag). = 0.31) is extracted as a corresponding pattern. Then, as corresponding patterns in the rear bigram, two bigrams having a delimiter flag common to the corresponding pattern in the previous bigram “1 red wine 0 boiled 1” (probability coefficient p2 = 0.45) and “1 red wine 0 boiled 0” (Probability coefficient p3 = 0.11) is extracted.

Then, the probability coefficient of the corresponding pattern of the previous bigram is distributed based on the probability coefficient of the corresponding pattern of the subsequent bigram, and the probability coefficient of the trigram is calculated. That is, the probability coefficient of the trigram segmentation pattern is calculated as p1 · (p2 / (p2 + p3)) as the probability pa when “1 red wine 0 boiled 1” comes after “0 red wine 0”. Similarly, the probability pb that “1 red wine 0 boiled 0” comes after “0 red wine 0” can be calculated as p1 · (p3 / (p2 + p3)).
This calculation formula is an arbitrary formula (for example, pa = p1 ^ 2 · (p2) in which the probability coefficient p1 of the corresponding pattern in the previous (n−1) gram is distributed by the probability coefficient of the corresponding pattern in the subsequent (n−1) gram. It can be replaced with {circumflex over (^) / (p2 + p3)} 2).
Here, the probability coefficient p1 of the corresponding pattern in the previous (n-1) gram is sorted by the probability coefficient of the corresponding pattern in the subsequent (n-1) gram, but the processing order may be reversed. The same applies hereinafter.
In addition, a table for storing the probability coefficient of the corresponding pattern of (n-1) grams before and after and the calculated value in association with each other is stored in the storage unit 470. A probability coefficient may be obtained.

  Similarly, when calculating the trigram separation pattern using the probability coefficient of the previous bigram and the subsequent monogram, the probability coefficient of the corresponding pattern of the previous bigram is changed to the probability coefficient of the corresponding pattern of the subsequent monogram. Based on the distribution (FIG. 11B).

  A method for calculating the trigram segmentation pattern using only the monogram probability coefficient will be described with reference to FIG. At this time, first, the probability coefficient p7 of the corresponding pattern in the previous monogram is assigned to the probability coefficients p8 and p9 of the corresponding pattern in the subsequent monogram to obtain the probability coefficient p10 of the previous bigram (c1). Then, using the probability coefficient p10 in the previous bigram obtained in (c1), the trigram probability coefficient is calculated in the same manner as in FIG.

  The calculation process shown in FIG. 11 executed by the probability coefficient output device 40 of this embodiment will be specifically described with reference to flowcharts (FIGS. 12 to 13).

When the user performs an operation for photographing a menu, the menu display device 1 and the probability coefficient output device 40 of the present embodiment perform the menu display processing of FIG. 6 and the menu division processing of FIG. 7 as in the first embodiment. Then, the probability coefficient acquisition process of FIG. 8 is executed.
The probability coefficient output device 40 of the present embodiment executes the probability coefficient calculation process 2 (FIG. 12) in step S305 of the probability coefficient acquisition process (FIG. 8).

  In the probability coefficient calculation process 2, first, the (n-1) gram generation unit 430 generates two (n-1) grams which are partial strings of the target character string (n gram) (here, the previous bigram in FIG. 11A). And the subsequent bigram), and attention is paid to any bigram (here, the bigram) (step S601).

  Next, whether the determination unit 420 can acquire the probability coefficient for the attention bigram (previous bigram) or the bigram stored in the n-gram list storage unit 4710 as in step S302 of the probability coefficient acquisition process (FIG. 8). The number of bigram data corresponding to the list is compared with the number of thresholds for a predetermined bigram (step S602).

  If the number of data of the previous bigram is equal to or larger than the bigram threshold (step S602; YES), it can be determined that a probability coefficient can be acquired for the previous bigram. Is acquired (step S603).

  If the number of data of the previous bigram is smaller than the threshold value (step S602; NO), it can be determined that the probability coefficient cannot be obtained for the previous bigram. Therefore, as shown in (c1) of FIG. Calculate the probability coefficient. That is, it is determined whether or not the current n-1 is 1 (step S604). If it is not 1 (step S603; NO), 1 is subtracted from 1 to 2 (step S605), and the previous bigram is subtracted from the subtracted n. The probability coefficient calculation process 2 is recursively executed as the target subsequence to generate a delimiter pattern, and the probability coefficient is acquired (step S606).

  On the other hand, when n-1 is 1 (step S604; YES), since n cannot be further reduced, a delimiter pattern is generated for a monogram for which a probability coefficient cannot be obtained, and the probability coefficient is set to a default value (here Is 0.5) (step S607).

  Next, it is determined whether or not the processing has been completed for both (n-1) grams before and after (step S608). If the probability coefficient has not been acquired for either (n-1) gram before or after (step S608; NO), the unprocessed (n-1) gram is regarded as the attention (n-1) gram (step S609). The process is repeated from step S602.

  On the other hand, if it is determined that the probability coefficients have been acquired for both (n-1) grams before and after (step S608; YES), a process of calculating probability coefficients for each delimiter pattern (delimiter pattern calculation process 2) is executed. Then (step S609), the probability coefficient calculation process 2 ends.

  The delimiter pattern calculation process 2 executed in step S610 will be described with reference to FIG. In the calculation process 2 for each delimiter pattern, first, among all delimiter patterns that can be generated from the target subsequence (target trigram), a delimiter pattern for which the probability coefficient requested from the menu analysis unit 30 is to be acquired is generated (step S701). .

  Then, paying attention to the k-th delimiter pattern of the generated delimiter pattern using k as a counter variable (step S702). In the examples of FIGS. 11A and 11B, “0 1 red wine 0 boiled 1” is the notable separation pattern. In the example of (c1) in FIG. 11 (c), “0 red wine 0” is the notable separation pattern.

  Next, the (n−1) -gram delimiter pattern (corresponding pattern) corresponding to the delimiter pattern of interest shown in FIG. 11 is generated in step S603, step S606 or step S607 of the probability coefficient calculation process 2 (FIG. 12). (N-1) Gram separation patterns are extracted (step S703). In the case of (c2) in FIG. 11 (a) or (c), “0 1 red wine 0” and “1 red wine 0 boiled 1” are the corresponding patterns. In the case of FIG. 11B, “0 red wine 0” and “0 boiled 1” are the corresponding patterns. Further, in the example of (c1) in FIG. 11C, “1 of 0” and “red wine 0” are the corresponding division patterns.

  Then, the probability coefficient of the target separation pattern is calculated from the extracted probability coefficient of the corresponding pattern by the calculation method shown in FIG. 11 (step S704).

  Next, it is determined whether probability coefficients have been calculated for all the delimiter patterns generated in step S701 (step S705). If there is an unprocessed division pattern (step S705; NO), k is incremented (step S706), and the processing from step S702 is repeated for the next division pattern.

  On the other hand, when the process has been completed for all the delimiter patterns (step S705; YES), the delimiter pattern calculation process 2 is terminated.

As described above, according to the probability coefficient output device 40 according to the present embodiment, when sufficient teacher data including the focused n-gram cannot be obtained, the previous n−1 gram (target n−1 gram) is obtained. The obtained random variable is assigned to the subsequent n-1 gram, and the random variable of the division pattern in the n gram is calculated (or vice versa).
That is, (n-1) the teacher data is divided by the gram sequence, and the division method of a certain division pattern is divided based on the probability of the next division pattern that can be divided by the division method of the division pattern. Since the n-gram probability coefficient is calculated, the probability coefficient can be calculated based on more information. Therefore, the calculation accuracy is high.

  In addition, even when the probability coefficient cannot be obtained for a part of (n-1) grams, the probability coefficient of n grams can be calculated using the probability coefficient of (n-1) grams as long as it can be obtained. (N-2) There is little deterioration in accuracy compared with the case of using a probability coefficient of gram to monogram.

(Modification)
As mentioned above, although embodiment of this invention was described, embodiment of this application is not restricted to this, A various deformation | transformation is possible.
For example, in the first and second embodiments, when the number of n-gram data is equal to or less than a predetermined threshold, the calculation is performed from the probability coefficient of (n-1) gram to monogram, but the embodiment of the present invention is not limited thereto. Absent. For example, in such a case, the probability coefficient of (n-1) gram to monogram is calculated, and the probability coefficient of n gram is obtained from the number of data below a predetermined threshold, and the calculated value and the added value are used as the addition average. It may be a value to be obtained.

In the first and second embodiments, the teacher data is stored in the teacher data storage unit 4730. However, the teacher data may be stored not in the probability coefficient output device but in an external device.
At this time, the probability coefficient output device uses the communication unit 405 to access the external device and acquire teacher data.

  In addition, as a delimiter pattern, a delimiter flag is uniquely defined between all words, and a delimiter pattern corresponding to only a case where the delimiter flags are completely matched is compared. However, it is also possible to define that some delimiter flags are defined as unknowns in the delimiter pattern, and that unknown parts are not taken into account when comparing delimiter flags.

  In the first and second embodiments, when the probability coefficient of the n-gram delimiter pattern is obtained, the appearance probability of the delimiter pattern in the teacher data, which is a tagged character string, is obtained one by one. However, a configuration is also possible in which a probability coefficient output device or an external device stores a pattern probability coefficient list in which the probability coefficients of the segmentation pattern are registered, and the probability coefficient is acquired with reference to this pattern probability coefficient list. An example of such a pattern probability coefficient list will be described with reference to FIG. FIG. 14 is an example of a trigram pattern probability coefficient list in which the probability coefficients corresponding to the trigram and the delimiter flag are registered. For example, if the numerical value 0.02 is registered in the column of the pattern “0010” and the row of “Pig-Rose”, the probability coefficient of the delimiter pattern “0 pork 0 rose 1 0” is 0. .02.

  In the first and second embodiments, the word string to be analyzed is a menu, but the present invention can be applied to word strings of any category other than the menu. The word string to be analyzed according to the present invention is preferably a word string of a category characterized by the fact that the words that appear are limited and the rules for how to separate the words are limited. Examples of word strings in such categories include addresses, medicinal benefits / instructions, etc. in addition to menus.

  In addition, the probability coefficient output devices of the first and second embodiments receive information indicating n words (n-grams) from the menu analysis unit 30 and a delimiter pattern that requires a probability coefficient among the n-gram delimiter patterns. Although the probability coefficient is calculated and output for the division pattern that is transmitted and requires the probability coefficient, the process executed by the probability coefficient output device according to the present invention is not limited to this. For example, the probability coefficient output device may receive only n-grams from an external device, generate all delimiter patterns that can be defined for the n-gram, and calculate and output the probability coefficients of all generated delimiter patterns.

  In addition, a central part that performs processing for the probability coefficient output device including the information processing unit 401, the data storage unit 402, the program storage unit 403, and the like is not a dedicated system, but a normal computer system. It can be realized using. For example, a computer program for executing the above operation is stored and distributed in a computer-readable recording medium (flexible disk, CD-ROM, DVD-ROM, etc.), and the computer program is installed in the computer. Thus, an information terminal that executes the above-described processing may be configured. Alternatively, the computer program may be stored in a storage device included in a server device on a communication network such as the Internet, and the information processing device may be configured by being downloaded by a normal computer system.

  In addition, when the function of the probability coefficient output device is realized by sharing of the OS (operating system) and application program, or in cooperation with the OS and application program, only the application program portion is stored in a recording medium or a storage device. May be.

  It is also possible to superimpose a computer program on a carrier wave and distribute it via a communication network. For example, the computer program may be posted on a bulletin board (BBS: Bulletin Board System) on a communication network, and the computer program may be distributed via the network. The computer program may be started and executed in the same manner as other application programs under the control of the OS, so that the above-described processing may be executed.

  A part of the processing to be executed may be realized by using a computer independent of the probability coefficient output device. .

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to the specific embodiment which concerns, This invention includes the invention described in the claim, and its equivalent range It is. Hereinafter, the invention described in the scope of claims of the present application will be appended.

(Appendix 1)
A word string acquisition unit for acquiring the attention word string;
An extraction unit for extracting a plurality of partial word strings including one or more words included in the attention word string acquired by the word string acquisition unit;
The partial word sequence extracted by the extraction unit with a delimiter pattern corresponding to each delimiter pattern between the case where the word sequence is delimited and the case where the word sequence is not delimited between each word between the words constituting the word sequence A probability coefficient acquisition unit that acquires for each of the extracted delimiter patterns, and acquires a delimiter probability coefficient indicating the probability that the teacher word string including the partial word string is delimited in a delimiter corresponding to the delimiter pattern;
A probability coefficient obtaining unit for obtaining a separation probability coefficient of the separation pattern of the attention word string based on the separation probability coefficient acquired by the probability coefficient acquisition unit;
An information processing apparatus comprising:

(Appendix 2)
The teacher word string is a word string belonging to the same category as the attention word string, and is a word string defining whether or not the word string is divided between words of the word string,
A determination unit that determines whether a sufficient number of word strings including the partial word string extracted by the extraction unit can be acquired as a teacher word string for acquiring the break probability coefficient;
When the extraction unit determines that a sufficient number of word strings including the partial word string extracted by the determination unit cannot be acquired, the partial word string of the extracted partial word string is further extracted. To
The information processing apparatus according to appendix 1, wherein

(Appendix 3)
The probability coefficient acquisition unit, among the delimiter patterns that can be defined for the partial word string, a delimiter pattern that is the same delimiter between words corresponding to the delimiter pattern of the attention word string acquired by the probability coefficient acquiring unit get,
The information processing apparatus according to appendix 1 or 2, characterized in that:

(Appendix 4)
The delimitation probability coefficient of the delimiter pattern of the attention word string obtained by the probability coefficient acquisition unit increases as at least one of the delimitation probability coefficients acquired by the probability coefficient acquisition unit increases.
The information processing apparatus according to any one of supplementary notes 1 to 3, wherein:

(Appendix 5)
The partial word strings extracted by the extraction unit are each composed of the same number of words.
The information processing apparatus according to any one of supplementary notes 1 to 4, wherein

(Appendix 6)
The extraction unit extracts at least a front partial word string that is a partial word string including the first word of the attention word string and a rear partial word string that is a partial word string including the last word;
The probability coefficient acquisition unit uses the delimitation probability coefficient of the delimiter pattern acquired from the target partial word string that is either the previous partial word string or the rear partial word string as the delimiter pattern of the target partial word string For corresponding words, corresponding to the same delimiter, allocating based on the delimiter probability coefficient of the delimiter pattern obtained from the partial word string that is not the attention word string of the previous partial word string or the rear partial word string, Obtaining a delimitation probability coefficient of the delimiter pattern of the attention word string;
The information processing apparatus according to any one of supplementary notes 1 to 3, wherein:

(Appendix 7)
The attention word string and the teacher word string are word strings expressing menus,
The information processing apparatus according to any one of supplementary notes 1 to 6, wherein:

(Appendix 8)
On the computer,
Processing to obtain the word sequence of interest,
A process of extracting a plurality of partial word strings including one or more words included in the acquired attention word string;
For each of the extracted partial word strings, a delimiter pattern corresponding to each of the case where the word string is divided and the case where the word string is not divided between words constituting the word string is obtained. Processing,
A process of obtaining a delimitation probability coefficient indicating the probability that the teacher word string including the partial word string is delimited by the delimiter corresponding to the acquired delimiter pattern for each of the extracted delimiter patterns;
A process for obtaining a delimitation probability coefficient of the delimiter pattern of the attention word string based on the acquired delimitation probability coefficient;
A program characterized by having executed.

  DESCRIPTION OF SYMBOLS 1 ... Menu display apparatus, 10 ... Input part, 20 ... OCR, 30 ... Menu analysis part, 40 ... Probability coefficient output apparatus, 50 ... Translation part, 60 ... Display part, 70 ... Information processing part, 401 ... Information processing part, 402 ... Data storage unit, 403 ... Program storage unit, 405 ... Input / output unit, 405 ... Communication unit, 406 ... Internal bus, 407 ... Control program, 410 ... Word string acquisition unit, 420 ... Determination unit, 430 ... (n- 1) Gram generation unit, 440 ... probability coefficient acquisition unit, 450 ... probability coefficient calculation unit, 460 ... output unit, 470 ... storage unit, 4710 ... n-gram list storage unit, 4720 ... setting storage unit, 4730 ... teacher data storage unit

Claims (6)

A word string acquisition unit for acquiring a word string having a plurality of words;
An extraction unit for extracting a plurality of partial word strings including one or more words included in the word string acquired by the word string acquisition unit;
The partial word sequence extracted by the extraction unit with a delimiter pattern corresponding to each delimiter pattern between the case where the word sequence is delimited and the case where the word sequence is not delimited between each word between the words constituting the word sequence A word string belonging to the same category as the word string for each of the extracted division patterns is obtained for each of the division probability coefficients indicating the degree of probability that the partial word string is divided by the division method corresponding to the division pattern. A teacher word string having the same pattern as the delimiter pattern from a teacher word string storage unit storing a teacher word string defining whether or not the word string is delimited between words of the word string Based on the number of teacher word strings extracted and extracted, a probability coefficient acquisition unit to acquire,
A dividing unit that divides the word string acquired by the word string acquisition unit based on the division probability coefficient acquired by the probability coefficient acquisition unit;
An information processing apparatus comprising: A discriminating unit for discriminating whether or not there is a sufficient number of teacher word strings in the teacher word string storage unit to obtain the segmentation probability coefficient for the partial word string extracted by the extraction unit;
The extraction unit further extracts a partial word string of the extracted partial word string when it is determined that there is not a sufficient number of teacher word strings for the determination unit to acquire the break probability coefficient,
The information processing apparatus according to claim 1 . When the number of teacher word strings stored in the teacher word string storage unit is not sufficient, and the number of words in the extracted partial word string is 1, the probability coefficient acquisition unit calculates the segmentation probability Set the specified value for the coefficient,
The information processing apparatus according to claim 2 . The partial word strings extracted by the extraction unit are each composed of the same number of words.
The information processing apparatus according to any one of claims 1 to 3, characterized in that. The word string and the teacher word string are word strings expressing a menu,
The information processing apparatus according to claim 1 , wherein the information processing apparatus is an information processing apparatus.   Computer
  A word string acquisition unit for acquiring a word string having a plurality of words;
  An extraction unit for extracting a plurality of partial word strings including one or more words included in the word string acquired by the word string acquisition unit;
  The partial word sequence extracted by the extraction unit with a delimiter pattern corresponding to each delimiter pattern between the case where the word sequence is delimited and the case where the word sequence is not delimited between each word between the words constituting the word sequence A word string belonging to the same category as the word string for each of the extracted division patterns is obtained for each of the division probability coefficients indicating the degree of probability that the partial word string is divided by the division method corresponding to the division pattern. A teacher word string having the same pattern as the delimiter pattern from a teacher word string storage unit storing a teacher word string defining whether or not the word string is delimited between words of the word string Based on the number of teacher word strings extracted and extracted, a probability coefficient acquisition unit to acquire,
  A dividing unit that divides the word string acquired by the word string acquisition unit based on the division probability coefficient acquired by the probability coefficient acquisition unit;
  Program to function as.

Images