JP6056291B2 - Information processing apparatus, data display apparatus, and program - Google Patents

Description

  The present invention relates to an information processing device, a data display device, and a program.

  A translation device that translates a word string (original text) including a plurality of words into another language is known. When translating an original sentence, such a translation apparatus performs translation processing by estimating which word (word-to-word) the original sentence is divided.

  In relation to a method for estimating how to divide a document or a word string, Patent Document 1 proposes a technique for inferring how to divide a document using a parser that has been programmed in advance with the grammar rules of the language to which the document belongs. . Patent Document 2 proposes a technique for dividing a character string that is not divided into words.

JP-A-6-309310 JP-A-10-254874

  In the technique of Patent Document 1, in order to infer which words are separated from each other in the original text, a syntax analyzer that is programmed with the grammar rules of the language to which the original text belongs is used. Therefore, it was necessary to create a parser for each language to which the original text belongs, spending a lot of development costs and days.

  Patent Document 2 discloses a technique of dividing a character string that is not divided into words, but discloses a method for determining which word and word are separated from each other. Absent.

  In an application that divides a character string into words and presents information on each word, a syntax analyzer for the language in which the character string to be analyzed is written is required.

  However, it is difficult to construct a parser by learning only from the teacher data of the analysis result, and it is necessary to design a parser corresponding to each language. Therefore, without using a syntax analyzer, the connection probability between words is calculated from the teacher data, and the character string is analyzed based on the probability.

  The word n-gram is used to calculate the connection probability. However, since the teacher data includes n-grams (for example, n-grams composed only of particles and adverbs) due to a series of words that are not semantically related, the analysis accuracy is reduced by the n-grams. There are also problems.

The present invention has been made in view of the above circumstances, and an object thereof is to perform word division of a character string without using a syntax analyzer corresponding to each language.
Another object of the present invention is to provide an information processing apparatus, a data display apparatus, and a program that improve analysis accuracy when a character string is divided into words.

In order to achieve the above object, an information processing apparatus according to the present invention converts a character string to be analyzed into a word string and sequentially extracts a partial string including words included in the word string; For each partial sequence extracted by the partial sequence extraction unit, a delimiter pattern generation unit that generates all patterns indicating whether or not the word sequence is delimited between words, and a teacher for each delimiter pattern generated by the delimiter pattern generation unit A probability coefficient acquisition unit that acquires a division probability coefficient based on a division method in data and performs weighting based on an attribute of a word included in the partial sequence, a division probability coefficient acquired by the probability coefficient acquisition unit, and the division probability Le for multiplying the person division probability coefficients of common neighboring division pattern of the previous subsequence adjacent to delimiting pattern of subsequence corresponding to large coefficient to calculate the route probability factor And a route selection unit that selects a route so as to follow a partial sequence having a large route probability coefficient calculated by the route probability coefficient calculation unit, and a delimiter pattern in the route selected by the route selection unit. And a word string dividing unit for dividing the string.

  According to the present invention, it is possible to improve accuracy by performing analysis while paying attention to words effective for character string division.

(A) is a block diagram which shows the function structure of the menu display apparatus which concerns on embodiment, (b) is a block diagram which shows the physical structure of the menu display apparatus which concerns on embodiment. (A) is a figure which shows the picked-up image which the menu display apparatus which concerns on embodiment acquires, (b) is a figure which shows the result of having divided the word string by the menu display apparatus which concerns on embodiment, (c) is It is a figure which shows the display data which attached | subjected the comment to the word with the menu display apparatus which concerns on embodiment. (A) is a figure which shows the relationship between the character string in the teacher data of the menu display apparatus which concerns on embodiment, and a character string with a tag, (b) is a word string and division | segmentation in the teacher data of the menu display apparatus which concerns on embodiment. It is a figure which shows the relationship between a flag, n-gram (trigram), and a division | segmentation pattern. It is a block diagram which shows the function structure of the character string analysis part which concerns on embodiment. It is a figure explaining the process which the character string analysis part which concerns on embodiment performs. (A) is a figure which shows the table | surface explaining the correction process which the character string analysis part which concerns on embodiment performs, (b) shows the graph explaining the correction process which the character string analysis part which concerns on embodiment performs. FIG. It is a block diagram which shows the structure of the probability process part which concerns on embodiment. It is a figure which shows the probability coefficient list | wrist which concerns on embodiment. (A) is a figure which shows the example of the process which calculates the probability coefficient of a trigram from the probability coefficient of the bigram in the probability coefficient calculation process which concerns on embodiment, (b) is a figure of the monogram in the probability coefficient calculation process which concerns on embodiment. It is a figure which shows the example of the process which calculates the probability coefficient of a trigram from a probability coefficient. It is a flowchart which shows the menu display process which the menu display apparatus which concerns on embodiment performs. It is a flowchart which shows the menu division | segmentation process which the menu display apparatus which concerns on embodiment performs. It is a flowchart which shows the division | segmentation probability coefficient acquisition process which the menu display apparatus which concerns on embodiment performs. It is a flowchart which shows the division | segmentation probability coefficient calculation process which the menu display apparatus which concerns on embodiment performs. It is a flowchart which shows the route probability coefficient calculation process which the menu display apparatus which concerns on embodiment performs. (A) is a figure which shows the example of the teacher data which concerns on embodiment, (b) is a figure which shows the trigram list when attaching a part of speech to the word in the teacher data which concerns on embodiment. It is a figure which shows the probability coefficient list | wrist which weighted based on the part of speech attached to the word in the teacher data which concerns on embodiment.

  Hereinafter, an information processing apparatus, a data display apparatus, and a menu display apparatus according to an embodiment for implementing a program will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals.

  The menu display device 1 according to the embodiment is intended for analysis of menus such as dishes, but the present invention can also be applied to other categories such as addresses, medicinal effects, instructions, etc. .

  The menu display device 1 i) a photographing function for photographing paper or the like describing a character string (menu, menu, etc.) belonging to a specific category to be analyzed; ii) recognizing a character string to be analyzed from the photographed image Iii) a function of analyzing the extracted character string and converting it into a word string, iv) a function of outputting a probability coefficient indicating a probability that the menu is divided at a predetermined part (between words) of the character string, v ) A function of dividing a word string based on a probability coefficient, vi) a function of converting the divided word string into display data, vii) a function of displaying display data, and the like.

  As shown in FIG. 1A, the menu display device 1 includes an image input unit 10, an OCR 20, a character string analysis unit 30, a probability processing unit 40, a conversion unit 50, and a term dictionary storage unit 60. And a display unit 80 and an operation input unit 90.

  The image input unit 10 includes an image capturing device such as a camera and an image processing unit, and acquires an image obtained by capturing a menu with such a physical configuration. The image input unit 10 transmits the acquired image to the OCR 20 of the information processing unit 70.

  Note that it is possible to input other than images. For example, the user may input a menu character string from a keyboard, or may input a menu character string by voice.

  As shown in FIG. 1B, the information processing unit 70 has a function corresponding to a computer. Physically, the information processing unit 701, a data storage unit 702, a program storage unit 703, and an input / output unit A unit 704, a communication unit 705, and an internal bus 706.

  The information processing unit 701 includes a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and the like, and executes processing in the menu display device 1 described later according to a control program 707 stored in the program storage unit 703.

  The data storage unit 702 includes a RAM (Random Access Memory) and the like, and is used as a work area for the information processing unit 701.

  The program storage unit 703 includes a non-volatile memory such as a flash memory or a hard disk, and stores a control program 707 for controlling the operation of the information processing unit 701 and data for executing the following processing.

  The input / output unit 704 controls input / output of information to / from the image input unit 10, the display unit 80, the operation input unit 90, an external device, and the like connected to the information processing unit 70 via a USB (Universal Serial Bus) or a serial port. It is an I / O (Input / Output) interface unit.

  The communication unit 705 includes a local area network (LAN) device, a modem, and the like, and transmits the processing result of the information processing unit 701 to an external device connected via a communication line such as the Internet or a LAN. In addition, information is received from an external device and transmitted to the information processing unit 701.

  Note that the information processing unit 701, the data storage unit 702, the program storage unit 703, the input / output unit 704, and the communication unit 705 are respectively connected by an internal bus 706, and can transmit and receive information.

  The central part for performing the processing of the information processing unit 70 can be realized by using a normal computer system without depending on a dedicated system. For example, a computer program for executing processing may be stored in a computer-readable recording medium and distributed, and the computer program may be installed in the computer. The recording medium includes a flexible disk, a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), and the like.

  Alternatively, the computer program may be stored in a storage device included in a server device on a communication network such as the Internet and downloaded by a normal computer system.

  Further, when the function is realized by sharing of an OS (Operating System) and an application program, or by cooperation between the OS and the application program, only the application program part is stored in a recording medium or a storage device, and is controlled by the OS. The process may be executed by

  It is also possible to superimpose a computer program on a carrier wave and distribute it via a communication network. For example, a computer program may be posted on a bulletin board (BBS: Bulletin Board System) on a communication network and distributed via the network.

  Further, part of the processing executed by the menu display device 1 may be realized using a computer independent of the menu display device 1.

  The information processing unit 70 functions as the OCR 20, the character string analysis unit 30, the probability processing unit 40, the conversion unit 50, and the term dictionary storage unit 60 with the above physical configuration.

  The OCR 20 is an optical character reader that recognizes characters in an image transmitted from the image input unit 10 and acquires a character string (such as a dish name) written in a menu. The OCR 20 transmits the acquired character string to the character string analysis unit 30.

  When an input means other than an image is used, a means for recognizing characters is provided accordingly. For example, in the case of voice input, a character string is acquired by a voice recognition device.

  The character string analysis unit 30 converts the character string transmitted from the OCR 20 into a word string W. The word string W is a character string divided into words. For example, when the character string “boiled pork belly in red wine” is converted into the word string W, “pork, rose meat, red wine, boiled” is obtained.

  The character string analysis unit 30 extracts a partial sequence (word n-gram) composed of words constituting the word sequence W. Note that n represents the number of words to be extracted. In order to determine whether or not a word is separated between words in the word string W, an n-gram technique is used.

  The character string analysis unit 30 defines a delimiter pattern for the extracted word n-gram and transmits it to the probability processing unit 40. Note that the delimiter pattern includes a word n-gram and information corresponding to the case where the word string W is delimited and not delimited between the words.

  The character string analysis unit 30 receives the break probability coefficient output by the probability processing unit 40. The delimitation probability coefficient indicates a coefficient indicating the probability that teacher data including the word n-gram will be delimited by the delimiter pattern. The word n-gram, the division pattern, and the division probability coefficient will be described later.

  The character string analysis unit 30 divides the word string W using the delimitation probability coefficient received from the probability processing unit 40 and outputs the partial character string to the conversion unit 50. Specific processing executed by the character string analysis unit 30 will be described later.

  The probability processing unit 40 receives a delimiter pattern including a delimiter flag indicating a word n-gram and its delimiter from the character string analyzing unit 30. The probability processing unit 40 stores information indicating the probability that the teacher data is partitioned by the partition pattern.

  When the delimiter pattern is transmitted from the character string analysis unit 30, the probability processing unit 40 refers to the stored probability coefficient list, acquires the delimitation probability coefficient, and transmits it to the character string analysis unit 30. The functional configuration of the probability processing unit 40 and the specific processing to be executed will be described later.

  The conversion unit 50 refers to the term dictionary storage unit 60 and converts the divided partial character string transmitted from the character string analysis unit 30 into display data for each portion.

  Specifically, the conversion unit 50 transmits a word included in the partial character string to the term dictionary storage unit 60 and acquires comment data of the word from the term dictionary storage unit 60. The conversion unit 50 generates display data by arranging the word of the menu that is the original sentence and the explanation data of the word for each part. The conversion unit 50 transmits the generated display data to the display unit 80.

  Note that the explanation data is not limited to the meaning of the word, and various contents can be prepared depending on the application, such as a translation of the word into another language.

  The term dictionary storage unit 60 stores a term dictionary in which words and the like included in teacher data are registered in association with data for explaining them.

  When a word or the like is transmitted from the conversion unit 50, the term dictionary storage unit 60 transmits comment data associated with the word or the like in the stored term dictionary to the conversion unit 50. If the word or the like is not in the term dictionary, empty data indicating that is transmitted.

  The display unit 80 is composed of a liquid crystal display or the like, and displays information transmitted from the conversion unit 50. In addition, a means for reflecting the output result such as a printer is included.

  The operation input unit 90 includes an operation reception device that accepts user operations, such as a touch panel, a keyboard, a button, and a pointing device, and a transmission unit that transmits information on operations received by the operation reception device to the information processing unit 70. The The operation input unit 90 receives a user operation with such a physical configuration and transmits the information to the information processing unit 70.

  Here, the relationship between the image obtained by photographing the menu by the menu display device 1, the partial character string obtained by dividing the menu, and the display data obtained by converting the partial character string will be described with reference to FIG.

  When the user uses the image input unit 10 to photograph a restaurant menu or the like, the menu display device 1 acquires an image I1 as shown in FIG.

  The character string of the menu is extracted from the image I1 by the OCR 20, and the character string analyzing unit 30 divides the character string into words and then divides them into words to be divided into partial character strings as shown in FIG. Is transmitted to the conversion unit 50.

  As shown in FIG. 2 (c), the conversion unit 50 converts the word included in the menu, which is the original sentence, into display data with the meaning added as an explanatory sentence for each partial character string and displays it.

  Here, a character string to be analyzed according to the present embodiment, a tagged character string as teacher data, a word n-gram, a delimiter flag, and a delimiter pattern will be described with reference to FIG.

  The character string to be analyzed in the present embodiment is a character string indicating a menu or the like as shown in FIG. Here, the character string is “pig rose meat boiled in red wine with warm vegetables”.

  In addition, the teacher data is a tagged character string sandwiched between tags for each word or a strongly connected group, as shown in the lower part of FIG. A large amount of teacher data is created in advance in the data storage unit 702, and each time a character string is analyzed, it is accumulated and learned.

  Since the minimum unit of a tagged character string is a word, after making the character string into a word unit, add tags such as <m>, <c>, <s>, <w> at the beginning of the chunk, and </ M>, </ c>, </ s>, </ w>, etc.

  In the example of Fig. 3 (a), the tagged string is "<m> <c> <s> <w> pig </ w> <w> rose </ w> <w> </ w> < / s> <s> <w> red wine </ w> <w> boiled </ w> </ s> </ c> <s> <w> warm vegetables </ w> <w> with </ w> </ s> </ m> ".

  In this tagged string, the words <w> </ w> indicate the words "pork", "rose meat", "no", "red wine", "boiled", "warm vegetables", Divided into seven words. In addition, tags <s> </ s> indicating the mass of ingredients, cooking methods, etc., can be grouped into three categories: “pig rose meat”, “boiled in red wine”, and “with warm vegetables”.

  In addition, the tags <c> <c /> indicating the lump of the dish name and the like are used to collect “boiled pork belly in red wine” and are distinguished from “with warm vegetables”. That is, it is divided into cases where the cooking name includes other modifiers such as “Provence style” and “Specialties”. Furthermore, a character string is divided for each menu by tags <m> </ m> indicating menus.

  Here, in the tagged character string, chunks are specified by tags <w>, <s>, <c>, <m>, but the definition format is not limited to this. For example, an arbitrary character string that includes a unique mark (a single-byte space or the like) that divides a character string included in a predetermined category into words, and a unique mark that is separated by at least one separation method other than words You may decide how to delimit.

  The tagged character string is data that is manually tagged by collecting character strings belonging to a specific category (here, menu or dish name) of a specific language (here, Japanese). The method of tagging the tagged character string is not limited to manual operation, and any known tagging method such as a syntax analyzer may be used.

  Next, FIG. 3B shows the relationship among the word string, word n-gram, delimiter flag, and delimiter pattern. An n-gram sequence is obtained by extracting a partial sequence including n words from the word sequence W.

  For example, if the number of words is seven, (first word, second word,..., Nth word), (second word, third word,..., N + 1th word),. (8−nth word,..., 6th word, 7th word), and a substring of the number of words−n + 1 can be extracted.

  An n-gram sequence is a word n-gram, in which words are selected in order from the top of the word sequence W, and extracted from the selected words so that n words are included in order. The word n-gram of n = 3 is called a trigram, the word n-gram of n = 2 is called a bigram, and the word n-gram of n = 1 is called a monogram (unigram).

  As shown in FIG. 3 (b), from "Pig roses with warm wine cooked in red wine", Trigrams "Pig, rose meat, no", "Rose meat, red wine", ..., "Boiled, warm vegetables , Append ”can be obtained.

  The word string W of the menu can be expressed in a tree shape by a tag structure as shown in FIG. By drawing a delimiter line at a predetermined height of the tree, it is possible to determine which word is delimited and how to delimit. This corresponds to which tag of the tagged character string is divided.

  In the example of the tree in FIG. 3B, the words are grouped below the dividing line, the part having the tag <m> or </ m>, the part having the tag <c> or </ c>, the tag < The menu is separated at the part where s> or </ s> is present. For each word in the word string W, information indicated by 1 when delimited and 0 when not delimited is called a delimiter flag.

  It should be noted that the criteria for determining which tag is to be delimited by a certain part can be freely set. For example, it is possible to make an arbitrary setting such as determining that a <s> </ s> tag is to be separated only by a portion and placing a separation flag.

  For a word n-gram, a pattern that defines whether or not the word string W is separated between each word is defined by arranging a word and a separation flag side by side.

  For example, for the three words (word A, word B, and word C) that make up the trigram, the delimiter pattern corresponding to the case where the trigram is not delimited between any words including the front of word A and the back of word C is “ “0A0B0C0”, the delimiter pattern corresponding to the delimiter between all words is “1A1B1C1”.

  When the number of teacher data divided by a certain delimiter pattern is m among all the teacher data (M items) composed of a word n-gram, the coefficient calculated by m / M is a delimiter probability coefficient Define as The delimitation probability coefficient indicates the probability of being delimited by the delimiter pattern in the teacher data.

  If a sufficient number of tagged character strings to be used as teacher data are prepared without bias, the delimitation probability coefficient indicates the probability of being delimited by the delimiter corresponding to the delimiter pattern when the menu includes word n-grams. It becomes an indicator.

  In addition, a symbol (SOS: Start of sentence) indicating the beginning and a symbol (EOS: End of sentence) indicating the end are arranged as dummy words at the beginning of the word string W, respectively. May be.

  Next, the functional configuration of the character string analysis unit 30 will be described with reference to FIG. As shown in FIG. 4, the character string analysis unit 30 includes a character string acquisition unit 310, a segmentation unit 320, an n-gram sequence generation unit (partial sequence extraction unit) 330, an n-gram selection unit 340, a delimiter pattern generation unit 350, a route A probability coefficient defining unit 360, a route selecting unit 370, a word string dividing unit 380, and an output unit 390 are included.

  The character string acquisition unit 310 receives the character string to be analyzed extracted by the OCR 20 and transmits it to the segmentation unit 320.

  The segmentation unit 320 executes segmentation processing that divides the character string acquired by the character string acquisition unit 310 into words. For the separation processing, any method for extracting a word from a character string can be used. Here, the method exemplified in Patent Document 2 is used.

  If the menu to be analyzed is a language such as English or French that is separated by a space for each word, the dividing process is performed by dividing the word by a space.

  The segmentation unit 320 converts the character string of the menu into a word string W by the segmentation process and transmits it to the n-gram sequence generation unit 330.

  When the word string W of the menu is transmitted from the segmentation unit 320, the n-gram string generation unit 330 extracts the word n-gram from the word string W and transmits the n-gram string that is the set to the n-gram selection unit 340. .

  The word n-gram selects words in order from the top of the word string W, and extracts a partial string so that n words are included in order from the selected word. The n-gram sequence is composed of partial sequences consisting of the same number of words.

  The n-gram selection unit 340 sequentially selects the word n-gram of the generated n-gram sequence from the head (or the tail) to the tail (or the head) as the attention n-gram, and delimits the selected attention n-gram. This is transmitted to the pattern generation unit 350.

  When the notice n-gram is transmitted, the delimiter pattern generation unit 350 generates a delimiter pattern that can be defined for the notice n-gram. There are 2 ^ (n + 1) division patterns in total, and there are 8 division patterns in the case of bigrams.

  The delimiter pattern generation unit 350 transmits the generated delimiter pattern to the route probability coefficient definition unit 360. A delimiter pattern generated from a word n-gram is called a corresponding delimiter pattern of the word n-gram.

  The route probability coefficient definition unit 360 transmits the transmitted corresponding partition pattern of the target n-gram to the probability processing unit 40 and receives the partition probability coefficient of the corresponding partition pattern from the probability processing unit 40. A route probability coefficient described later is calculated from the received break probability coefficient.

  The route probability coefficient definition unit 360 includes a probability coefficient acquisition unit 3610, a correction unit 3630, and a route probability coefficient calculation unit 3640 in order to realize such a function.

  The probability coefficient acquisition unit 3610 transmits all the division patterns generated by the division pattern generation unit 350 to the probability processing unit 40, and acquires all the division probability coefficients of each division pattern based on the division method statistics in the teacher data. . If it is necessary to adjust the obtained break probability coefficient, the correction unit 3630 performs correction processing.

  In calculating the route probability coefficient, the correction unit 3630 adjusts the value of the partition probability coefficient in each partition pattern based on some criterion. The reference includes the contents of the word n-gram, the state of the break flag, the size of the break probability coefficient, and the adjustment method includes squaring the break probability coefficient or taking the square root. The correction process executed by the correction unit 3630 will be described later.

  When the route probability coefficient calculation unit 3640 calculates the route probability coefficient of each partition pattern, the route probability coefficient calculation unit 3640 associates each partition pattern with the calculated route probability coefficient and transmits the result to the route selection unit 370.

  The route selection unit 370 obtains the maximum likelihood route from each division pattern transmitted from the route probability coefficient definition unit 360 and its route probability coefficient, and the division pattern that appears in the maximum likelihood route from the division pattern generated by the division pattern generation unit 350 Select. The route selection unit 370 transmits the selected division pattern to the word string division unit 380. The maximum likelihood route will be described later.

  The word string dividing unit 380 divides the word string W into partial character strings according to the dividing pattern transmitted from the route selecting unit 370, and transmits the partial character strings to the output unit 390.

  The output unit 390 outputs the transmitted partial character string to the conversion unit 50.

  Here, the route probability coefficient defined by the route probability coefficient defining unit 360 will be described with reference to FIG. The word n-gram extracted from the word string W is denoted as n-gram 1, n-gram 2,. Hereinafter, the bigram will be described as n = 2.

  When the word string W extracts bigram from “pork, rose meat, red wine, boiled, warm vegetables, with sauce”, n-gram 1 is “pork, rose meat”, n-gram 2 is “rose meat,” n Gram 3 is “no, red wine”, n-gram 4 is “red wine, boiled”, n-gram 5 is “boiled, warm vegetable”, and n-gram 6 is “warm vegetable, served”.

  Since n-gram 1 and n-gram 2 are adjacent word n-grams, they are called adjacent n-grams (adjacent subsequences). As shown in the first column of FIG. 5, eight patterns can be defined as the corresponding division patterns of n-gram 1.

For each corresponding partition pattern of n-gram 1, the partition probability coefficient acquired by the probability coefficient acquisition unit 3610 is pa _1y (y = 1 to 8 for eight patterns). The correction unit 3630 corrects pa _{11 to} pa ₁₈ and calculates a route probability coefficient using the corrected segment probability coefficient. In addition, let the division | segmentation probability coefficient after correction | amendment be _p1y (y = 1-8).

Next, as shown in the second column of FIG. 5, eight corresponding division patterns of n-gram 2 can be defined. Similarly, the partition probability coefficient of n-gram 2 is pa _2y (y = 1 to 8), and the corrected partition probability coefficient is p _2y (y = 1 to 8).

  In the corresponding partition pattern of n-gram 2 and the corresponding partition pattern of n-gram 1, a partition pattern having a common partition flag between corresponding words is set as a common adjacent partition pattern, and attention is paid thereto.

  That is, since n-gram 1 and n-gram 2 are shifted by one word as a partial sequence, from the second word to the last word of n-gram 1 and from the first word of n-gram 2 to the last one. Since the word up to the previous word is the same, the one having the same delimiter flag between words corresponding to the part is targeted.

  For example, since n-gram 1 and n-gram 2 have “rose meat” overlapping, the common adjacent delimiter pattern of n-gram 1 corresponding delimiter pattern “1 pig 0 rose meat 0” is the n-gram 2 corresponding delimiter pattern. Of these, “0 rose meat 0 0” and “0 rose meat 0 0” having “0 rose meat 0”.

  Further, as the common adjacent delimiter pattern of the corresponding delimiter pattern of n-gram 3, two common delimiter flags between corresponding words among the corresponding delimiter patterns of n-gram 2 can be cited.

  For example, the common adjacent delimiter pattern of “0 0 red wine 0” of the corresponding delimiter pattern of n-gram 3 is “0 0 of meat 0” having “0 of 0” among the corresponding delimiter patterns of n-gram 2. 1 rose meat 0 ”.

  The route probability coefficient takes into account the influence of the larger delimitation probability coefficient among the common adjacent delimitation patterns of n-gram (x−1) to the y-th delimitation probability coefficient of the corresponding delimitation patterns of n-gram x. Note that the segmentation probability coefficient is a corrected one, and the root probability coefficient is used after the calculation of the root probability coefficient. In n-gram 1, the segmentation probability coefficient after correction is set as the root probability coefficient.

For the root probability coefficient pp _2y of the corresponding partition pattern (y-th) of n-gram 2, the corrected partition probability coefficient p _2y and the corrected partition probability coefficient of the common adjacent partition patterns of n-gram 1 are large. It is the product of the delimiter probability coefficient after correction of the common adjacent delimiter pattern.

  Regarding the root probability coefficient of the corresponding partition pattern of n-gram 3, the corrected partition probability coefficient and the root probability coefficient of the common adjacent partition pattern having the larger root probability coefficient among the common adjacent partition patterns of n-gram 2 The product of At this time, the one selected in the n-gram 2 is the corresponding adjacent delimiter pattern, which is indicated by the solid line arrow in FIG.

  As described above, the route probability coefficient is calculated using the table in which the common adjacent delimiter patterns are associated with each other for the delimiter patterns of adjacent n-grams as shown in FIG. This table is called a route probability table.

Assume that the root probability coefficient of n-gram 1 is equal to the corrected partition probability coefficient, and the corresponding partition pattern (y-th) root probability coefficient pp _xy of n-gram x (2 and later) is the corrected partition probability coefficient p _xy. And the root probability coefficient of the corresponding adjacent delimiter pattern. The correction process will be described later.

  The route probability coefficient of the corresponding delimiter pattern calculated using the corrected probability coefficient is the most probable route (maximum likelihood route) from the first word n-gram (n-gram 1) to the last word n-gram (n-gram 6). ) Is a coefficient indicating the probability that the word string W will be delimited by the delimiter pattern that appears in the route.

  That is, it is shown that the highest likelihood route can be traced by selecting the route probability coefficient that is higher, and the probability that the word string W can be delimited by the delimiter pattern at that time is the highest.

  This route probability coefficient is a coefficient obtained by an arbitrary expression / calculation method that is an increasing function at least in a predetermined range with respect to each of the division probability coefficients after correction of the division pattern appearing in the maximum likelihood route. Also good.

  For example, it may be an average of the division probability coefficients after correction of the division pattern appearing in the maximum likelihood route. Further, a correspondence relationship between the corrected division probability coefficient and the root probability coefficient is obtained by experiments, a table storing the correspondence relationship is stored in advance in the program storage unit 703, and the root probability coefficient is determined by referring to this table. You may ask.

In this way, the root probability coefficient is calculated for each delimiter pattern of the next adjacent n-gram, and the root probability coefficient pp _{6y of} the last word n-gram (n-gram 6 in the example of FIG. 5) is calculated.

The route probability coefficient pp _6y is a coefficient indicating the likelihood of the most probable route delimitation among all the routes reaching the delimiter pattern from n-gram 1 to n-gram 6.

Therefore, the corresponding delimiter pattern having the largest value among the route probability coefficients pp _6y (y = 1 to 8) is selected. In the example of FIG. 5, “1 warm vegetable 0 with 1” (y = 4) is selected. The route that leads to the selected corresponding delimiter pattern is identified and set as the maximum likelihood route of the word string W.

  For the maximum likelihood route, a pattern having a larger root probability coefficient is selected from the adjacent correspondence division patterns of the selected division pattern, and the first word n-gram (n-gram 1) is selected from the last word n-gram (n-gram 6). ). A delimiter pattern that appears in the maximum likelihood route is selected as a delimiter pattern that delimits the word string W, and the word string W is delimited according to the delimiter pattern.

The correction process executed by the correction unit 3630 shown in FIG. 4 will be described with reference to FIG. For a certain n-gram x (here, bigram), the probability coefficient pa _xy (y = 1 to 8) before correction of the partition pattern (eight patterns in the case of bigram) corresponding to the partition flag, and the probability coefficient p after correction An example of _xy (y = 1 to 8) is shown in the table of FIG. In the present embodiment, a numerical value obtained by squaring the division probability coefficient before correction is set as a division probability coefficient after correction.

  The delimiter probability coefficient after correction in FIG. 6A is a numerical value obtained by squaring the delimiter probability coefficient before correction and normalized for comparison with that before correction. For example, the pattern “010” indicates a delimiter flag whose delimiter pattern is “0 pork 1 rose meat 0” if the bigram is “pork-rose meat”.

  FIG. 6B shows a graph in which probability coefficients before correction and probability coefficients after correction (with normalization) are arranged for each pattern. The black bars and the corrected probability coefficients shown in practice are emphasized more or less than the uncorrected probability coefficients indicated by white bars and dotted lines. The corrected probability coefficient has a larger contrast than the uncorrected probability coefficient, and therefore has a greater effect on the root probability coefficient.

This will be specifically described. The route finally selected is selected according to the relative size of the product of the division probability coefficients after correction of the division pattern appearing in each route (final route probability coefficients pp _{61 to} pp ₆₈ ).

If the values of a certain column x (p _{x1 to} p _x8 ) are all squared and the other columns are the same, the final route probability coefficient (pp _{61 to} pp ₆₈ ) has a value of that column x ( The size of p _{x1 to} p _x8 ) appears more emphasized. Therefore, if the division probability coefficient of the column x is squared, the influence of the value of the column x on the selection result can be expanded.

  When the root probability coefficient is obtained by multiplying the segmented probability coefficient after correction and the selected root probability coefficient as in this embodiment, the magnitude of the final root probability coefficient is the same even if the value of each column is multiplied by an arbitrary value. It does not change.

  Next, the configuration of the probability processing unit 40 will be described with reference to FIG. As shown in FIG. 7, the probability processing unit 40 includes a delimiter pattern acquisition unit 410, a determination unit 420, an (n-1) gram generation unit 430, a probability coefficient acquisition unit 440, a probability coefficient calculation unit 450, and a probability coefficient output unit 460. And a storage unit 470.

  The delimiter pattern acquisition unit 410 acquires a delimiter pattern that needs to acquire a delimiter probability coefficient from the character string analysis unit 30. Here, the delimiter pattern is composed of information indicating a word n-gram and a delimiter flag between words.

  In this case, it is assumed that the whole delimiter pattern generated by the delimiter pattern generation unit 350 is acquired for the word n-gram selected by the n-gram selection unit 340. The delimiter pattern acquisition unit 410 transmits the acquired delimiter pattern to the determination unit 420.

  The determination unit 420 determines whether or not a delimitation probability coefficient indicating the probability that the menu is delimited according to the delimiter pattern transmitted from the delimiter pattern acquisition unit 410 can be acquired.

  The determination unit 420 refers to the probability coefficient list stored in the probability coefficient storage unit 4710 of the storage unit 470 in the determination process. Specific contents of the determination process executed by the probability coefficient list and determination unit 420 will be described later.

  If the determination unit 420 determines that the partition probability coefficient of the partition pattern can be acquired, the determination unit 420 transmits the partition pattern to the probability coefficient acquisition unit 440. On the other hand, if it is determined that the delimiter probability coefficient of the delimiter pattern cannot be acquired, the delimiter pattern is transmitted to the (n-1) gram generation unit 430.

  (N−1) When the gram generation unit 430 receives the delimiter pattern from the determination unit 420, the gram generation unit 430 includes a portion (n−1) composed of words from the first word constituting the delimiter pattern to (n−1) th ) And a part (n-1) gram composed of the second word to the nth word.

  The (n-1) gram generation unit 430 sets a delimiter pattern for the two generated (n-1) grams based on the delimiter pattern of the word n-gram, and transmits the delimiter pattern to the determination unit 420.

  When the delimiter 420 receives the delimiter pattern of two parts (n-1) grams from the (n-1) gram generator 430, the determinator 420 obtains delimiter probability coefficients for each of the two parts (n-1) grams. Determine if you can.

  If the break probability coefficient cannot be obtained by either one of the two part (n-1) grams, the (n-1) gram generation unit 430 generates three parts (n-2) grams, and then becomes a monogram. The discrimination process is repeated in the same manner.

  Each division pattern of the partial j-gram (j = (n−1) to 1) at the time when any division probability coefficient can be acquired is transmitted to the probability coefficient acquisition unit 440. Note that the number of partial j-grams that are generated every time the (n-1) gram generation unit 430 is routed increases by one. Specific contents of processing executed by the determination unit 420 and the (n-1) gram generation unit 430 will be described later.

  Probability coefficient acquisition unit 440 receives the delimiter pattern of the word n-gram acquired by delimiter pattern acquisition unit 410 from determination unit 420 and acquires the delimiter probability coefficient from the probability coefficient list stored in probability coefficient storage unit 4710. And transmitted to the probability coefficient output unit 460.

  On the other hand, even when a plurality of partial j-gram delimiter patterns are transmitted via the (n-1) gram generation unit 430, delimiter probability coefficients are acquired from the probability coefficient list for all delimiter patterns. All the delimiter patterns and delimiter probability coefficients are transmitted to the probability coefficient calculation unit 450.

  The probability coefficient calculation unit 450 calculates a delimitation probability coefficient of the delimiter pattern of the word n-gram from all delimiter patterns and delimitation probability coefficients transmitted from the probability coefficient acquisition unit 440 and transmits the delimitation probability coefficient to the probability coefficient output unit 460. The processing for calculating the delimiter probability coefficient of the word n-gram in the probability coefficient calculation unit 450 will be described later.

  The probability coefficient output unit 460 outputs the break probability coefficient acquired by the probability coefficient acquisition unit 440 to the character string analysis unit 30.

  The storage unit 470 receives information from each unit of the probability processing unit 40 and stores the transmitted information. In addition, in response to commands from each unit of the probability processing unit 40, information to be stored is output.

  The storage unit 470 includes a probability coefficient storage unit 4710 that stores a later-described probability coefficient list, and a setting storage unit 4720 that stores setting parameters for the probability processing unit 40 to execute processing.

  An example of the probability coefficient list stored in the probability coefficient storage unit 4710 will be described with reference to FIG. FIG. 8 is an example of a probability coefficient list in which break probability coefficients corresponding to bigrams and break flag patterns are registered.

  In the probability coefficient list, all teacher data stored in the data storage unit 702 are converted into word n-grams, and the probabilities are tabulated and registered in the probability coefficient storage unit 4710 according to the number of data of the division patterns. .

  For example, if 100 menus are accumulated in the teacher data, there are some cases that contain “pork rose meat”, and those that have “pig” and “rose meat” separated. When there are two cases, the value of the probability coefficient list in the division flag “010” of the bigram (pig, rose meat) is 0.02.

  When the probability coefficient acquisition unit 440 acquires the partition probability coefficient of the partition pattern “0 pig 1 rose 0”, refer to the row of the bigram “pork-rose meat” and the column of pattern “010” in the probability coefficient list. Returns the value 0.02. As described above, the probability coefficient list is a list indicating the probability that the teacher data is partitioned by the partition pattern of the partition pattern of the word n-gram.

  Next, a process in which the probability coefficient calculation unit 450 calculates a delimitation probability coefficient of a delimiter pattern of a word n-gram using a delimiter pattern of a plurality of word j-grams and the delimiter probability coefficient thereof will be described with reference to FIG. To do.

  For example, when a certain trigram is not registered in the probability coefficient list, the division probability coefficient of the trigram is calculated from the division probability coefficient in the bigram or monogram.

  As an example, when a trigram (no red wine, boiled) has not been registered, a method of calculating a delimitation probability coefficient of the target delimiter pattern “0 red wine 1 boiled 1” is shown.

  At this time, if the first bigram (no red wine) and the second bigram (red wine, boiled) are registered, as shown in FIG. 9A, the division probability coefficient of the division pattern “one red wine 0 of 0” And “1 red wine 0 boiled 1” are obtained, and the average is taken to obtain the division probability coefficient of the trigram.

  If no bigram is registered, if the first monogram (no), the middle monogram (red wine), and the last monogram (boiled) are registered, as shown in FIG. The division probability coefficients of “1 of 0”, “1 red wine 0”, and “0 boiled 1” are acquired, and the average is taken to obtain the division probability coefficient of the trigram.

  The method of calculating the break probability coefficient is not limited to the above, and a method of distributing the break probability coefficient of the corresponding pattern in the first half bigram by the break probability coefficient of the corresponding pattern in the second half bigram or the like can also be adopted.

  Next, processing executed by the information processing unit 70 of the menu display device 1 will be described with reference to a flowchart. When the user performs an operation of acquiring a menu image using the image input unit 10, the menu display device 1 starts the menu display process illustrated in FIG. 10.

  In the menu display process, first, an image is acquired by photographing a menu using the image input unit 10 (step S101).

  The OCR 20 reads characters based on the acquired image and acquires a recognized character string (step S102).

  When the OCR 20 acquires a character string and transmits it to the character string analysis unit 30, the segmentation unit 320 of the character string analysis unit 30 first executes a segmentation process that divides the character string into units of words, and converts the character string into the word sequence. Conversion to W (step S103).

  The n-gram string generation unit 330 generates an n-gram string from the word string W (step S104). In the following description, n = 2.

  The character string analysis unit 30 estimates which part of the word string W the menu is divided into, and executes a menu dividing process for dividing the menu (step S105). The details of the menu division process will be described later.

  When the menu is divided into partial character strings in the menu division processing, the conversion unit 50 executes processing for generating display data for the i-th partial sequence with the counter variable set to i.

  That is, the explanation data of each word included in the i-th partial sequence is acquired from the term dictionary storage unit 60 and converted into display data as shown in FIG. 2C (step S106).

  It is determined whether or not the process of converting to the display data has been completed for all of the partial columns obtained in step S105 (step S107). If not completed (step S107; NO), the counter variable i is incremented (step S108). ) The process is repeated from step S106 on the next partial sequence.

  On the other hand, when it is determined that all partial columns have been converted to display data (step S107; YES), the display unit 80 displays the obtained display data (step S109). The menu display process ends here.

  The menu division process executed in step S105 will be described with reference to FIG. In the menu division process, first, the n-gram selection unit 340 selects q as the target n-gram by using q as a counter variable from 1 to the number of n-gram sequences and the q-th word n-gram in the n-gram sequence (step S201).

  Here, the n-gram selection unit 340 selects the attention n-gram from the first word n-gram in order toward the subsequent word n-gram, but the order of selecting the attention n-gram may be reversed.

  Next, the delimiter pattern generation unit 350 generates a delimiter pattern that can be defined for the target n-gram (step S202). Here, 2 ^ (n + 1) = 8 division patterns are generated. The delimiter pattern generation unit 350 arranges the generated delimiter patterns in a route probability table as shown in FIG.

  Next, r is a counter variable from 1 to the number of partition patterns, and the r-th partition pattern is selected as the target partition pattern among the partition patterns generated in step S202 (step S203).

The probability coefficient acquisition unit 3610 transmits the attention break pattern to the probability processing unit 40, and the probability processing unit 40 starts processing to obtain the break probability coefficient p _qr of the attention break pattern (step S204).

  That is, the probability coefficient acquisition unit 440 acquires a delimitation probability coefficient for the target delimiter pattern of the target n-gram from the probability coefficient list stored in the probability coefficient storage unit 4710. Details of the delimitation probability coefficient acquisition process will be described later.

When the partition probability coefficient p _qr of the target partition pattern is acquired in step S204, the root probability coefficient pp _qr is calculated using the previously acquired root probability coefficient pp _q-1y (y = 1 to 8) in the previous row ( Step S205). Details of the route probability coefficient calculation process will be described later.

After calculating the route probability coefficient pp _qr in step S205, the information processing unit 70 determines whether or not the processing for calculating the route probability coefficient for all the delimiter patterns generated in step S202 is completed (step S206).

That is, when the calculation is completed for all of the route probability coefficients (pp _{q1 to} pp _q8 ) for the eight _segmentation patterns generated for the target n-gram (the word n-gram in the q-th column), it is determined that the processing is completed. The

  If it is determined that processing has not been completed for all the delimiter patterns (step S206; NO), r is incremented (step S207), and the processing is repeated from step S203 for the next delimiter pattern.

  On the other hand, if it is determined that all the segmentation patterns have been processed (step S206; YES), it is then determined whether or not the processing for calculating the root probability coefficient for all the word n-grams generated in step S104 has been completed (step S208). ).

  If it is determined that all the word n-grams have not been processed (step S208; NO), q is incremented (step S209), and the process is repeated from step S201 for the next word n-gram.

On the other hand, if it is determined that all the word n-grams have been processed (step S208; YES), the route selection unit 370 selects the one having the largest route probability coefficient from the corresponding delimiter patterns of the last word n-gram. In the example of FIG. 5, the root probability coefficient pp ₆₄ indicated by the symbol “*” is selected.

  From the last word n-gram to the first word n-gram, among the common adjacent delimiter patterns of the selected delimiter pattern, the delimiter pattern having the larger root probability coefficient is sequentially selected to obtain the maximum likelihood route ( Step S210).

  When a delimiter pattern on the maximum likelihood route is selected in step S210, the word string dividing unit 380 divides the word string W by the method of delimiting the delimiter pattern into a partial character string (step S211). The output unit 390 transmits the partial character string to the conversion unit 50, and the menu division process ends.

  The delimitation probability coefficient acquisition process executed in step S204 will be described with reference to FIG. In step S204, when the delimiter pattern acquisition unit 410 receives a delimiter pattern including the word n-gram and delimiter flag from the menu analysis unit 30, the probability coefficient output unit 40 starts delimiter probability coefficient acquisition processing.

  In the delimiter probability coefficient acquisition process, first, the determination unit 420 refers to the probability coefficient list stored in the probability coefficient storage unit 4710 (step S301).

  As a result of the reference, it is confirmed whether or not there is a delimitation probability coefficient of the target n-gram in the probability coefficient list (step S302).

  If there is a delimiter probability coefficient of the target n-gram (step S302; YES), a delimiter probability coefficient of the delimiter pattern of the target n-gram is acquired from the probability coefficient list (step S303).

  On the other hand, when the delimiter probability coefficient of the target n-gram is not in the probability coefficient list (step S302; NO), the delimiter probability coefficient calculation process is executed by changing the level from the word (n-1) gram to the monogram (step S305).

  When the delimitation probability coefficient is acquired in step S303 or step S305, the acquired delimitation probability coefficient is output to the character string analysis unit 30 (step S306). This is the end of the separation probability coefficient acquisition process.

  The delimitation probability coefficient calculation process executed in step S305 will be described with reference to FIG. In the delimitation probability coefficient calculation process, first, the (n-1) gram generation unit 430 generates two part (n-1) grams from the target n-gram (step S401).

  Whether the determination unit 420 can obtain a delimitation probability coefficient for both of the two parts (n−1) grams or the part (n−1) grams of the probability coefficient list stored in the probability coefficient storage unit 4710. A thing is confirmed with reference to (step S402).

  If the delimitation probability coefficients of both partial (n-1) grams can be acquired (step S402; YES), the delimitation probability coefficient of the target delimiter pattern is calculated using the delimitation probability coefficients of the partial (n-1) gram. Therefore, the process proceeds to step S406.

  If the segmentation probability coefficient of any part (n-1) gram is not in the probability coefficient list (step S402; NO), the part (n-2) gram is used next, and so on. Try to calculate the break probability coefficient.

  Since the stage cannot be lowered from the monogram, it is determined whether the part (n-1) gram generated in step S401 is a monogram (step S403). If it is not a monogram (step S403; NO), the stage is further lowered. (Step S404), trial from step S401.

  For example, at the stage of the partial (n−2) gram, since the number of words in the partial sequence is reduced by 2, the number of partial sequences that can be generated is increased by two. When the target n-gram is a trigram, the portion (n-2) gram becomes a monogram, so that three monograms are generated.

  On the other hand, if the part (n-1) gram generated in step S401 is a monogram (step S403; YES), the level cannot be lowered further, so that a monogram having no delimiter probability coefficient in the probability coefficient list is delimited. After the pattern is generated, a default value (0.5 in this case) is set as the division probability coefficient (step S405).

  If the partial (n-1) gram which can acquire all the division | segmentation probability coefficients is determined by step S401-step S405, it respond | corresponds with the attention division | segmentation pattern transmitted from the character string analysis part 30 about all the partial (n-1) grams. A delimiter pattern having a delimiter flag common to the words to be generated is generated (step S406).

  For all the generated segment (n-1) gram delimiter patterns, delimiter probability coefficients are acquired in the same manner as in step S303 of the probability coefficient acquisition process (step S407).

  The division probability coefficients of the division patterns of all the obtained part (n-1) grams are averaged to obtain the division probability coefficient of the target division pattern (step S408). This completes the probability coefficient calculation process.

  The route probability coefficient calculation process executed in step S205 will be described with reference to FIG. In the route probability coefficient calculation process, first, an effective coefficient is defined in the correction unit 3630 (step S501). The correction process executed by the correction unit 3630 is performed based on the effectiveness indicated by the effective coefficient. The effective coefficient is a value indicating the effectiveness based on the attribute of the word included in the partial sequence, and correction using the effective coefficient will be described later.

  Correction unit 3630 determines whether the effective coefficient determined in step S501 is equal to or greater than a predetermined threshold (step S502). The threshold value varies depending on what the effective coefficient is.

  If it is equal to or greater than the threshold (step S502; YES), a correction process for increasing the influence on route selection is executed (step S503). Specifically, the obtained partitioning probability coefficient is squared to obtain the corrected partitioning probability coefficient.

  On the other hand, when the effective coefficient is smaller than the threshold value (step S502; NO), step S503 is skipped.

  Furthermore, the route probability coefficient calculation unit 3640 extracts a common adjacent delimiter pattern having a common delimiter flag from the corresponding delimiter patterns of adjacent n-grams (step S504). In the first loop, the processing for the adjacent n-grams has not been completed yet, so this processing is skipped.

  Next, among the common adjacent delimiter patterns extracted by the route probability coefficient calculation unit 3640, a delimiter pattern having a larger route probability coefficient is selected, and a path from the selected common adjacent delimiter pattern to the target delimiter pattern is determined as the target delimiter. A maximum likelihood path to the pattern is selected (step S505). In the first loop, the processing for the adjacent n-grams has not been completed yet, so this processing is skipped.

  The route probability coefficient calculation unit 3640 multiplies the route probability coefficient of the common adjacent partition pattern selected in step S505 by the partition probability coefficient of the target partition pattern acquired in the partition probability coefficient acquisition process or the partition probability coefficient corrected in step S503. Then, it is set as the route probability coefficient of the noticeable separation pattern (step S506). In the first loop, the partition probability coefficient of the target partition pattern is set as the root probability coefficient. This completes the route probability coefficient calculation process.

  Here, the effective coefficient will be described with reference to FIG. FIG. 15 is a diagram showing a trigram list when parts of speech are attached to words in the teacher data. When the character string is divided into words, attention is paid to the effectiveness of the words, and the connection probability between the co-occurrence words is weighted to improve the analysis accuracy. Therefore, in calculating the route probability coefficient, weighting is performed in advance based on the part of speech of the word.

  As shown in FIG. 15A, the teacher data is represented by a tagged character string, but a part of speech is attached to the word when the teacher data is created. There are methods for adding a part of speech, such as preparing a tag representing a part of speech or adding a parameter representing a part of speech to a tag <w> </ w> representing a word.

  Here, a method of adding a parameter representing the part of speech to a tag indicating a word will be described. For example, if the word part of speech is a noun <wn> </ w>, a verb <wv> </ w>, an adjective <wa> </ w>, a particle <wx> < / w> tag.

  As shown in FIG. 15B, when the trigram list is created from the teacher data, the frequency of appearance of the part of speech of each trigram can be grasped. It is also possible to add part of speech information by analyzing the part of speech of each word when creating a trigram.

  Of the part of speech of a word, particles and the like are words that have little meaning in the menu. On the other hand, nouns, verbs, adjectives and the like are important words in the menu, and these words are effective words. Effective words also have a significant effect on the separation of menu strings.

  In the example of FIG. 15, the effective words are “clam”, “wakame”, “white wine”, and “steamed”, and how these words are separated is important. For this reason, the influence of a trigram containing many valid words increases.

  That is, in each trigram, the part-of-speech appearance frequency is considered to have an influence when calculating the root probability coefficient. Therefore, if it is reflected in the division probability coefficient of the division pattern when the route is selected, the accuracy of the analysis is improved.

  Therefore, the trigram list has an effective coefficient, and the number of effective words is registered. When many valid words are included, for example, a trigram, when two or more valid words are included, the influence is reflected. In other words, weights are given to those having high effectiveness.

  Next, correction using a preset effective coefficient will be described with reference to FIG. FIG. 16 is a diagram illustrating a probability coefficient list in which weighting is performed based on parts of speech attached to words in teacher data.

  The trigram probability coefficient list is created by assigning a delimiter probability coefficient for each delimiter pattern to the trigram list. The delimitation probability coefficient is the ratio of the number of trigram delimiter pattern data among all the teacher data.

  For example, there are several menus that contain (clams, wakame), but some menus are divided between “clams” and “wakame”, but some menus are not separated, and many menus are registered in the teacher data. By doing so, it becomes possible to estimate the possibility of separation.

  Here, as shown in FIG. 16, (clam, and wakame) includes two effective words and has an effective coefficient of 2, and (and wakame) has only one effective word and has an effective coefficient. 1. It is important whether or not it is divided between “clam” and “wakame”, but whether it is divided between “to” and “wakame” depends on the word before “to” and the importance is large. Low.

  Therefore, in the case of trigrams, if the effective coefficient is 2 or more, it will be an important judgment material when there are many effective words and it is divided between words, so that the delimiter probability coefficient of the trigram is squared. .

  In the correction process, the dividing probability coefficient acquired from the probability coefficient list may be squared, or when the probability coefficient list is created, the dividing probability coefficient is registered based on the effective coefficient and is corrected in advance. You may acquire a completed division | segmentation probability coefficient.

  By squaring, the partitioning pattern with a high probability has a higher partitioning probability coefficient, and the partitioning pattern with a lower probability has a lower partitioning probability coefficient. In other words, the possibility of being divided by the division pattern is greatly increased, and the route is greatly influenced.

  It is possible to change the processing stage according to the size of the effective coefficient. For example, if the effective coefficient is 3, it is raised to the third power. Since the division probability coefficient is smaller than 1, the numerical value becomes smaller as the power is raised, and may be normalized.

  Further, although the effective word is set based on the part of speech, it is also possible to set the effective word according to other attributes of the word such as a category. For example, a word that is easily used for a dish name or an ingredient name is a valid word, or the standard of the valid word is arbitrary.

  As described above, according to the menu display device 1 according to the embodiment, the character string representing the menu can be divided by the probability coefficient list generated based on the teacher data. Therefore, a character string can be divided between appropriate words without preparing a syntax analysis program for each language. Then, without embedding language grammar rules in the program, it is possible to divide the menu of a plurality of words into units representing one ingredient, cooking method, and the like.

  Further, in the present embodiment, when a word string is segmented, a method is used in which the segmentation probability coefficient has a stronger influence on the final segmentation method, whereby it is possible to estimate a segmentation method with high accuracy.

  In addition, since the correction unit corrects the partition probability coefficient based on the effectiveness and selects the partition pattern based on the root probability coefficient that reflects the effectiveness more strongly, it is possible to accurately estimate the word string partition method. .

  Further, in selecting a delimiter pattern, not only the delimiter probability coefficient of the delimiter pattern of the word n-gram but also the delimiter probability coefficient of the common adjacent delimiter pattern that shares the delimiter flag of the adjacent delimiter pattern is taken into account. Separation method can be determined based on the information.

  In addition, since the root probability coefficient is obtained as an increasing function of the partition probability coefficient of the corresponding partition pattern, the greater the probability that the partition data is partitioned by the teacher data including the word n-gram, the greater the probability that the partition pattern is The probability of being selected as a delimiter is increased, and the method of delimiting teacher data can be reflected in the delimiter of word strings.

  In addition, since the teacher data is generated from a character string of a predetermined category (here, a menu), it is more than the case where the delimiter probability coefficient of the delimiter pattern is obtained using teacher data of a broad category (for example, the whole Japanese). The division probability coefficient that matches the category can be obtained.

  Moreover, since the menu can be photographed and the character string can be recognized and analyzed and displayed, the user can capture the character string without adding the character string by hand and display it with the commentary data added. . Therefore, even if the menu is written in a language that the user does not know, comment data can be displayed.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to the specific embodiment which concerns, This invention includes the invention described in the claim, and its equivalent range It is. Hereinafter, the invention described in the scope of claims of the present application will be appended.

(Appendix 1)
A character string acquisition unit that acquires a character string to be analyzed, and a partial string extraction unit that converts the character string acquired by the character string acquisition unit into a word string and sequentially extracts partial strings that are included in the word string A delimiter pattern generating unit that generates all patterns for determining whether or not the word string is delimited between each partial sequence extracted by the partial sequence extracting unit, and each delimiter generated by the delimiter pattern generating unit A probability coefficient acquisition unit that obtains a division probability coefficient based on a division method in the teacher data for the pattern, weights based on the attribute of the word included in the partial sequence, and a division probability coefficient acquired by the probability coefficient acquisition unit, A route for calculating a route probability coefficient based on a common adjacent separation pattern of a partial sequence adjacent to a separation pattern of a partial sequence corresponding to the separation probability coefficient, which has a larger separation probability coefficient. A probability coefficient calculation unit, a route selection unit that selects a route so as to follow a partial sequence having a large route probability coefficient calculated by the route probability coefficient calculation unit, and the word string in a delimiter pattern in the route selected by the route selection unit An information processing apparatus comprising: a word string dividing unit that divides

(Appendix 2)
The information processing apparatus according to supplementary note 1, wherein the probability coefficient acquisition unit weights a part of speech of a word included in a subsequence as an attribute of the word.

(Appendix 3)
The information processing apparatus according to appendix 1, wherein the probability coefficient acquisition unit weights a category of a word included in a partial sequence as an attribute of the word.

(Appendix 4)
The information processing apparatus according to any one of appendices 1 to 3, wherein the probability coefficient acquisition unit acquires a segmentation probability coefficient weighted in advance based on an attribute of a word included in the partial sequence.

(Appendix 5)
An image input unit that captures an image of a character string, a character string acquisition unit that acquires a character string from an image captured by the image input unit, and a character string obtained by converting the character string acquired by the character string acquisition unit into a word string A partial sequence extraction unit that sequentially extracts partial sequences composed of words included in the sequence, and generates all patterns for which partial sequence the word sequence is delimited for each partial sequence extracted by the partial sequence extraction unit A delimiter pattern generation unit that obtains delimitation probability coefficients based on how to delimit in the teacher data for each delimiter pattern generated by the delimiter pattern generation unit, and weights based on the attribute of the word included in the partial sequence The delimitation probability among the acquisition unit, the delimiter probability coefficient acquired by the probability coefficient acquisition unit, and the common adjacent delimiter pattern of the subsequence adjacent to the delimiter pattern of the subsequence corresponding to the delimiter probability coefficient A route probability coefficient calculation unit that calculates a route probability coefficient based on a larger number, a route selection unit that selects a route so as to follow a subsequence with a large route probability coefficient calculated by the route probability coefficient calculation unit, A word string dividing unit that divides the word string by a delimiter pattern in the route selected by the route selecting unit; a conversion unit that converts each word string divided by the word string dividing unit into corresponding display data; and the conversion unit A data display device comprising: a display unit that displays the display data converted by.

(Appendix 6)
A process for acquiring a character string to be analyzed in a computer, a process for converting the acquired character string into a word string and sequentially extracting partial strings composed of words included in the word string, and for each of the extracted partial strings Processing for generating all patterns of whether or not the word sequence is divided between words, obtaining a delimitation probability coefficient based on the delimitation method in the teacher data for each generated delimiter pattern, and the word included in the partial sequence Based on the process of weighting based on the attribute of the above, the obtained delimitation probability coefficient, and the larger delimitation probability coefficient among the common adjacent delimiter patterns of the partial sequences adjacent to the delimiter pattern of the partial sequence corresponding to the delimitation probability coefficient A route probability coefficient calculation process, a route selection process so that the calculated route probability coefficient traces a partial sequence, and the selected route probability coefficient. A program characterized by executing a process of dividing the word string at division pattern in.

  DESCRIPTION OF SYMBOLS 1 ... Menu display apparatus, 10 ... Image input part, 20 ... OCR, 30 ... Character string analysis part, 40 ... Probability processing part, 50 ... Conversion part, 60 ... Term dictionary memory | storage part, 70 ... Information processing part, 80 ... Display 90, operation input unit, 701 ... information processing unit, 702 ... data storage unit, 703 ... program storage unit, 704 ... input / output unit, 705 ... communication unit, 706 ... internal bus, 707 ... control program, 310 ... character Sequence acquisition unit, 320 ... Sequence writing unit, 330 ... n-gram sequence generation unit (partial sequence extraction unit), 340 ... n-gram selection unit, 350 ... separation pattern generation unit, 360 ... route probability coefficient definition unit, 3610 ... acquire probability coefficient Part, 3630 ... correction part, 3640 ... route probability coefficient calculation part, 370 ... route selection part, 380 ... word string division part, 390 ... output part, 410 ... delimiter pattern acquisition part, 420 ... discrimination 430 ... (n-1) gram generation unit, 440 ... probability coefficient acquisition unit, 450 ... probability coefficient calculation unit, 460 ... probability coefficient output unit, 470 ... storage unit, 4710 ... probability coefficient storage unit, 4720 ... setting storage unit , I1 ... Image

Claims (8)

A partial string extraction unit that converts a character string to be analyzed into a word string and sequentially extracts partial strings composed of words included in the word string;
A delimiter pattern generation unit that generates all patterns of whether or not the word string is delimited between words for each partial sequence extracted by the partial sequence extraction unit;
A probability coefficient acquiring unit that acquires a delimitation probability coefficient based on a delimitation method in teacher data for each delimiter pattern generated by the delimiter pattern generation unit, and weights based on an attribute of a word included in the partial sequence;
Multiplying the delimitation probability coefficient acquired by the probability coefficient acquisition unit by the one with the larger delimitation probability coefficient among the common adjacent delimiter patterns of the previous partial sequence adjacent to the delimiter pattern of the partial sequence corresponding to the delimitation probability coefficient and route probability factor calculation section that calculates the route probability coefficients,
A route selection unit that selects a route so as to follow a partial sequence having a large route probability coefficient calculated by the route probability coefficient calculation unit;
An information processing apparatus comprising: a word string division unit that divides the word string by a division pattern in a route selected by the route selection unit.   The information processing apparatus according to claim 1, wherein the probability coefficient acquisition unit weights a part of speech of a word included in the partial sequence as an attribute of the word.   The information processing apparatus according to claim 1, wherein the probability coefficient acquisition unit weights a category of a word included in a partial sequence as an attribute of the word.   The information processing apparatus according to claim 1, wherein the probability coefficient acquisition unit acquires a segmentation probability coefficient that is weighted in advance based on an attribute of a word included in the partial sequence.   5. The correction unit for correcting the division probability coefficient multiplied by the route probability coefficient calculation unit and the plurality of division probability coefficients of the common adjacent division pattern before multiplication, respectively. The information processing apparatus according to any one of claims.   The information processing apparatus according to claim 5, wherein the correction unit corrects the plurality of division probability coefficients so that contrast between the plurality of division probability coefficients of the common adjacent division pattern is increased. . A character string acquisition unit for acquiring a character string from an image ;
A partial string extraction unit that sequentially converts a character string acquired by the character string acquisition unit into a word string and extracts a partial string including words included in the word string;
A delimiter pattern generation unit that generates all patterns of whether or not the word string is delimited between words for each partial sequence extracted by the partial sequence extraction unit;
A probability coefficient acquiring unit that acquires a delimitation probability coefficient based on a delimitation method in teacher data for each delimiter pattern generated by the delimiter pattern generation unit, and weights based on an attribute of a word included in the partial sequence;
It multiplies the division probability coefficients obtained by the probability coefficient acquisition unit, and a person division probability coefficients of common neighboring division pattern of the previous subsequence adjacent to delimiting pattern of the portion column corresponding to the division probability factor is greater and route probability factor calculation section that calculates the route probability coefficients,
A route selection unit that selects a route so as to follow a partial sequence having a large route probability coefficient calculated by the route probability coefficient calculation unit;
A word string dividing unit that divides the word string by a division pattern in the route selected by the route selection unit;
A conversion unit that converts each word string divided by the word string dividing unit into corresponding display data;
A data display device comprising: a display unit configured to display display data converted by the conversion unit. On the computer,
Processing to obtain the character string to be analyzed,
A process of converting the acquired character string into a word string and sequentially extracting partial strings composed of words included in the word string;
A process for generating all patterns of whether or not the word string is divided between which words for each of the extracted partial strings;
A process of obtaining a delimitation probability coefficient based on the delimiter in the teacher data for each generated delimiter pattern, and weighting based on the attribute of the word included in the partial sequence,
A route probability coefficient obtained by multiplying the obtained delimiter probability coefficient by the larger delimiter probability coefficient among the common adjacent delimiter patterns of the previous partial sequence adjacent to the delimiter pattern of the partial sequence corresponding to the delimiter probability coefficient Processing to calculate,
A program for executing a process of selecting a route so as to follow a partial sequence having a large calculated route probability coefficient, and a process of dividing the word string by a delimiter pattern in the selected route.

Images