JP2005267249A - Data processing system, server and communication device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a data processing system, a server and a communication device capable of improving the accuracy of character recognition in image data taken by the communication device. <P>SOLUTION: When country information is transmitted with image data from a portable terminal 10, a probability model generation part 204 performs weighting to a language used for character recognition based on the country information, according to an instruction from a control part 209, to generate a probability model of language. When a character string to be character-recognized is composed of a plurality of words, the probability model generation part 204 divides the character string of a recognition result transferred from a character recognition part 203 to words based on a language dictionary 206, extracts candidates of a word estimated to come next from a word just before a character to be recognized or a word before this word, performs weighting to the extracted candidates of the word, and generates a probability model of the word estimated to come next. The character recognition part 203 performs the character recognition according to the weighting shown by the probability model of language and the probability model of the word. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a data processing system, a server, and a communication device that perform character recognition and translation of image data captured by a communication device using a server.

  With the development of information communication technology in recent years, high functionality of communication devices such as portable terminals is rapidly progressing. One of the enhancements is the popularization of mobile terminals with built-in digital camera functions. In addition, with the 3G (3rd Generation) of mobile terminals, many communication systems that have existed so far are being unified, and the possibility that mobile terminals that are used everyday can be used even in foreign countries is increasing.

  Based on this background, when you visit a foreign country, such as when traveling, you can take a picture of the displayed characters with a digital camera built into your mobile device and recognize the characters in the image data Many systems that can translate are proposed. However, generally, in order to perform character recognition, an image with a considerably high resolution is required as represented by OCR (Optical Character Reader). Although the number of pixels of digital cameras built in portable terminals has increased dramatically in recent years, it has not yet reached the accuracy necessary for highly accurate character recognition. Further, the processing capability of a CPU (Central Processing Unit) built in the portable terminal is not sufficient for character recognition, and further improvement in character recognition is expected.

  As a conventional technology that compensates for the shortage of processing capability of such portable terminals and improves the accuracy of character recognition, portable terminal type images that transmit images taken with the camera of the portable terminal to the server and perform character recognition and translation on the server There is a processing system, and the type of facility where the mobile terminal is located is specified from the current position and map data of the mobile terminal equipped with the GPS (Global Positioning System) function, or specified by the user instructing from the mobile terminal Alternatively, the recognition accuracy is improved by switching to a recognition dictionary or language dictionary corresponding to the type of facility instructed (see, for example, Patent Document 1). In addition, the portable terminal of this portable terminal type image processing system binarizes captured image data into white and black so as to separate the color image from the background, reducing the amount of data transferred to the server, and the transmission time Is shortened.

JP 2003-178067 A

  The conventional technology described above has a problem that unless the facility where the mobile terminal is located is specified or designated, the range of the recognition dictionary and the language dictionary cannot be limited, and improvement in recognition accuracy cannot be expected.

  In addition, when binarization of the photographed color image is not performed correctly, there is a problem that there is an increased possibility that character recognition will be incorrect and not correctly translated.

  An object of the present invention is to provide a data processing system, a server, and a communication device that can improve the accuracy of character recognition of image data captured by a communication device.

The present invention relates to a server having a character recognition function and a translation function, and a communication device having a function of transferring image data taken for character recognition and translation to the server and displaying the translation result transferred from the server. In a data processing system having
The communication device transfers country information indicating where the communication device is located to the server together with the image data,
The server is a data processing system that assigns a weight to a language used for character recognition based on the transferred country information and performs character recognition of the image data according to the weight given to the language.

In addition, the present invention provides a server having a character recognition function and a translation function, and a communication having a function of transferring image data taken for character recognition and translation to the server and displaying the translation result transferred from the server. In a data processing system having a device,
The communication device transfers language information indicating a language designated by a user to the server together with the image data,
The server is a data processing system that performs character recognition of the image data using a language indicated by transferred language information.

  Further, according to the present invention, when the server performs character recognition, the server is a word composed of characters that have already been character-recognized, and is based on the word immediately before the character to be character-recognized or the word immediately before it. A candidate for a word including a character for character recognition is extracted, a weight is given to the extracted candidate, and a character is recognized according to the weight given to the candidate word.

  Further, the present invention is characterized in that the communication device transfers image data obtained by performing a color reduction process on photographed image data to the server.

Further, the present invention provides a server having a character recognition function and a translation function.
Receives image data from the communication device and country information indicating in which country the communication device is located, assigns a weight to a language used for character recognition based on the received country information, and assigns a weight to the language The server performs character recognition of the image data according to the above.

Further, the present invention provides a server having a character recognition function and a translation function.
A server that receives image data and language information indicating a language designated by a user from a communication device, and performs character recognition of the image data using a language indicated by the received language information.

  Further, according to the present invention, when the server performs character recognition, the server is a word composed of characters that have already been character-recognized, and is based on the word immediately before the character to be character-recognized or the word immediately before it. A candidate for a word including a character for character recognition is extracted, a weight is given to the extracted candidate, and a character is recognized according to the weight given to the candidate word.

Further, the present invention provides a communication device having a function of transferring image data taken for character recognition and translation to a server and displaying a translation result transferred from the server.
Country information indicating in which country the communication apparatus is located is transferred together with the image data.

Further, the present invention provides a communication device having a function of transferring image data taken for character recognition and translation to a server and displaying a translation result transferred from the server.
A communication apparatus is characterized in that language information indicating a language designated by a user is transferred together with the image data.

  In the invention, it is preferable that the communication device transfers image data obtained by performing a color reduction process on photographed image data.

  According to the present invention, since the communication device automatically informs the server of the country information of the country in which the communication device is located, the server gives a large weight to the language mainly used in the country based on the country information. In addition, some other languages that are not mainly used are given some weighting, so even if there is a character that is not used very much in that country, it can be recognized. Recognition accuracy can be improved.

  Further, according to the present invention, since character recognition is performed in a language specified by the user, a language unrelated to the character to be recognized can be excluded from the recognition target, and the accuracy of character recognition can be improved.

  Further, according to the present invention, when character recognition is performed, a character composed of characters that have already been character-recognized and based on the word immediately before the character to be character-recognized or the two previous words Word candidates including characters to be recognized are extracted, a large weight is given to frequently appearing words among the extracted candidates, and characters are recognized according to the weight given to the candidate words, so that the character candidates to be recognized are limited. And the accuracy of character recognition can be improved.

  In addition, according to the present invention, since the image data subjected to the color reduction processing is transferred to the captured image data so as not to hinder character recognition, the character recognition accuracy is improved as compared with the case of binarization. In addition, the amount of data transferred to the server is smaller than when no color reduction processing is performed, and the transmission time can be shortened.

  Further, according to the present invention, based on the country information automatically notified from the communication device, the language mainly used in the country can be recognized by giving a large weight, and other languages not mainly used can be recognized. However, although it is small, some weighting is performed, so that even if there is a character that is not frequently used in the country, the character can be recognized, and the accuracy of character recognition can be improved.

  Further, according to the present invention, since character recognition is performed in a language specified by the user, a language unrelated to the character to be recognized can be excluded from the recognition target, and the accuracy of character recognition can be improved.

  Further, according to the present invention, when character recognition is performed, a character composed of characters that have already been character-recognized and based on the word immediately before the character to be character-recognized or the two previous words Word candidates including characters to be recognized are extracted, a large weight is given to frequently appearing words among the extracted candidates, and characters are recognized according to the weight given to the candidate words, so that the character candidates to be recognized are limited. And the accuracy of character recognition can be improved.

  Further, according to the present invention, the country information indicating the country in which the communication device is located can be provided to the server, so that the accuracy of character recognition can be improved.

  Further, according to the present invention, the language information indicating the language designated by the user can be provided to the server, so that the accuracy of character recognition can be improved.

  In addition, according to the present invention, image data obtained by performing color reduction processing on captured image data to such an extent that does not hinder character recognition can be provided to the server, so that character recognition accuracy can be improved compared to binarization. In addition, the amount of data transferred to the server is smaller than when no color reduction processing is performed, and the transmission time can be shortened.

  FIG. 1 is a block diagram illustrating a configuration of a mobile terminal 10 of a communication apparatus according to an embodiment of the present invention. The mobile terminal 10 is a communication device such as a mobile phone, for example, and includes a transmission unit 101, a reception unit 102, a photographing unit 103, a display unit 104, a key input unit 105, an image processing unit 106, a display driver 107, a memory 108, and A control unit 109 is included. Although not shown, the portable terminal 10 is also equipped with a voice input unit such as a microphone and a voice output unit such as a speaker.

  The control unit 109 is a control unit that controls the mobile terminal 10 as a whole, and temporarily stores, for example, a CPU (Central Processing Unit) realized by a microcomputer, a memory that stores a program, and information necessary for processing. And a memory.

  The key input unit 105 includes keys for inputting numbers of telephone numbers, characters of e-mails, operation buttons for operating a built-in digital camera, and the like, and information and instructions input by the user from the key input unit 105 are And transferred to the control unit 109.

  When the user desires translation, the user first points the photographing unit 103 toward the subject and operates the key input unit 105 to photograph an image including characters. The photographing unit 103 is a built-in digital camera configured by a camera module using, for example, a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and controls a user instruction received from the key input unit 105. The image is received from the unit 109. The photographed image is subjected to image compression and color reduction processing by the image processing unit 106 and then stored as image data in a readable / writable storage medium, for example, a memory 108 such as a semiconductor memory.

  After photographing, the user inputs a language to translate and a translation instruction. The input language to be translated and the translation instruction are notified to the control unit 109. Upon receiving the notification, the control unit 109 instructs the transmission unit 101 to transmit the image data stored in the memory 108 to the server together with the language to be translated input by the user. A transmitting unit 101 that transmits data by wireless communication reads image data from the memory 108 and transmits the read image data to the server together with the language to be translated input by the user.

In order to maintain a good communication state, the mobile terminal 10 that accompanies movement periodically communicates with a base station to know which base station is nearby. At this time, the mobile terminal 10 receives the location information of the base station from the base station and holds it in the memory. This location information includes information on the country where the base station is located (hereinafter referred to as country information). The country information may be, for example, a country name or a number for specifying the country name. Also, an IC on which contractor information is recorded, which is used to identify a user by being inserted into the mobile terminal 10
There is similar country information in a SIM (Subscriber Identity Module) card which is an (Integrated Circuit) card, and this information may be used. The control unit 109 transmits this country information when transmitting image data to the server.

  If the user can determine the language name of the photographed characters, the language name may be instructed from the key input unit 105, and the instructed language may be transferred to the server as language information instead of country information. The language information may be, for example, a language name or a language number that can identify the language name. As a method for instructing the language by the user, a list of language names may be displayed on the display unit 104, and the user may select a language name from the list, or the language name may be directly input from the key input unit 105.

  Further, the image data to be transmitted to the server may be transmitted by performing a color reduction process to such an extent that the recognition rate of character recognition is not lowered, and reducing the data amount. The degree of color reduction can vary depending on the condition and environment of the image being shot.For example, there are several color reduction steps, such as for bright places, dark places, dark colors, and light colors. You may prepare it and let the user select it while trying the success or failure of the translation.

  The receiving unit 102 that receives data by wireless communication receives the translated result after the characters of the transmitted image data are recognized from the server, stores the received translation result in the memory 108, and then stores the received translation result in the control unit 109. Notify that the translation result has been received. Upon receiving the notification, the control unit 109 instructs the display driver 107 to display the translation result. The display driver 107 is a drive unit that drives the display unit 104 such as a liquid crystal display. The display driver 107 reads the translation result from the memory 108 and displays the translation result on the display unit 104 to show the translation result to the user.

  FIG. 2 is a block diagram showing a configuration of the server 20 according to the embodiment of the present invention. The server 20 is a computer in a data processing system that collects and distributes data such as e-mail and voice information transmitted and received via a wireless communication network or the Internet. The server 20 includes a transmission unit 201, a reception unit 202, a character recognition unit 203, a probability model. A generation unit 204, a translation unit 205, a language dictionary 206, a translation dictionary 207, a memory 208, and a control unit 209 are configured.

  When receiving the image data transmitted from the portable terminal, the receiving unit 202 that receives data via wireless communication or the Internet stores the received image data in a readable / writable storage medium, for example, a memory 208 such as a semiconductor memory, The control unit 209 is notified that the image data has been received together with the language to be translated and country information or language information.

  The control unit 209 is a control unit that controls the entire server 20, and includes, for example, a CPU realized by a microcomputer, a memory that stores a program, and a memory that temporarily stores information necessary for processing. Is done. When notified from the receiving unit 202 that the image data has been received, the control unit 209 instructs the character recognition unit 203 to provide country information or language information.

  The character recognition unit 203 reads image data from the memory 208, extracts a character part from the image data, and performs character recognition on the extracted character. When performing character recognition, a character having the highest probability is selected as a recognized character from characters registered in the language dictionary 206, which is a multilingual language dictionary. The character recognition unit 203 transfers the recognition result of the recognized character to the translation unit 205. Based on the translation dictionary 207, the translation unit 205 translates into the language to be translated instructed by the control unit 209, stores the translation result in the memory 208, and notifies the control unit 209 of the completion of translation when the translation is completed. To do.

  When receiving the notification of translation completion, the control unit 209 instructs the transmission unit 201 to transmit the translation result to the terminal device 10. The transmission unit 201 that transmits data via wireless communication or the Internet reads the translation result from the memory 208 and transmits it to the terminal device 10.

  When the country information is transmitted together with the image data, the probability model generation unit 204 indicates the country information for the language registered in the language dictionary 206 based on the country information instructed from the control unit 209. Weight is calculated from the percentage of languages used in the country. For example, if the country information is Japan, the language used mainly in the country, such as Japanese weight 0.8, English weight 0.1, and other language weight 0.1 Give a big weight to. The probability model of this language, that is, the established model of Japanese 0.8, English 0.1, and other language 0.1 is notified from the probability model generation unit 204 to the character recognition unit 203.

  The character recognition unit 203 is registered in the language dictionary 206 when given a probability model of the notified language, that is, a probability model of Japanese 0.8, English 0.1, and other languages 0.1. When the character with the highest probability is selected as a recognized character from the characters, a value obtained by multiplying the probability of the selected character by the weight of the language of the character is used as the probability of the character.

  For example, when “u” or “i” is considered as a character candidate after recognition, if the probability of not considering weighting is 0.1 and “u” is 0.9, the weight is In other words, multiplying by 0.8 and Japanese 0.1 respectively, “I” becomes 0.08, “u” becomes 0.09, and the value of “u” is large, so “u” If “i” is 0.5 and “u” is 0.5, the weight is multiplied by “i” is 0.4 and “u” is 0. .05, and the value of “I” is large, so “I” is recognized.

  Therefore, the greater the weight, the higher the likelihood that the language is recognized. In addition, since other languages that are not mainly used are given some weighting, they can be recognized even if there are characters that are rarely used in that country. Even in a country where a plurality of official languages are used, weighting is performed according to the proportion of the official languages used, so it is possible to cope with them.

  In addition, when language information is transmitted together with image data instead of country information, the character recognition unit 203 recognizes a character after recognition from characters in the language indicated as language information in the language dictionary 206. Select. Characters in languages that are not instructed are excluded from recognition targets to reduce the possibility of misrecognizing characters.

  Further, when the character string for character recognition is composed of a plurality of words, the server 20 uses the language dictionary 206 to determine the next word from the immediately preceding word or the previous word, that is, the word including the character to be recognized. To improve the accuracy of character recognition.

  The probability model generation unit 204 divides the character string of the recognition result transferred from the character recognition unit 203 into words, and selects a word immediately before the recognized character or a word candidate estimated to come next from the previous word. A candidate is extracted and given a high weight to a candidate that is likely to appear frequently, a probability model of a word estimated to come next is generated and notified to the character recognition unit 203. When the character recognition unit 203 selects a character with the highest probability from characters registered in the language dictionary 206 as a recognized character, the character recognition unit 203 multiplies the probability of the character to be selected by the weight of the word of the character. Use the value as the probability of that character.

  For example, if the immediately preceding word is the word “I” in Japanese and the candidate that frequently appears as a word including the character to be recognized is “ha” or “ga”, the weight of “ha” is A probability model is generated in which a large weight is given to the next frequently occurring word, such as 0.2, the weight of “GA” is 0.2, and the other weights are 0.6. When “ha” or “ni” is considered as a character candidate after recognition, if the probability that weighting is not considered is “ha” is 0.8 and “ni” is 0.2, the weight is multiplied. In other words, “ha” is 0.2 and other 0.6, respectively, “ha” is 0.16, “ni” is 0.12, and “ha” is large. If the probability of recognizing and not considering the weight is “ha” is 0.5 and “ni” is 0.5, the weight is multiplied by “ha” is 0.1, and “ni” is 0. 3. Since the value of “ni” is large, it is recognized as “ni”.

  In this case, if it is not weighted, it will always be recognized only as “ha”, but the word candidate estimated from the previous word is also weighted as a candidate, so it may be recognized as “ni”. Come.

  A data processing system according to an embodiment of the present invention can be realized by combining the mobile terminal 10 and the server 20 described above. In this case, the mobile terminal 10 is not limited to one, and a plurality of mobile terminals 10 may be provided.

  FIG. 3 is a flowchart showing a character recognition and translation processing procedure of image data in the data processing system according to the embodiment of the present invention. The processing of the mobile terminal 10 is shown on the left side, and the processing of the server 20 is shown on the right side. doing. The process is started when the user wants to translate characters included in the captured image data and starts operating the mobile terminal 10.

  In step S <b> 1, the mobile terminal 10 performs a photographing process of photographing a subject according to a user instruction and taking it as image data. In step S2, a translation request from the user is accepted. In step S3, image compression processing is performed in order to reduce the amount of data transmitted to the server 10. However, the color reduction processing may be performed after the color reduction processing is performed so as not to hinder character recognition.

  In step S4, the captured image data and country information indicating the country in which the mobile terminal is obtained from the base station information and the like are transmitted to the server 20. In step S5, the server 20 receives the transmitted image data and country information.

  In step S6, a language probability model in which the language used for character recognition is weighted based on the transmitted country information, for example, when the country information is Japan, the Japanese weight is 0.8, A probability model having a weight of 0.1 and other language weights of 0.1 is generated. In step S7, it is confirmed whether or not there are a plurality of words to be recognized. If there are a plurality of words, the process proceeds to step S8, and if not, the process proceeds to step S9.

  In step S8, based on the language dictionary 206, a next word, that is, a word candidate including a character to be recognized is extracted from the immediately preceding word or the preceding word, and the extracted word candidate is weighted. A probabilistic model of the word that is supposed to come, for example, the word just before is the word "I" in Japanese, and the word that appears frequently as a word containing the character to be recognized is "ha" or "ga" If there is, a probability model of a word that gives a large weight to the next frequently occurring word, such as “ha” weight 0.2, “ga” weight 0.2, and other weights 0.6. Is generated.

  In step S9, if there is no language probability model, word probability model, or word probability model, character recognition is performed using weighting only the language probability model. In step S10, a translation process is performed based on the recognition result. In step S <b> 11, the translation result is transmitted to the mobile terminal 10.

  In step S <b> 12, the mobile terminal 10 receives the translation result transmitted from the server 20. In step S13, the received translation result is displayed as text data on a display unit such as a liquid crystal display.

  FIG. 4 is a flowchart showing processing steps for character recognition and translation of image data in a data processing system according to another embodiment of the present invention. When the user knows the language of the photographed character, instead of country information, the user designates the language and performs character recognition using the designated language, as in FIG. In addition, processing is started by a user operation.

  In step S <b> 20, the mobile terminal 10 captures a subject according to a user instruction, and performs a capturing process for capturing the subject as image data. In step S21, a translation request from the user is accepted. In step S <b> 22, the language name input by the user from the key input unit 105 of the mobile terminal 10 is captured.

  In step S23, an image compression process is performed to reduce the amount of data transmitted to the server 10, but the color reduction process may be performed after the color reduction process is performed so as not to hinder character recognition. In step S24, the captured image data and language information indicating the language name input by the user are transmitted to the server 20. In step S <b> 25, the server 20 receives the image data and language information transmitted from the mobile terminal 10.

  In step S26, based on the transmitted language information, the language dictionary used for character recognition is limited to only the language dictionary corresponding to the transmitted language information. In step S27, it is confirmed whether there are a plurality of words to be recognized. If there are a plurality of words, the process proceeds to step S28, and if not, the process proceeds to step S29.

  In step S28, based on the language dictionary 206, the next word, that is, the word candidate including the character to be recognized is extracted from the previous word or the previous word, and the extracted word candidate is weighted. A probabilistic model of the word that is supposed to come, for example, the word just before is the word "I" in Japanese, and the word that appears frequently as a word containing the character to be recognized is "ha" or "ga" If there is, a probability model of a word that gives a large weight to the next frequently occurring word, such as “ha” weight 0.2, “ga” weight 0.2, and other weights 0.6. Is generated.

  In step S29, using the language dictionary corresponding to the transmitted language information and the word probability model, if there is no word probability model, character recognition is performed using only the language dictionary corresponding to the transmitted language information. . In step S30, a translation process is performed based on the recognition result. In step S31, the translation result is transmitted to the portable terminal 10.

  In step S <b> 32, the mobile terminal 10 receives the translation result transmitted from the server 20. In step S33, the transmitted translation result is displayed as text data on a display unit such as a liquid crystal display.

It is a block diagram which shows the structure of the portable terminal 10 of the communication apparatus which is one Embodiment of this invention. It is a block diagram which shows the structure of the server 20 which is one Embodiment of this invention. It is a flowchart which shows the process sequence of the character recognition and translation of image data in the data processing system which is one Embodiment of this invention. It is a flowchart which shows the process sequence of the character recognition and translation of image data in the data processing system which is the other form of implementation of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Mobile terminal 20 Server 101,201 Transmission part 102,202 Reception part 103 Image | photographing part 104 Display part 105 Key input part 106 Image processing part 107 Display driver 108,208 Memory 109,209 Control part 203 Character recognition part 204 Probability model generation part 205 Translation Section 206 Language Dictionary 207 Translation Dictionary

Claims (10)

Data having a server having a character recognition function and a translation function, and a communication device having a function of transferring image data taken for character recognition and translation to the server and displaying a translation result transferred from the server In the processing system,
The communication device transfers country information indicating where the communication device is located to the server together with the image data,
The data processing system, wherein the server assigns a weight to a language used for character recognition based on the transferred country information and performs character recognition of the image data according to the weight given to the language. Data having a server having a character recognition function and a translation function, and a communication device having a function of transferring image data taken for character recognition and translation to the server and displaying a translation result transferred from the server In the processing system,
The communication device transfers language information indicating a language designated by a user to the server together with the image data,
The data processing system, wherein the server performs character recognition of the image data using a language indicated by the transferred language information.   When performing character recognition, the server performs character recognition based on a word composed of characters that have already been character-recognized and immediately before the character on which character recognition is performed or on the word immediately before the character to be character-recognized. 3. The data processing system according to claim 1, wherein word candidates including characters are extracted, weights are assigned to the extracted candidates, and characters are recognized according to the weights assigned to the candidate words.   The data processing system according to claim 1, wherein the communication device transfers image data obtained by performing a color reduction process on captured image data to the server. In a server having a character recognition function and a translation function,
Receives image data from the communication device and country information indicating in which country the communication device is located, assigns a weight to a language used for character recognition based on the received country information, and assigns a weight to the language The server performs character recognition of the image data according to the above. In a server having a character recognition function and a translation function,
A server which receives image data and language information indicating a language designated by a user from a communication device, and performs character recognition of the image data using a language indicated by the received language information.   When performing character recognition, the server performs character recognition based on a word composed of characters that have already been character-recognized and immediately before the character on which character recognition is performed or on the word immediately before the character to be character-recognized. 7. The server according to claim 5, wherein word candidates including characters are extracted, weights are assigned to the extracted candidates, and characters are recognized according to the weights assigned to the candidate words. In a communication device having a function of transferring image data taken for character recognition and translation to a server and displaying a translation result transferred from the server,
Country information indicating in which country the communication apparatus is located is transferred together with the image data. In a communication device having a function of transferring image data taken for character recognition and translation to a server and displaying a translation result transferred from the server,
A communication apparatus, wherein language information indicating a language designated by a user is transferred together with the image data.   10. The communication apparatus according to claim 8, wherein the communication apparatus transfers image data obtained by performing a color reduction process on photographed image data.

Images