KR20140068302A - System and Method for servicing contents using recognition of natural scene text - Google Patents

Abstract

The present invention relates to a system and a method to service content using natural scene image text. The system to service the content comprises a mobile device with an information gate application including a common character recognizing/translating engine which recognizes and translates text of a natural scene image of various application fields captured through a camera and a user interface unit; and a distribution server including a content database (DB) storing content corresponding to a natural scene image text. The user information unit of the information gate application provides a menu for objects and a menu for languages. When receiving object information selected from the menu for objects and language information selected from the menu for languages, the distribution server provides the corresponding content DB such that the mobile device may download it. The information gate application transmits the object information selected from the menu for objects and language information selected from the menu for languages to the distribution server; recognizes and translates the natural scene image text through the common character recognizing/translating engine; downloads a content DB from the distribution server; and searches the content DB to display the content. According to an embodiment of the present invention, after natural scene images obtained by directly imaging a notice, a signboard, food, general terminology, a map, a trade name, and the likes, in various fields through a camera or an already captured image recognized and translated; a content DB is searched using the translated images as search words, whereby a user can very easily obtain the desired information in various languages and in various application fields.

Description

TECHNICAL FIELD [0001] The present invention relates to a content service system and method using natural image text recognition,

The present invention relates to content providing for text, and more particularly, to a content service system and method using natural image text recognition.

Conventional character recognition has been commercialized by concentrating on image generation through a scanner and document processing. However, in recent years, as a result of the development of the mobile terminal environment, a service for collecting information through the image acquisition through a mobile camera and character recognition of a natural scene image and providing contents for the recognized text Is required.

Accordingly, mobile telecommunication companies, portal vendors, solution providers, and content providers that can build services as well as mobile terminal and camera manufacturers are developing technologies for recognizing texts of natural images and providing various services using them.

Therefore, it is necessary to extract and recognize specific scene information existing in a natural scene image obtained by using a camera, translate it in real time, and utilize it for information search.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a contents service system using natural image text recognition.

Another object of the present invention is to provide a content service method using natural image text recognition.

Another object of the present invention is to provide a mobile device used in a contents service system using natural image text recognition.

According to an aspect of the present invention, there is provided a content service system using natural image text recognition, including a common character recognition / translation engine for recognizing and translating texts of natural images of various application fields photographed through a built- A mobile device in which an information gate app is installed; And a distribution server for storing content corresponding to the recognized and translated natural image text, wherein the user interface of the information gate app is capable of selecting content information about the recognized natural image text And the distribution server receives the destination information and the language information selected by the user in a menu for each destination and a menu for each language in the user interface of the information gate application And a content DB corresponding to the received destination information and language information is downloaded so that the mobile device equipped with the information gateway app can download the information content. The information gateway app of the mobile device provides a menu for each purpose and a menu for each language The target information and the language selected by the user The content server receives the contents data from the distribution server, and transmits the contents data to the distribution server. The contents data is recognized and translated through the common character recognition / translation engine, And the content corresponding to the search word is displayed.

Wherein the common character recognition / translation engine of the information gate app comprises a natural image character region extraction unit for searching for an area where a character string exists in the natural image to extract a character; A multilingual character recognition unit for recognizing a character string of the extracted character area; And a multilingual translation unit for searching the multilingual database and translating the recognized character string into various languages.

The content service system using natural image text recognition according to the present invention further includes a management tool server for updating the content DB included in the distribution server.

According to an aspect of the present invention, there is provided a content service system using natural image text recognition, including a common character recognition / translation engine for recognizing and translating texts of natural images of various application fields photographed through a built- A mobile device in which an information gate app is installed; And a distribution server that stores contents corresponding to the recognized and translated natural image text, wherein the user interface unit of the information gate app can select contents information related to the recognized natural image text And a language-specific menu that allows a user to select a translation language, wherein the distribution server provides a natural-language text recognized through a common framework of the information-gate-app and a purpose- A content DB corresponding to the received destination information and language information is retrieved using the natural image text as a search word, and the content corresponding to the search word is retrieved from the content DB, To the mobile device on which the information gate app is installed And the information gateway app of the mobile device transmits the destination information and the language information selected by the user via the destination menu and the language menu of the user interface to the distribution server, Recognizes and translates text, receives contents corresponding to the natural visual text from the distribution server, and displays the contents on a screen of the mobile device.

According to another aspect of the present invention, there is provided a mobile device for use in a content service system using natural image text recognition, comprising: a character recognition unit for recognizing and translating a text of a natural image of various application fields, / Translation engine; And a user interface unit for providing a menu for each purpose and a language-based menu for selecting a content of the recognized text through the common character recognition / translation engine, Characterized in that the information gate application is equipped with an information gating app for displaying corresponding contents, the information gating app recognizes and translates the text of the natural image through the character recognition / translation engine, A content DB corresponding to the recognition and translation is downloaded from the distribution server to search for the content, and the retrieved content is displayed on the display screen by searching the downloaded content DB using the recognized and translated text as a search word .

Wherein the common character recognition / translation engine of the information gate app comprises a natural image character region extraction unit for searching for an area where a character string exists in the natural image to extract a character; A multilingual character recognition unit for recognizing a character string of the extracted character area; And a multilingual translation unit for searching the multilingual database and translating the recognized character string into various languages.

According to another aspect of the present invention, there is provided a mobile device for use in a content service system using natural image text recognition, comprising: a character recognition unit for recognizing and translating a text of a natural image of various application fields, / Translation engine; And a user interface unit for providing a menu for each purpose and a language-based menu for selecting a content of the recognized text through the common character recognition / translation engine, Wherein the information gate application recognizes and translates the text of the natural image through the character recognition / translation engine to display the recognized and translated natural image text and the target And the distribution server transmits the information and the language information to the distribution server. When the distribution server retrieves the contents DB by using the text of the natural image as a search word, the contents server receives the contents and displays the contents on the display screen.

According to another aspect of the present invention, there is provided a content service method using natural image text recognition, comprising: a common character recognition / translation engine for recognizing and translating texts of natural images of various application fields photographed through a built- 1. A content service method using natural image text recognition in a system for servicing a content related to a natural image text including a mobile device equipped with an information gate app including a content database and a distribution server, A step of recognizing and translating a text of a natural image if language information and object information are selected by a user in a language-specific menu and an object-specific menu provided by an information gateway application installed in the device; Downloading a content DB corresponding to the recognition and translation from the distribution server to retrieve content corresponding to the recognized and translated text; Searching the downloaded content DB using the recognized and translated text as a search word, and displaying the searched content on a display screen of the mobile device.

According to another aspect of the present invention, there is provided a content service method using natural image text recognition, comprising: a common character recognition / translation engine for recognizing and translating texts of natural images of various application fields photographed through a built- 1. A content service method using natural image text recognition in a system for servicing a content related to a natural image text including a mobile device equipped with an information gate app including a content database and a distribution server, A step of recognizing and translating a text of a natural image if language information and object information are selected by a user in a language-specific menu and an object-specific menu provided by an information gateway application installed in the device; Sending the recognized and translated text to the distribution server; The distribution server receiving the recognized and translated natural image text and searching for a content DB corresponding to the natural image text using the received natural image text as a search term; Transmitting the retrieved content to an information gate app of the mobile device; And the information gate app of the mobile device displays the searched content on a display screen.

According to the contents service system and method using natural image text recognition according to the present invention, texts of a natural image photographed by a camera are recognized and translated, and the contents are searched as a search word to provide contents for recognized texts.

Through this, it is possible to obtain a natural image by photographing directly with a camera in various application fields such as signs, signboards, food, general terms, maps, product names, etc., or recognize images already taken by the present invention, By searching the DB, the user can get the desired information in various languages and various application fields very easily.

FIG. 1 shows an example of a general configuration to which a contents service system using natural image text recognition according to the present invention is applied.
2 is a block diagram illustrating a configuration of a content service system using natural image text recognition according to the present invention.
FIG. 3 shows a first embodiment of a contents service system using natural image text recognition according to the present invention.
FIG. 4 shows a second embodiment of a contents service system using natural image text recognition according to the present invention.
5 is a block diagram of a more detailed configuration of the information gate app.
6 shows an example of a language-specific selection menu and a destination-specific selection menu that are provided to a user through a user interface unit provided by the information gate application of the mobile device.
7 shows an example of a content DB.
8 shows a more detailed configuration and function of the management tool server and the distribution server.
FIG. 9 is a flowchart illustrating a first embodiment of a content service method using natural image text recognition according to the present invention.
FIG. 10 is a flowchart illustrating a content service method using natural image text recognition according to a second embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory only and are not restrictive of the invention, It should be understood that various equivalents and modifications may be present.

FIG. 1 illustrates an example of a general configuration to which a content service system using natural image text recognition according to the present invention is applied. The mobile terminal includes a plurality of mobile devices 110 and 120 having information gaps 112 and 122, A management server server 140 that is connected to the distribution server 130 having the content DBs 132 and 134 via the network 100 and updates and manages the content DBs 132 and 134 of the distribution server 130 ). The mobile devices 110 and 120 execute the information gate applications 112 and 122 to capture a natural image, recognize the text included in the natural image, translate the text into another language if necessary, 130, and displays the retrieved contents on the display screen of the mobile devices 110, For example, if a Japanese user travels to Korea and wants to get information about Korean food, he shoots a food menu provided by a Korean restaurant with a camera on a smartphone, recognizes it and translates it into Japanese. As a search word, a Japanese food content DB can be searched to easily obtain contents related to a desired food.

2 is a block diagram illustrating a configuration of a content service system using natural image text recognition according to the present invention. The content service system using natural image text recognition according to the present invention may include a mobile device 200 and a distribution server 240 and may further include a management tool server 250. [

The mobile device 200 is installed with the information gate app 210, and typically can be a smart phone, but it is not limited to a smart phone but includes various mobile devices such as a tablet PC.

The information gate application 210 may include a natural scene image of various application fields photographed through a built-in camera, a preview image currently being photographed by a camera, or an image already stored in a gallery of a mobile device (Not shown) mounted on the mobile device 200, which recognizes and translates the text and provides the contents to the user for the recognized and translated natural image text. That is, the information gate application 210 acquires an image through a camera shooting mode and an image loading mode, and the camera shooting mode provides a preview mode real-time mode and a snap shooting mode. The acquired image recognizes text through natural image recognition and translation software package and provides search and translation results on the screen in connection with the contents DB, and has a screen configuration corresponding thereto.

5 is a block diagram illustrating a more detailed configuration of the information gate app 210 and 500. The information gate app 210 and 500 includes a common character recognition / translation engine 214 and 520 and a user interface unit 212 and 560, Respectively.

The common character recognition / translation engine 214, 520 may include a natural scene image (e.g., a natural scene image) such as a name, a food, a cultural property, a sign, a billboard, And includes a natural image character region extraction unit 522, a multilingual character recognition unit 524, and a multilingual translation unit 526.

The natural image character region extraction unit 522 searches for an area where a character string exists in the natural image, and extracts the character. The natural image character region extracting unit 522 will be described in more detail as follows. The natural image character region extracting unit 522 first converts an input file for a natural image into a form capable of image processing and corrects distortion included in the image. Then, the character region candidate group is selected in the input image, the background image and the character string of the character region are separated from each other, and the extracted character string is separated into individual characters. Then, the combination information of the words, A language model including information and a mistaken pattern information is applied to character verification by language-based character extraction.

The multilingual character recognition unit 524 recognizes the character string of the extracted character area. The multilingual character recognition unit 524 may recognize a character or a logo including the graphic element, extract a corner point and a junction point representing characteristics of the image to be used, Vectorized, and the vectorized image feature information is matched with the image of the image matching DB by referring to the image matching DB to find the most similar image information.

The multilingual translation unit 526 searches the multilingual database and translates the recognized character string into various languages. The multilingual translation unit 526 will be described in more detail as follows. The multilingual translation unit 526 includes a general term translation database in which a common word or a combination of words is matched with a plurality of words or combinations of words, and a specialized term translation database in which a combination of words or words specified is matched one by one . When the recognized character string is the common word or a combination of words, the translation is translated into a plurality of words or combinations of words referring to the general term translation database. If the recognized character string is the combination of the specialized words or words Refers to the specialized term translation database and translates into one word or a combination of words.

The common character recognition / translation engine described above is described in detail in "Natural language multilingual character recognition and translation system and method using a mobile camera ", which is already filed by the applicant of the present invention. Preferably, And recognizes and translates.

Also, the common character recognition / translation engine 214 and 520 can recognize the natural image text using the general term DB and the specialized term DB. The general term DB refers to a database of general terms referenced in post-processing when recognizing a natural image text. The specialized term DB refers to a special application field such as a new address, a billboard, a book information, a person information, Cultural assets, traffic signs, etc., are used as databases. For example, when a signboard is photographed to recognize a text, the text is recognized by referring to characters and a signboard DB which are processed for recognition in a post-processing process. In this case, in the present invention, the general term DB and the specialized term DB used in the natural image text recognition process are downloaded from the distribution server 240 and installed in the mobile device 200, It can also be installed by default on mobile devices.

The user interface units 212 and 560 provide a user-selectable menu 562 for selecting the content information related to the text included in the recognized natural image and a language-specific menu 564 for selecting a translation language do. 6 shows an example of a language-specific selection menu 620 and a destination-specific selection menu 630, which are provided to the user through the user interface unit 212, 560 provided by the information gate application 610 of the mobile device 600 . Referring to FIG. 6, if the user desires to view the content of the recognized text in another language, the user selects a desired language in the language-specific selection menu 620. For example, if a Japanese person travels to Korea and wants to see information about food in Japanese, she shoots the food menu with a camera, recognizes and translates the food, selects the language selection menu in Japanese, selects food in the menu for each destination , Food contents can be seen in Japanese.

The distribution server 240 is provided with content DBs 242 and 244 and the content DBs 242 and 244 are provided for the text of the recognized and translated natural image through the information gate app 210 of the mobile device 200 The content is being stored. 7 shows an example of a contents DB. The contents DB 719 includes a place name DB 712, a book DB 714, a food DB 716, A term DB 718 and a cultural property DB 719. [ Reference numeral 730 denotes only the book DB 732 in Japanese, among the Japanese DB 710. Referring to FIG. 7, if the user desires to know the name of a book in Japanese on the distribution server 240, the Japanese language is selected from the language-specific selection menu and the book is selected from the object- May be downloaded to the mobile device 200 from the distribution server 240.

The management tool server 250 updates the content DBs 242 and 244 included in the distribution server 240 and includes various management tools 252. [ FIG. 8 shows a more detailed configuration and functions of the management tool servers 800 and 250 and the distribution servers 850 and 240. The management tool server 810 includes various management tools 810. The management tool 810 includes a language-specific DB management tool 812, a character recognition engine learning tool 814, a character recognition performance tuning tool 816, And the like. The management tool server 800 distributes the DB managed by the management tool 810 and measures the performance of the character recognition engine 820. [

The language-specific DB management tool 812 manages a recognition / translation DB used to recognize and translate the extracted characters, and provides registration, change, deletion, and inquiry functions necessary for the management. The recognition / translation DB includes a font DB for storing character information used for recognizer learning by font type, an image matching DB consisting of original image information and feature extraction information, a word consisting of word frequency and word combination information A language model DB composed of information of a character, recognition information consisting of a word, an appearance frequency of a word, and a combination of characters, and a recognition pattern information composed of a mistaken pattern of a word and a word, A character recognition model DB for managing a list of recognition models, and a translation DB including a word in the reference language, a foreign language word table having a form of a foreign language word matched with the reference language, and a foreign language word table matched with the reference language table.

The character recognition engine learning tool 814 provides a learning function for generating a character recognition model DB. More specifically, the character recognition engine learning tool 814 generates a learned character recognition model by receiving the font information of the font DB, and uses the boundary information of each character Extracts image features and expresses them as feature vectors, reconstructs the feature vectors by extracting only important elements in the dimension of the feature vectors to reduce the memory and calculation amount used for learning and recognition, To generate a reference vector representing the class.

The character recognition performance tuning tool 816 provides performance tuning. More specifically, the character recognition performance tuning tool 816 uses the DB information received from the language-specific DB management tool and a test sample in a specialized field to acquire DB and engine tuning information And performs tuning of the engine using the selected DB and engine tuning information.

Then, the distribution server 850 classifies the recognition / translation language model required by the user into the content DB using the language information and the destination information selected by the user, and classifies the recognition / translation language model into a search / translation purpose required by the user 852) and transmits the contents DB 854 to the mobile device by the classification. The distribution server 850 preferably performs distribution authentication when distributing the content DB to the mobile device.

FIG. 3 shows a first embodiment of a contents service system using natural image text recognition according to the present invention.

In the first embodiment of the contents service system using natural image text recognition according to the present invention, a content service is performed as follows.

When the distribution server 350 receives the destination information and the language information selected by the user in the menu for each destination and the menu for each language in the user interface of the information gate application 310, DB to be downloaded by the mobile device 300 on which the information gate app 310 is installed. For example, when a user selects Japanese from a language-specific menu, Japanese becomes language information, and when a user selects a book from a target-specific menu, the book becomes the target information, and the distribution server selects the book DB in Japanese as the content DB So that the mobile device 300 can be downloaded.

Meanwhile, the information gate application 310 of the mobile device 300 transmits the destination information and the language information selected by the user to the distribution server 350 through the menu for each purpose and the menu for each language of the user interface, Recognizes and translates the natural image text through the recognition / translation engine 214, downloads the contents DB 350 from the distribution server 350, and uses the recognized natural image text as a search word to download the downloaded content DB 320 and displays the content corresponding to the search word on the screen of the mobile device 300. [ In the above-described first embodiment, the content DB used in the information gate app resides in the distribution server, and the content DB in the distribution server can be loaded through the information gate application environment setting at the time when the user uses the content DB . At this time, the user may be allowed to pay the content DB fee.

FIG. 4 shows a second embodiment of a contents service system using natural image text recognition according to the present invention. In the second embodiment of the contents service system using natural image text recognition according to the present invention, a contents service is performed as follows.

The distribution server 450 receives the natural image text recognized through the common character recognition / translation engine of the information gate application 410 and the user interface menu of the information gateway application 410, When the selected object information and the language information are received, the contents DB 460 corresponding to the received object information and the language information is retrieved using the natural image text as a search word, To the mobile device 400 in which the mobile device 400 is installed.

The information gate application 410 of the mobile device 400 transmits the destination information and the language information selected by the user to the distribution server 450 through the menu for each purpose and the menu for each language of the user interface, And receives the contents corresponding to the natural image text from the distribution server 450 and displays the contents on the screen of the mobile device 400. [

FIG. 9 is a flowchart illustrating a first embodiment of a content service method using natural image text recognition according to the present invention.

A mobile device equipped with an information gate app including a common character recognition / translation engine for recognizing and translating texts of natural images taken by a built-in camera through various application fields, and a distribution server having a content DB The method of contents service using natural image text recognition in a system for servicing contents related to natural image text is as follows.

First, when language information and destination information are selected by the user in the menu for each language and the menu for each object provided by the information gate app installed in the mobile device (step S900), the language information and destination information are transmitted to the distribution server, The text of the natural image is recognized and translated (step S910)

The contents DB corresponding to the recognition and translation is downloaded from the distribution server to search contents corresponding to the recognized and translated text (step S920)

The downloaded content DB is retrieved using the recognized and translated text as a search word in step S930 and the searched content is displayed on the display screen of the mobile device in step S940.

FIG. 10 is a flowchart illustrating a content service method using natural image text recognition according to a second embodiment of the present invention.

A mobile device equipped with an information gate app including a common character recognition / translation engine for recognizing and translating texts of natural images taken by a built-in camera through various application fields, and a distribution server having a content DB A content service method using natural image text recognition in a system for servicing contents related to a natural image text is performed as follows.

When the language information and the destination information are selected by the user in the menu for each language and the menu for the object provided by the information gate app installed in the mobile device (step S1000), the text of the natural image is recognized and translated (step S1010) The information gateway app transmits the recognized and translated text to the distribution server (step S1020). The distribution server receives the recognized and translated natural image text and uses the received natural image text as a search word, The content DB corresponding to the video text is retrieved (step S1030). The retrieved content is transmitted to the information gate app of the mobile device (step 1040). The information gateway app of the mobile device displays the retrieved content on the display screen (Step S1050)

The present invention can be embodied as a computer readable code on a computer-readable recording medium (including all devices having an information processing function). A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of computer-readable recording devices include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

100: network 110, 120: mobile device
112, 122: information gate application 130: distribution server
132, 134: content DB 130: management tool server
200: Mobile device 210: Information Gate App
212: user interface unit 214: common character recognition / translation engine
240: distribution server 242, 244: contents DB
250: Administration Tools Server 252: Administration Tools
300: Mobile Device 310: Information Gate App
320: Content DB 350: Distribution server
360: Content DB 400: Mobile device
410: Information Gate App 450: Distribution Server
460: Content DB 500: Information Gate App
520: common character recognition / translation engine 522: natural image character area extraction unit
524: Multilingual character recognition unit 526: Multilingual translation unit
560: User interface part 562: Language selection menu
564: Purpose-specific selection menu 600: Mobile device
610: Information Gate Apps 620: Language-specific selection menu
630: Selection menu for each purpose 710: Japanese DB
712: Placement DB 714: Book DB
716: Food DB 718: General term DB
719: Cultural assets DB 730: Japanese DB
732: Book DB 800: Administration Tools Server
810: Administration Tools 812: Language-specific DB Administration Tools
814: Character Recognition Engine Learning Tool 816: Character Recognition Performance Tuning Tool
820: Deployment Management & Performance Measurement 850: Deployment Server
852: Classification by language & purpose 854: Contents DB

Claims (10)

A mobile device installed with an information gate app including a common character recognition / translation engine for recognizing and translating texts of natural images of various application fields photographed through a built-in camera, and a user interface; And
And a distribution server including a content DB storing contents corresponding to the recognized and translated natural image text,
The user interface of the information gate app
Providing a menu for each object and a language-based menu for selecting a translation language, the contents information relating to the recognized natural image text,
The distribution server
When receiving the destination information and the language information selected by the user in the menu for each purpose and the menu for each language in the user interface of the information gate app, the content DB corresponding to the received destination information and the language information, Provide mobile devices for download,
The information gat app of the mobile device
Transmitting the object information and language information selected by the user to the distribution server through a menu for each object and a menu for each language of the user interface, recognizing and translating the natural image text through the common character recognition / translation engine, Wherein the contents DB is downloaded from the contents database, and the contents DB is searched by using the recognized natural image text as a search word, and a content corresponding to the search word is displayed. 2. The method of claim 1, wherein the common character recognition / translation engine of the information gate app
A natural image character region extracting unit for searching a region where a character string exists in the natural image to extract a character;
A multilingual character recognition unit for recognizing a character string of the extracted character area; And
And a multilingual translation unit for searching the multilingual database and translating the recognized character string into various languages. The method according to claim 1,
Further comprising a management tool server for updating a content DB included in the distribution server. A mobile device installed with an information gate app including a common character recognition / translation engine for recognizing and translating texts of natural images of various application fields photographed through a built-in camera, and a user interface; And
And a distribution server including a content DB storing contents corresponding to the recognized and translated natural image text,
The user interface unit of the information gate app
Providing a menu for each object and a language-based menu for selecting a translation language, the contents information relating to the recognized natural image text,
The distribution server
Upon receiving the natural image text recognized through the common framework of the information gate application and the object information and the language information selected by the user in the object-specific menu and the language-specific menu provided by the user interface unit of the information gate application, Searching a content DB corresponding to the received destination information and language information using a text as a search word, providing a content corresponding to the search word to a mobile device equipped with the information gate app,
The information gat app of the mobile device
Transmitting the object information and language information selected by the user to the distribution server through a menu for each object and a menu for each language of the user interface, recognizing and translating the natural image text through the common character recognition / translation engine, And displaying the content on the screen of the mobile device. The content service system according to claim 1, 5. The system of claim 4, wherein the common character recognition / translation engine
A natural image character region extracting unit for searching a region where a character string exists in the natural image to extract a character;
A multilingual character recognition unit for recognizing a character string of the extracted character area; And
And a multilingual translation unit for searching the multilingual database and translating the recognized character string into various languages. A common character recognition / translation engine for recognizing and translating texts of natural images of various applications photographed through a built-in camera; And
And a user interface unit for providing a menu for each object and a language-based menu for selecting a translation language, the contents of the text recognized through the common character recognition / translation engine,
And an information gate app for displaying contents corresponding to the recognized and translated text is mounted.
The Information Gate app
Recognizing and translating the text of the natural image through the character recognition / translation engine,
Downloading a content DB corresponding to the recognition and translation from the distribution server to retrieve content corresponding to the recognized and translated text
Searching the downloaded content DB using the recognized and translated text as a search word, and displaying the searched content on a display screen. The method of claim 6, wherein the common character recognition / translation engine of the information gate app
A natural image character region extracting unit for searching a region where a character string exists in the natural image to extract a character;
A multilingual character recognition unit for recognizing a character string of the extracted character area; And
And a multilingual translation unit for searching the multilingual database and translating the recognized character string into various languages. A common character recognition / translation engine for recognizing and translating texts of natural images of various applications photographed through a built-in camera; And
And a user interface unit for providing a menu for each object and a language-based menu for selecting a translation language, the contents of the text recognized through the common character recognition / translation engine,
And an information gate app for displaying contents corresponding to the recognized and translated text is mounted.
The Information Gate app
The text of the natural image is recognized and translated through the character recognition / translation engine to transmit the recognized and translated natural image text, the destination information and the language information to the distribution server,
Wherein the distribution server receives the content and displays it on a display screen when the distribution server searches the content DB using the text of the natural image as a search word and displays the content on a display screen. . A mobile device equipped with an information gate app including a common character recognition / translation engine for recognizing and translating texts of natural images taken by a built-in camera through various application fields, and a distribution server having a content DB A method of providing a content using natural image text recognition in a system for providing contents related to a natural image text,
Recognizing and translating the text of the natural image if the language information and the destination information are selected by the user from the menu for each language and the menu for each object provided by the information gate app installed in the mobile device;
Downloading a content DB corresponding to the recognition and translation from the distribution server to retrieve content corresponding to the recognized and translated text;
And displaying the retrieved contents on a display screen of the mobile device by searching the downloaded contents DB using the recognized and translated text as a search word. A mobile device equipped with an information gate app including a common character recognition / translation engine for recognizing and translating texts of natural images taken by a built-in camera through various application fields, and a distribution server having a content DB A method of providing a content using natural image text recognition in a system for providing contents related to a natural image text,
Recognizing and translating the text of the natural image if the language information and the destination information are selected by the user from the menu for each language and the menu for each object provided by the information gate app installed in the mobile device;
Sending the recognized and translated text to the distribution server;
The distribution server receiving the recognized and translated natural image text and searching for a content DB corresponding to the natural image text using the received natural image text as a search term;
Transmitting the retrieved content to an information gate app of the mobile device;
Wherein the information gating application of the mobile device displays the searched content on a display screen.

Images