KR20120063127A - Mobile terminal with extended data - Google Patents

Abstract

PURPOSE: A data extension portable terminal is provided to enable a user to conveniently search information from the Internet by receiving a click signal related to data. CONSTITUTION: A control unit(110) displays image data transmitted by a camera(300) on a display unit(200). The display unit includes a character recognition unit for recognizing character. The control unit displays words on the display unit by recognizing the words from the character recognition unit. When the displayed word is pushed in the display unit, multimedia data is displayed on the display unit. The display unit is divided into the character recognition unit and a background unit. The background unit is darker than the character recognition unit.

Description

Mobile terminal with extended data

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data expansion portable terminal, and more particularly, to a data expansion portable terminal which displays multimedia data automatically retrieved through the Internet by recognizing image data.

The multimedia-based learning method for improving the information recognition ability of human beings has been gaining much acclaim since it was proved effective by Richard Meyer in 1977.

In order to implement the recognition theory (CTML) of multimedia learning on a mobile terminal, a character recognition technology is required.

OCR (Optical Character Recognition) technology refers to a technology for converting and storing Korean, English and numeric fonts included in a scanned image into editable text. Such character recognition technology has an advantage that the input operation of the computer can be directly performed without other input devices by automatically reading and identifying printed or handwritten characters in one field of pattern recognition and converting them into codes that can be understood by the computer.

Recently, high-resolution cameras are often built into mobile terminals such as smartphones and tablet PCs, and a tool for text recognition of images taken by the camera has been developed and commercialized.

However, since the character recognition technology has been developed under the assumption that it is used in a fixed device such as a desktop PC, the method of using the character recognition technology on the premise of using it in a device capable of maximizing mobility such as a mobile terminal In terms of development, there is little development.

In the past, text recognition technology was used in PDA, which can be called a mobile terminal, but in PDA, handwriting recognition technology was mainly used rather than camera, and the focus was on increasing handwriting recognition rate.

The present invention was devised to solve the above problems, and it is an extended data portable device which enables a user to more conveniently search for data related to a word recognized by the character recognition method and the character recognition method for maximizing mobility of the mobile terminal. The purpose is to provide a terminal.

In order to achieve the above object, the present invention provides a portable terminal comprising a camera for generating image data based on an external image, a control unit for controlling the image data, and a display unit for displaying a screen by the control unit. The control unit displays the image data transmitted by the camera on the display unit, the display unit is a character recognition unit which is a part for character recognition is provided in the display unit, the control unit is a single word inside the character recognition unit The present invention provides a data expansion portable terminal which recognizes and displays on the display unit, and multimedia data searched through the Internet is displayed on the display unit when a word displayed on the display unit is pressed.

The display unit may be divided into a character recognition unit which is a part for character recognition and a background unit which is a part except for the character recognition unit, and the background unit is preferably displayed with a lighter brightness than the character recognition unit.

Here, the letter recognition unit displays the top, bottom, left and right center lines, and can recognize a word closest to the center line.

Frames based on the image data transmitted by the camera are stored in the storage unit, and the controller receives the character recognition result data by the predetermined number of frames among the frames stored in the storage unit, and recognizes the most recognized character among them. It is preferable to display the result data in the display section.

In another embodiment, the present invention provides a portable terminal having a control unit for controlling an image containing a character and a display unit for displaying a screen by the control unit, the control unit recognizes a character portion in the image and extracts a word; The present invention provides a data expansion portable terminal, which displays the multimedia data searched through the Internet using the word as a search word.

Preferably, the multimedia data is an image file, and the image file is displayed above the word.

The controller may calculate a distance from the center of the display unit to the image file and display the size of the image file larger as the distance from the center of the display unit is closer.

In the display unit, a range of the character recognition unit, which is a part for character recognition, is determined by the user.

In the character recognition module, the character recognition module divides the character recognition unit into a predetermined number of character recognition packets to perform character recognition, and the character recognition word is transmitted to the internet search module to start an internet search process. It is preferable to start the character recognition process of the next character recognition packet.

According to the present invention, information about a desired part can be easily obtained in real time, and related data can be received with a single click, thereby facilitating internet search anytime and anywhere.

In addition, according to the present invention, when a book or a document is read, a related image is automatically displayed on the document with a single click, so that the user can have fun reading and easily retrieve related data.

In addition, text recognition and Internet search results are displayed very quickly, so that users do not feel bored.

1 is a block diagram showing a basic configuration of a data expansion portable terminal according to the present invention;
2 is a flowchart illustrating a processing procedure of a data expansion portable terminal according to the present invention;
3 is an exemplary view showing an embodiment of use of a data expansion portable terminal according to the present invention;
4 is an exemplary view showing an example of use following FIG. 3;
5 is a flow chart showing another embodiment of a data expansion portable terminal according to the present invention;
6 is an exemplary view showing an embodiment of use according to the procedure of FIG. 5;
7 is an explanatory diagram for explaining a character recognition process before FIG. 6;
8 is an exemplary view showing an example of use of a data expansion portable terminal according to the present invention.

The configuration and operation of the embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Referring to Figure 1 describes the basic configuration of a data expansion portable terminal according to the present invention.

The data expansion portable terminal according to the present exemplary embodiment includes a control unit 110, a display unit 200, a camera 300, a storage unit 400, and a wireless communication unit 500.

The control unit 110 may be configured as a central processing unit such as a CPU, and serves to control most functions of the portable terminal, such as operation of various applications and multitasking. On the other hand, the control unit 110 is provided with a character recognition module 112 and the Internet search module 114.

The character recognition module 112 may be used in various ways, and generally includes a preprocessing step, a document structure analysis and registration step, a character separation step, a pattern matching step, and a post processing step.

Here, the preprocessing step refers to an operation of correcting the tilt of an optically or electrically converted character or removing miscellaneous items such as shadows. In addition, the document structure analysis and registration step refers to a process of separating, ordering, or defining a document format from images and letters (numbers) that are components of a document. The character separation step refers to a process of extracting an image of each character string, separating each row or word, and then separating each individual character again. The pattern matching step refers to a task of extracting candidate characters and calculating reliability by classifying them into models having the most similar features using a pattern matching algorithm in each of the separated character images. The post-processing step refers to a task of correcting a recognition error or a recognition error by cross-reference by words or surveys based on a dictionary.

On the other hand, the Internet search module 114 receives the text data recognized from the character recognition module 112 serves to perform an Internet search using the text data as a search word.

Specifically, the Internet search module 114 performs a search using a predetermined search portal such as Google, Naver or Daum using the text data as a search word. At this time, if necessary, the Internet search module 114 may transmit and receive data wirelessly through the wireless communication unit 500.

The search results are stored in the storage unit 400 divided by types such as images, videos, and dictionaries.

The camera 300 generates image data by converting an image input through a lens into an electrical signal. Image data generated by the camera 300 is processed by the controller 110.

The display unit 200 serves to display the content output from the control unit 110 so that the user can see it, and may be composed of LCD, OLED, etc., and at the same time, the touch screen is preferably provided.

The storage unit 400 is a part in which various data are temporarily or permanently stored, and an S-RAM for temporarily storing image data generated by the camera 300, and text data recognized by the character recognition module 112. The stored FLASH-ROM, the basic program necessary for the overall operation of the portable terminal, and the EEP-ROM that stores various parameters used in the mobile communication network may be included.

Referring to Fig. 2, a processing method inside the data expansion portable terminal according to the present embodiment will be described.

When the present embodiment starts, the camera 300 is opened and the camera 300 is operated. When the camera is activated, an external image acquires image data at a rate of 30 frames per second. In addition, the generated frame is stored in the storage unit 400, and when a predetermined amount of data is filled in the storage unit 400, a frame added while the previous frame is deleted is newly stored.

The display unit 200 displays image data acquired from the camera 300. In addition, the controller 110 selects a frame stored in the storage 400 at regular intervals.

For example, the frame may be selected from five frames per second, the interval can be changed according to the actual use environment.

Referring to FIG. 3, in the present embodiment, the display unit 200 is divided into a character recognition unit 210 that is a portion where character recognition is performed and a background unit 220 that is a portion where character recognition is not performed.

The character recognition unit 210 is preferably located in the center portion of the display unit 200.

The character recognition unit 210 is displayed brightly and clearly so that the user can see clearly. On the other hand, the background unit 220 has a sharpness lower than that of the character recognition unit 210 and is displayed darker.

In this way, the display unit 200 is divided into the character recognition unit 210 and the background unit 220 to enable real-time character recognition, and the user can know the portion of the character recognition clearly by changing the brightness display.

Horizontal and vertical center lines are formed in the character recognition unit 210, and the character recognition is performed on a character located at or near the intersection of the center lines. That is, since only one word is recognized inside the character recognition unit 210, character recognition can be performed at a very high speed, so that the character recognition is performed in real time without the user pressing the shutter of the camera 300. The center line may be displayed on the display unit 200 to be viewed by the user, or may be used only for calculation in the controller 110 without being viewed from the outside.

On the other hand, in the present embodiment, since the character recognition is performed in a plurality of selected frames in real time, the possibility of the character recognition result changing in real time increases. In such a situation, if a character recognition result that changes from time to time is exposed to the display unit, the user may have difficulty in selecting which result, and there is a possibility of incorrectly selecting the character recognition result.

Therefore, in the present embodiment, text data, which is a result of character recognition for each frame, is sequentially stored in a storage unit temporarily stored. Then, the controller 110 selects the character recognition result data that most matches based on a predetermined number (for example, five) among the stored character recognition result data, and displays the character recognition button 240 on the display unit 200. . The controller 110 deletes the previous character recognition result data and repeats the comparison including the character recognition result data added later.

In this way, the user can see the character recognition result that is stably displayed, not the character recognition result that changes rapidly.

Next, when the user clicks the text recognition button 240, the text recognition result data is transmitted to the Internet search module 114, the display unit 200 of the image 232, video 234 and Wikipedia 236 Three menus are displayed. In addition, the Internet search module 114 performs an Internet search through the wireless communication unit 500.

The Internet search may be searched through Google, Naver or the following portal sites, and the search is performed based on the text data of the character recognition button 240.

The Internet search results are searched for multimedia data related to the character recognition result data, and are divided into an image file such as JPG, a video such as WMV or MP4, and stored in the storage unit 400 divided into an encyclopedia wikipedia. When the user selects and clicks one of the image 232, the video 234, and the Wikipedia 236, the related data is displayed on the display unit 200 as shown in FIG. 4. The video may be transmitted in a streaming manner.

Next, with reference to Figs. 5 and 6, a description will be given of a mobile terminal which automatically expands data for words described in a book or document.

In the present embodiment, a book or a document may be used as a picture file photographed using a camera or a picture file obtained through another path.

When the present embodiment starts, an image for data expansion is selected. In this case, an image stored in the storage unit 400 may be selected, and when the user photographs by pressing the shutter by using a camera, the captured image is automatically selected.

If an image is selected, the selected image is displayed on the display unit 200. After that, the character recognition unit 210 which is a part to be recognized is set. The setting of the character recognition unit 210 is preferably set by the multi-touch function of the display unit 200.

When the character recognition unit 210 is set, the character recognition unit is divided into a plurality of predetermined character recognition packets. For example, the character recognition packet may divide the character recognition unit 210 into a predetermined interval up and down, and each interval may be a character recognition packet. In the present embodiment, the character recognition module 112 scans how many character lines are present in the character recognition unit 210 set first. As shown in FIG. 7, portions corresponding to each line are divided into character recognition packets 211, 213, 215, 217, and 219.

After being divided into a plurality of character recognition packets, character recognition is performed for each character recognition packet. After the character recognition is completed, the part of speech of the recognized word is extracted. The character recognition module 112 selects only the nouns of the extracted parts of speech and transmits them to the Internet search module 114. Meanwhile, the internet search module 114 performs an internet search based on the text data received from the character recognition module 112.

More specifically, when the character recognition of the first character recognition packet 211 is completed in the character recognition module 112, the character recognition module 112 extracts the parts of speech of the recognized words to search the text data of the words determined as nouns on the Internet. The character recognition of the second character recognition packet 213 is started simultaneously with the transmission to the module 114.

In this way, since the character recognition of the second character recognition packet 213 and the Internet search of the nouns included in the first character recognition packet 211 is performed at the same time, the user can quickly see the search results and do not feel bored.

The controller 110 displays a related image on the corresponding word among the results searched by the internet search module 114. At this time, the center of the upper, lower, left and right is formed in the character recognition unit 210, the distance between the center of the character recognition unit 210 and the corresponding word is calculated based on the center line, the related image is sized according to the distance Is adjusted. That is, the farther the distance is, the smaller the size of the related image is. The closer the distance is, the larger the size of the related image is displayed. This makes it easier for the user to click the related images and to make them visually appealing.

When the related image is clicked, the contents of the image file are displayed, and when the corresponding word of the display unit 200 is clicked, the display unit 200 displays three menus of an image 232, a video 234, and a Wikipedia 236. . When the user selects and clicks one of the image 232, the video 234, and the Wikipedia 236, the relevant data is displayed on the display unit 200.

In the data expansion portable terminal according to the present embodiment, as shown in FIG. 8, an image selected by a user is automatically stored and can be viewed again later with a single click. In addition, it is possible to listen to the voice by adding the TTS (Text To Speech) function.

110: control unit 112: character recognition module
114: Internet search module 200: display unit
210: character recognition unit 220: background unit
300: camera 400: storage
500: wireless communication unit

Claims (8)

In the portable terminal having a camera for generating image data based on an external image, a control unit for controlling the image data and a display unit for displaying a screen by the control unit,
The control unit displays image data transmitted by the camera on the display unit,
The display unit is a character recognition unit which is a part for character recognition is provided in the display unit,
The control unit recognizes only one word inside the character recognition unit and displays the word on the display unit.
And when the word displayed on the display is pressed, multimedia data retrieved through the Internet is displayed on the display. The method of claim 1,
The display unit may be divided into a character recognition unit which is a part for character recognition and a background unit which is a part except for the character recognition unit, and the background unit may be displayed with a lighter brightness than the character recognition unit. The method of claim 2,
The character recognition unit displays the top, bottom, left and right center lines, and recognizes the word closest to the center line. The method of claim 1,
Frames based on the image data transmitted by the camera are stored in the storage unit, and the controller receives the character recognition result data by the predetermined number of frames among the frames stored in the storage unit, and recognizes the most recognized character among them. And a resultant data on the display unit. In a portable terminal having a control unit for controlling an image containing a character and a display unit for displaying a screen by the control unit,
And the controller extracts a word by recognizing a character part in the image, and displays the multimedia data searched through the Internet on the display unit using the word as a search word. The method of claim 5,
And said multimedia data is an image file, said image file being displayed on said word. The method of claim 6,
And the control unit calculates a distance from the center of the display unit to the image file, and displays a larger size of the image file as the distance from the center of the display unit is closer. The method of claim 5,
In the display unit, a range of the character recognition unit, which is a part for character recognition, is determined by the user.
In the character recognition module, the character recognition module divides the character recognition unit into a predetermined number of character recognition packets to perform character recognition, and the character recognition word is transmitted to the internet search module to start an internet search process. And a character recognition process for the next character recognition packet.

Images