WO2011122717A1 - Method for providing text relation information using a mobile terminal, and mobile terminal - Google Patents
Method for providing text relation information using a mobile terminal, and mobile terminal Download PDFInfo
- Publication number
- WO2011122717A1 WO2011122717A1 PCT/KR2010/001916 KR2010001916W WO2011122717A1 WO 2011122717 A1 WO2011122717 A1 WO 2011122717A1 KR 2010001916 W KR2010001916 W KR 2010001916W WO 2011122717 A1 WO2011122717 A1 WO 2011122717A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- text
- related information
- image
- mobile terminal
- character
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00127—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/60—Editing figures and text; Combining figures or text
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00127—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
- H04N1/00204—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture with a digital computer or a digital computer system, e.g. an internet server
- H04N1/00244—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture with a digital computer or a digital computer system, e.g. an internet server with a server, e.g. an internet server
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00127—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
- H04N1/00326—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus
- H04N1/00328—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus with an apparatus processing optically-read information
- H04N1/00331—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus with an apparatus processing optically-read information with an apparatus performing optical character recognition
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
- H04N2201/0077—Types of the still picture apparatus
- H04N2201/0084—Digital still camera
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
- H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
- H04N2201/3201—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
- H04N2201/3225—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of data relating to an image, a page or a document
- H04N2201/3245—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of data relating to an image, a page or a document of image modifying data, e.g. handwritten addenda, highlights or augmented reality information
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
- H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
- H04N2201/3201—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
- H04N2201/3261—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of multimedia information, e.g. a sound signal
- H04N2201/3266—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of multimedia information, e.g. a sound signal of text or character information, e.g. text accompanying an image
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
- H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
- H04N2201/3201—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
- H04N2201/3273—Display
Definitions
- the present invention relates to a method for providing text-related information using a mobile terminal and a mobile terminal, and more particularly, to a mobile terminal for providing related information on a text included in an original image image obtained through a camera provided in the mobile terminal.
- the present invention relates to a character information providing method and a portable terminal.
- portable terminals are used in various locations and environments, and thus various types of information (for example, movement, position / direction information, and image image information acquired through a camera) as well as input using a button key Etc.) can be used as a useful input.
- information for example, movement, position / direction information, and image image information acquired through a camera
- Etc. button key
- the portable terminal needs to provide various information to the user by using the advantages of the portable by displaying the information in the form of augmented reality in which virtual reality and real reality are mixed using various types of inputted information. There is.
- the portable terminal needs to perform optimized processing on various types of information (for example, an image image acquired through a camera) in consideration of the speed of the network, the processing speed of the terminal, and the like.
- An object of the present invention is to extract a character from the selected text area of the original image image obtained through the camera, to search for the character-related information corresponding to the extracted character through the character-related information server, and to retrieve the retrieved character-related information from the original image image
- the present invention provides a method and a portable terminal for providing text-related information using a portable terminal synthesized with the display.
- a method for providing text-related information using a mobile terminal wherein the mobile terminal captures an image through a camera to obtain an original image image, and receives a text area selected from the original image image. Extracts the text from the selected text area, and transmits the extracted text information to the text related information server, wherein the text related information server refers to the text related information database to search for text related information corresponding to the text information, and retrieves the found text. And transmitting the related information to the mobile terminal and displaying the text related information transmitted by the mobile terminal on the original image image.
- the mobile terminal acquires an original video image by capturing an image through a camera, selecting a text area from the original video image, and selecting the text area. Extracting the character feature from the character feature, transmitting the extracted character feature information to the character related information server, extracting the character according to the transmitted character feature information, and the character related information server refer to the character related information database And searching for text related information corresponding to the extracted text, transmitting the searched text related information to the mobile terminal, and displaying the text related information transmitted by the mobile terminal on the original image image.
- the mobile terminal captures an image through a camera to obtain an original video image, and the mobile terminal receives a selected text area from the original video image, Sending the information to the text related information server, extracting the text from the text area transmitted by the text related information server, and searching the text related information corresponding to the extracted text by the text related information server.
- the method may include transmitting the retrieved text related information to the mobile terminal and displaying the text related information transmitted by the mobile terminal on the original image image.
- the portable terminal is to capture the image through the camera to obtain the original image image, if the text area is selected from the original video image, to transmit information corresponding to the selected text area to the text-related information server And a display unit for synthesizing and displaying the text related information received from the text related information server corresponding to the information corresponding to the text area to the original video image.
- the text related information server is a text related information database for storing one or more text information and the text related information corresponding to each character information, information corresponding to the selected text area of the original image image obtained by the mobile terminal And a receiving unit for receiving the information, a related information searching unit for searching for text related information corresponding to the received text information with reference to the text related information database, and a related information providing unit for transmitting the retrieved text related information to the mobile terminal.
- character-related information corresponding to a character included in a selected text area of an original image image acquired through a camera is synthesized and displayed by an augmented reality technique.
- FIG. 1 is a diagram illustrating a text-related information providing system using a mobile terminal according to an embodiment of the present invention.
- FIG. 2 is a diagram showing the internal configuration of a text-related information server according to an embodiment of the present invention.
- FIG. 3 is a diagram illustrating an internal configuration of a portable terminal according to an embodiment of the present invention.
- FIG. 4 is a diagram illustrating an internal configuration of a character extracting unit according to an embodiment of the present invention.
- FIG. 5 is a diagram illustrating an original video image screen and a text related information providing screen according to an exemplary embodiment of the present invention.
- FIG. 6 is a flowchart illustrating a method of providing text-related information using a mobile terminal according to an embodiment of the present invention.
- FIG. 7 is a flowchart illustrating a text-related information providing method using a mobile terminal according to another embodiment of the present invention.
- FIG. 8 is a flowchart illustrating a text-related information providing method using a mobile terminal according to another embodiment of the present invention.
- FIG. 1 is a diagram illustrating a text-related information providing system using a mobile terminal according to an embodiment of the present invention.
- a text related information providing system using a portable terminal includes a portable terminal 100, a text related information server 200, and a personal computer 150.
- the mobile terminal 100 may communicate with the text-related information server 200 through the mobile communication network
- the text-related information server 200 may communicate with the personal computer 150 through the Internet network.
- the mobile terminal 100 may acquire an original image image by capturing an image through a camera, and receive a text area selected from the original image image by the user.
- the mobile terminal 100 extracts text from the selected text area, transmits the extracted text information to the text related information server, receives text related information corresponding to the text extracted from the text related server, and receives the received text.
- the relevant information can be synthesized and displayed on the original video image.
- the text related information server 200 may receive text information from the original video image from the mobile terminal 100 through the mobile communication network, search for text related information corresponding to the received text information, and transmit the text information to the mobile terminal 100. have.
- the text-related information server 200 may acquire one or more text information and text-related information corresponding to each text by the personal computer 150 connected to the Internet network or automatically search the Internet and store the text-related information in a database. have.
- the text related information server 200 when the text related information server 200 receives the text area image or the text feature information from the portable terminal 100, the text related information server 200 extracts the text information from the received text area image or the text feature information and corresponds to the extracted text information. You can search for text related information.
- FIG. 2 is a diagram illustrating an internal configuration of a text related information server according to an exemplary embodiment of the present invention.
- the text related information server 200 may include a text information receiver 201, a video image receiver 202, a text extractor 204, a related information searcher 206, The related information providing unit 208 and the character related information database 210 are included.
- the text information receiving unit 201 may receive text information extracted from a text area or an important feature necessary for extracting text from the original image image, that is, text feature information from the portable terminal. In this case, when the mobile terminal extracts text from the original video image and provides the text information to the text information server, the text information server does not need to perform the text extraction function.
- the video image receiving unit 202 may receive a text area image that is a text area portion of the original video image from the portable terminal. In addition, the video image receiving unit 202 may receive the entire original video image from the portable terminal.
- the character extractor 204 may extract text data (ie, character information) from the received character region image.
- the character extracting unit 204 is a pre-processing module for sharpening the contour by adjusting the brightness or contrast of the received text area image, a text extraction module for extracting the text data from the received text area and the text code of the text data It may include a post-processing execution module for extracting bays.
- the preprocessing module may perform a deblurring process of restoring the shaken image.
- the character extractor 204 may extract a character from the received character feature information.
- the related information retrieval unit 206 may search for text related information corresponding to the text information extracted from the text information received from the portable terminal or the text area image.
- the related information providing unit 208 may transmit the retrieved text related information to the mobile terminal, and the text related information may be provided as a virtual image that can be inserted into the original video image by an augmented reality technology.
- the text related information database 210 may collect one or more text and text related information corresponding to each text through a personal computer connected by an internet network, and store the data in advance or automatically search and store the Internet.
- FIG. 3 is a diagram illustrating an internal configuration of a mobile terminal according to an embodiment of the present invention.
- the mobile terminal 100 includes a wireless communication unit 302, a camera unit 304, an operation unit 306, a character extracting unit 308, a control unit 310, and a display unit ( 312, a sound output unit 314, a storage unit 316, and a power supply unit 318.
- the wireless communication unit 302 may include a mobile communication module for transmitting and receiving wireless signals through a mobile communication network and a wireless internet module for wireless internet access.
- the wireless communication unit 302 may transmit text information corresponding to the extracted text to a text related information server and receive text related information corresponding to the extracted text searched by the text related server.
- the camera unit 304 processes image frames such as still images or moving images obtained by the image sensor in the video call mode or the photographing mode. That is, the camera unit 304 may obtain an original image by capturing an image through a camera. In this case, the processed image frame may be displayed on the display unit 312.
- the operation unit 306 generates key input data input by the user for controlling the operation of the terminal.
- the operation unit 306 may be configured of a keypad, a dome switch, a touch pad, and the like.
- the touch pad may be implemented as a touch screen having a mutual layer structure with the display unit 312 described later.
- the display unit 312 may display the original image image acquired by the camera unit, and receive a text area selected from the original image image by the user.
- the text extractor 308 extracts text from a selected text area of the original video image.
- the character extracting unit 308 may perform a preprocessing module for sharply processing the contour by adjusting the brightness or contrast of the selected text area, a text extraction module for extracting text data from the selected text area, and extracts only a text code from the text data. It may include a post processing module.
- the preprocessing module may perform a deblurring process of restoring the shaken image.
- the controller 310 controls the overall operation of the mobile terminal, and may perform related control and processing for voice call, data communication, video call, and the like.
- the controller 310 may extract text from the selected text area and perform related control and processing to transmit the extracted text information to the text information server.
- the controller 310 may be configured to extract a text feature from the selected text area and transmit the extracted text feature information to a text related information server.
- the controller 310 may control to transmit the selected text area image to the text related information server.
- the controller 310 analyzes the synthesis position of the virtual image corresponding to the text related information transmitted from the text related information server based on the original video image, synthesizes the virtual image to the analyzed position of the original video image,
- the display unit outputs the augmented reality technique.
- the display unit 312 displays and outputs information processed by the portable terminal. For example, when the mobile terminal is in a call mode, a user interface (UI) or a graphic user interface (GUI) related to a call is displayed. In addition, when the mobile terminal is in a video call mode or a photographing mode, each of the captured and received images may be displayed simultaneously or simultaneously.
- UI user interface
- GUI graphic user interface
- the display unit 312 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, or a three-dimensional display. It may include at least one of (3D display).
- the display unit 312 synthesizes and displays the retrieved text related information to the original video image. At this time, the display unit 312 synthesizes a virtual image corresponding to the character-related information retrieved in the original image image, and displays the image by the augmented reality technique. In addition to the augmented reality technique, the display unit 312 may display the character related information in various ways, for example, a method of displaying an additional screen including the character related information on the original image image.
- the sound output unit 314 may output audio data received through the wireless communication unit 302 or stored in the storage unit 316 in a call mode, a broadcast reception mode, or the like.
- the sound output unit 314 may output a preset sound at the point of time of synthesizing and displaying the text related information searched on the original video image. That is, the sound output unit 314 may output the transmitted text related information in a voice or sound form through a text to speech (TTS).
- TTS text to speech
- the sound output unit 314 may receive and output voice or sound information corresponding to the virtual image from the text related information server.
- the storage unit 316 may store a program for processing and controlling the controller 310, and may temporarily store input or output data.
- the power supply unit 318 may receive the external power and the internal power by the control of the controller 110 to supply the power necessary for the operation of each component.
- FIG. 4 is a diagram illustrating an internal configuration of a character extracting unit according to an embodiment of the present invention.
- the character extracting unit may include a preprocessing module 402, a character extraction module 404, and a postprocessing module 406.
- the preprocessing module 402 may perform preprocessing to sharpen the outline by adjusting the brightness or contrast of the selected text area so as to increase the recognition rate in the obtained original video image.
- the preprocessing module 420 may perform a deblurring process of restoring the shaken image.
- the text extraction module 404 may extract text data from the selected text area (ie, a text image).
- the post-processing execution module 406 may reduce an error that may occur in the character extraction process by extracting only a character code from the extracted character data.
- FIG. 5 is a diagram illustrating an original video image screen and a text related information providing screen according to an embodiment of the present invention.
- the portable terminal when a mobile terminal according to the present invention captures an image through a camera unit and acquires an original image image, the portable terminal may display the original image image screen 510 through the display unit.
- the mobile terminal inserts the text-related virtual image area 525 into the original video image and displays the text-related information providing screen 520 in the augmented reality technique.
- the augmented reality technique is to add and represent virtual information on the real world (eg, a still image obtained by a camera).
- the real-world information may contain information that the user does not need, and sometimes the information that the user needs may be insufficient. Accordingly, by using the virtual information, information that is not needed can be made simple or invisible, and insufficient information can be supplemented.
- FIG. 6 is a flowchart illustrating a text-related information providing method using a mobile terminal according to an embodiment of the present invention.
- the mobile terminal captures an image through a camera to obtain an original image image (S602), and receives a text area selected from the original image image (S604).
- the mobile terminal extracts a character from the selected text area (S606).
- a preprocessing process of sharpening a contour by adjusting the brightness or contrast of the selected text area and a text extraction process of extracting text data from the selected text area.
- a post-processing process for extracting only a character code from the character data.
- the portable terminal transmits the extracted text information to the text related information server (S608). Accordingly, the text related information server searches for the text related information corresponding to the extracted text by referring to the text related information database. Provided to the terminal (S610).
- the portable terminal synthesizes the found text related information into an original video image.
- the mobile terminal displays the virtual image corresponding to the retrieved text related information on the original image image and displays the virtual image.
- FIG. 7 is a flowchart illustrating a text-related information providing method using a mobile terminal according to another embodiment of the present invention.
- the mobile terminal captures an image through a camera to obtain an original image image (S702), and receives a text area selected from the original image image (S704).
- the portable terminal extracts a character feature, which is an important feature for extracting a character from the selected text area (S706), and transmits the extracted character feature information to a text related information server (S708).
- the text related information server extracts the text with reference to the transmitted character feature information (S710), searches for the text related information corresponding to the extracted text, and provides the text to the portable terminal with reference to the text related information database (S710). S712).
- the portable terminal synthesizes the provided text related information into an original video image.
- the mobile terminal displays the virtual image corresponding to the provided text related information on the original image image and displays the image by the augmented reality technique.
- FIG. 8 is a flowchart illustrating a method for providing text-related information using a mobile terminal according to another embodiment of the present invention.
- the mobile terminal captures an image through a camera to obtain an original image image (S802), and receives a text area selected from the original image image by the user.
- the portable terminal may transmit the selected text area information to the text related information server (S804).
- the text related information server extracts the text from the selected text area (S806).
- a preprocessing process of sharpening the contour by adjusting the brightness or contrast of the selected text area and extracting the text data from the selected text area.
- the character extraction process and the post-processing process of extracting only the character code from the character data may be sequentially performed.
- the text related information server searches for text related information corresponding to the extracted text and provides it to the portable terminal by referring to the text related information database (S808).
- the portable terminal may display the text information related to the retrieved text by combining it with the original video image (S810).
- the mobile terminal may display the virtual image corresponding to the retrieved text related information by superimposing the original image image on the original image image and displaying the same by the augmented reality technique.
Abstract
Description
Claims (20)
- 휴대 단말기가 카메라를 통해 영상을 촬영하여 원본 영상 이미지를 획득하며, 상기 원본 영상 이미지 중 문자 영역을 선택받는 단계;The mobile terminal photographing an image through a camera to obtain an original image image, and receiving a text area selected from the original image image;상기 휴대 단말기가 선택된 문자 영역에서 문자를 추출하여, 추출된 문자 정보를 문자 관련 정보 서버로 전송하는 단계;Extracting, by the portable terminal, a character from a selected text area and transmitting the extracted text information to a text related information server;문자 관련 정보 서버가 문자 관련 정보 데이터베이스를 참조하여 상기 문자 정보에 상응하는 문자 관련 정보를 검색하며, 검색된 문자 관련 정보를 상기 휴대 단말기로 전송하는 단계; 및A text related information server searching for text related information corresponding to the text information by referring to a text related information database, and transmitting the found text related information to the mobile terminal; And상기 휴대 단말기가 전송된 문자 관련 정보를 상기 원본 영상 이미지에 합성하여 디스플레이하는 단계;Displaying, by the portable terminal, composing the transmitted text-related information into the original video image;를 포함하는 휴대 단말기를 이용한 문자 관련 정보 제공 방법.Character-related information providing method using a mobile terminal comprising a.
- 청구항 1에 있어서,The method according to claim 1,상기 추출 문자 정보 전송 단계는,The extracting character information transmitting step,선택된 문자 영역의 밝기 또는 명암을 조절하여 윤곽을 선명하게 처리하는 전처리 과정, 상기 선택된 문자 영역에서 문자 데이터를 추출하는 문자 추출 과정 및 상기 문자 데이터 중 문자 코드만을 추출하는 후처리 과정을 수행하여 문자를 추출하는 것을 특징으로 하는 휴대 단말기를 이용한 문자 관련 정보 제공 방법.By performing the pre-processing process to sharpen the outline by adjusting the brightness or contrast of the selected text area, a text extraction process for extracting the text data from the selected text area, and a post-processing process for extracting only the character code from the text data. Character-related information providing method using a mobile terminal, characterized in that for extracting.
- 청구항 2에 있어서,The method according to claim 2,상기 디스플레이 단계는,The display step,상기 전송된 문자 관련 정보에 상응하는 가상 이미지를 상기 원본 영상 이미지에 중첩하여 증강 현실 기법으로 디스플레이하는 것을 특징으로 하는 휴대 단말기를 이용한 문자 관련 정보 제공 방법.And displaying a virtual image corresponding to the transmitted text related information on the original video image and displaying the virtual image by augmented reality.
- 휴대 단말기가 카메라를 통해 영상을 촬영하여 원본 영상 이미지를 획득하며, 상기 원본 영상 이미지 중 문자 영역을 선택받는 단계;The mobile terminal photographing an image through a camera to obtain an original image image, and receiving a text area selected from the original image image;상기 휴대 단말기가 선택된 문자 영역에서 문자 특징를 추출하여, 추출된 문자 특징 정보를 문자 관련 정보 서버로 전송하는 단계;Extracting, by the portable terminal, a text feature from a selected text area and transmitting the extracted text feature information to a text related information server;상기 문자 관련 정보 서버가 상기 전송된 문자 특징 정보에 따라 문자를 추출하는 단계;Extracting, by the text related information server, text according to the transmitted text characteristic information;상기 문자 관련 정보 서버가 문자 관련 정보 데이터베이스를 참조하여 상기 추출된 문자에 상응하는 문자 관련 정보를 검색하며, 검색된 문자 관련 정보를 상기 휴대 단말기로 전송하는 단계; 및The text related information server searching for text related information corresponding to the extracted text by referring to a text related information database, and transmitting the found text related information to the mobile terminal; And상기 휴대 단말기가 전송된 문자 관련 정보를 상기 원본 영상 이미지에 합성하여 디스플레이하는 단계;Displaying, by the portable terminal, composing the transmitted text-related information into the original video image;를 포함하는 휴대 단말기를 이용한 문자 관련 정보 제공 방법.Character-related information providing method using a mobile terminal comprising a.
- 청구항 4에 있어서,The method according to claim 4,상기 문자 추출 단계는,The character extraction step,상기 문자 영역의 밝기 또는 명암을 조절하여 윤곽을 선명하게 처리하는 전처리 과정, 상기 선택된 문자 영역에서 문자 데이터를 추출하는 문자 추출 과정 및 상기 문자 데이터 중 문자 코드만을 추출하는 후처리 과정을 수행하여 문자를 추출하는 것을 특징으로 하는 휴대 단말기를 이용한 문자 관련 정보 제공 방법.Characters are processed by performing a preprocessing process of sharpening an outline by adjusting the brightness or contrast of the text area, a text extraction process of extracting text data from the selected text area, and a post-processing process of extracting only a text code from the text data. Character-related information providing method using a mobile terminal, characterized in that for extracting.
- 청구항 5에 있어서,The method according to claim 5,상기 디스플레이 단계는,The display step,상기 전송된 문자 관련 정보에 상응하는 가상 이미지를 상기 원본 영상 이미지에 중첩하여 증강 현실 기법으로 디스플레이하는 것을 특징으로 하는 휴대 단말기를 이용한 문자 관련 정보 제공 방법.And displaying a virtual image corresponding to the transmitted text related information on the original video image and displaying the virtual image by augmented reality.
- 휴대 단말기가 카메라를 통해 영상을 촬영하여 원본 영상 이미지를 획득하는 단계;Acquiring, by the mobile terminal, an image through a camera to obtain an original image image;상기 휴대 단말기는 상기 원본 영상 이미지 중 문자 영역을 선택받고, 선택된 문자 영역을 문자 관련 정보 서버로 전송하는 단계;The mobile terminal receiving a selection of a text area of the original video image and transmitting the selected text area to a text-related information server;상기 문자 관련 정보 서버가 상기 전송된 문자 영역으로부터 문자를 추출하는 단계;Extracting, by the text related information server, a text from the transmitted text area;상기 문자 관련 정보 서버가 문자 관련 정보 데이터베이스를 참조하여 상기 추출된 문자에 상응하는 문자 관련 정보를 검색하며, 검색된 문자 관련 정보를 상기 휴대 단말기로 전송하는 단계; 및The text related information server searching for text related information corresponding to the extracted text by referring to a text related information database, and transmitting the found text related information to the mobile terminal; And상기 휴대 단말기가 전송된 문자 관련 정보를 상기 원본 영상 이미지에 합성하여 디스플레이하는 단계;Displaying, by the portable terminal, composing the transmitted text-related information into the original video image;를 포함하는 휴대 단말기를 이용한 문자 관련 정보 제공 방법.Character-related information providing method using a mobile terminal comprising a.
- 청구항 7에 있어서,The method according to claim 7,상기 문자 추출 단계는,The character extraction step,상기 문자 영역의 밝기 또는 명암을 조절하여 윤곽을 선명하게 처리하는 전처리 과정, 상기 선택된 문자 영역에서 문자 데이터를 추출하는 문자 추출 과정 및 상기 문자 데이터 중 문자 코드만을 추출하는 후처리 과정을 수행하여 문자를 추출하는 것을 특징으로 하는 휴대 단말기를 이용한 문자 관련 정보 제공 방법.Characters are processed by performing a preprocessing process of sharpening an outline by adjusting the brightness or contrast of the text area, a text extraction process of extracting text data from the selected text area, and a post-processing process of extracting only a text code from the text data. Character-related information providing method using a mobile terminal, characterized in that for extracting.
- 청구항 8에 있어서,The method according to claim 8,상기 디스플레이 단계는,The display step,상기 전송된 문자 관련 정보에 상응하는 가상 이미지를 상기 원본 영상 이미지에 중첩하여 증강 현실 기법으로 디스플레이하는 것을 특징으로 하는 휴대 단말기를 이용한 문자 관련 정보 제공 방법.And displaying a virtual image corresponding to the transmitted text related information on the original video image and displaying the virtual image by augmented reality.
- 카메라를 통해 영상을 촬영하여 원본 영상 이미지를 획득하는 카메라부;A camera unit for capturing an image through a camera to obtain an original image image;상기 원본 영상 이미지 중 문자 영역이 선택된 경우, 상기 선택된 문자 영역에 상응하는 정보를 문자 관련 정보 서버로 전송하도록 하는 제어부; 및A controller for transmitting information corresponding to the selected text area to a text-related information server when a text area is selected from the original video image; And상기 문자 관련 정보 서버로부터 상기 문자 영역에 상응하는 정보에 대응하여 수신된 문자 관련 정보를 상기 원본 영상 이미지에 합성하여 디스플레이하는 표시부;A display unit configured to display, on the original video image, text related information received from the text related information server corresponding to the information corresponding to the text area;를 포함하는 휴대 단말기.Mobile terminal comprising a.
- 청구항 10에 있어서,The method according to claim 10,상기 제어부는, 상기 선택된 문자 영역에서 문자를 추출하도록 하고, 추출된 문자 정보를 상기 문자 영역에 상응하는 정보로 하여 상기 문자 관련 정보 서버로 전송하도록 하는 것을 특징으로 하는 휴대 단말기.And the controller is configured to extract a character from the selected text area, and transmit the extracted text information as information corresponding to the text area to the text related information server.
- 청구항 11에 있어서,The method according to claim 11,상기 선택된 문자 영역의 밝기 또는 명암을 조절하여 윤곽을 선명하게 처리하는 전처리 과정, 상기 선택된 문자 영역에서 문자 데이터를 추출하는 문자 추출 과정 및 상기 문자 데이터 중 문자 코드만을 추출하는 후처리 과정을 수행하여 문자를 추출하는 문자 추출부를 더 포함하는 것을 특징으로 하는 휴대 단말기.By performing the pre-processing process of sharpening the contour by adjusting the brightness or contrast of the selected text area, the character extraction process for extracting the text data in the selected text area, and the post-processing process for extracting only the character code from the text data The mobile terminal further comprises a character extracting unit for extracting.
- 청구항 10에 있어서,The method according to claim 10,상기 제어부는, 상기 선택된 문자 영역에서 문자 특징을 추출하고, 추출된 문자 특징 정보를 상기 문자 영역에 상응하는 정보로 하여 상기 문자 관련 정보 서버로 전송하도록 하는 것을 특징으로 하는 휴대 단말기.And the controller extracts a text feature from the selected text area and transmits the extracted text feature information to the text related information server as information corresponding to the text area.
- 청구항 10에 있어서,The method according to claim 10,상기 제어부는, 상기 선택된 문자 영역 이미지를 상기 문자 영역에 상응하는 정보로 하여 상기 문자 관련 정보 서버로 전송하도록 하는 것을 특징으로 하는 휴대 단말기.And the control unit transmits the selected text area image as information corresponding to the text area to the text related information server.
- 청구항 10에 있어서,The method according to claim 10,상기 제어부는, 상기 원본 영상 이미지에 기초하여, 상기 전송된 문자 관련 정보에 상응하는 가상 이미지의 합성 위치를 분석하고, 상기 가상 이미지를 상기 원본 영상 이미지의 분석된 위치에 합성하여, 상기 표시부를 통해 증강 현실 기법으로 출력하도록 하는 것을 특징으로 하는 휴대 단말기.The control unit analyzes a synthesis position of the virtual image corresponding to the transmitted character related information based on the original image image, synthesizes the virtual image to the analyzed position of the original image image, and then, through the display unit. A mobile terminal, characterized in that for output by the augmented reality technique.
- 하나 이상의 문자 정보와 상기 각 문자 정보에 상응하는 문자 관련 정보를 저장하는 문자 관련 정보 데이터베이스;A character-related information database for storing one or more character information and character-related information corresponding to each character information;휴대 단말기에 의해 획득된 원본 영상 이미지 중 선택된 문자 영역에 상응하는 정보를 수신하는 수신부;A receiving unit which receives information corresponding to a selected text area of an original image image acquired by the portable terminal;상기 문자 관련 정보 데이터베이스를 참조하여 상기 수신된 문자 정보에 상응하는 문자 관련 정보를 검색하는 관련 정보 검색부; 및A related information search unit for searching for text related information corresponding to the received text information by referring to the text related information database; And상기 검색된 문자 관련 정보를 상기 휴대 단말기로 전송하는 관련 정보 제공부;A related information provider for transmitting the retrieved text related information to the mobile terminal;를 포함하는 문자 관련 정보 서버.Character-related information server, including.
- 청구항 16항에 있어서,The method of claim 16,상기 문자 관련 정보 데이터베이스는 인터넷망을 통해 연결된 개인용 컴퓨터에 의해 상기 문자 관련 정보를 획득하여 저장하거나 인터넷을 자동 검색하여 상기 문자 관련 정보를 획득하여 저장하는 것을 특징으로 하는 문자 관련 정보 서버.The text related information server may obtain and store the text related information by a personal computer connected through an internet network, or automatically search the Internet to obtain and store the text related information.
- 청구항 16에 있어서,The method according to claim 16,상기 관련 정보 제공부는 상기 검색된 문자 관련 정보에 상응하는 가상 이미지 정보를 상기 휴대 단말기로 전송하는 것을 특징으로 하는 문자 관련 정보 서버.And the related information providing unit transmits virtual image information corresponding to the retrieved text related information to the mobile terminal.
- 청구항 16항에 있어서,The method of claim 16,상기 수신부가 휴대 단말기로부터 문자 영역 이미지 또는 문자 특징 정보를 수신하는 경우, 상기 수신된 문자 영역 이미지 또는 문자 특징 정보로부터 문자를 추출하는 문자 추출부를 더 포함하며,When the receiving unit receives the text area image or text feature information from the mobile terminal, further comprising a text extraction unit for extracting a character from the received text area image or text feature information,상기 관련 정보 검색부는 상기 추출된 문자에 상응하는 문자 관련 정보를 검색하는 것을 특징으로 하는 문자 관련 정보 서버.And the related information search unit searches for text related information corresponding to the extracted text.
- 청구항 19항에 있어서,The method of claim 19,상기 문자 추출부는, 상기 문자 영역 이미지의 밝기 또는 명암을 조절하여 윤곽을 선명하게 처리하는 전처리 수행 모듈, 상기 수신된 문자 영역에서 문자 데이터를 추출하는 문자 추출 모듈 및 상기 문자 데이터 중 문자 코드만을 추출하는 후처리 수행 모듈을 포함하는 것을 특징으로 하는 문자 관련 정보 서버.The text extractor may include a preprocessing module configured to sharply process an outline by adjusting brightness or contrast of the text area image, a text extraction module to extract text data from the received text area, and to extract only a text code from the text data. Character-related information server comprising a post-processing performing module.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/KR2010/001916 WO2011122717A1 (en) | 2010-03-30 | 2010-03-30 | Method for providing text relation information using a mobile terminal, and mobile terminal |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/KR2010/001916 WO2011122717A1 (en) | 2010-03-30 | 2010-03-30 | Method for providing text relation information using a mobile terminal, and mobile terminal |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011122717A1 true WO2011122717A1 (en) | 2011-10-06 |
Family
ID=44712397
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/KR2010/001916 WO2011122717A1 (en) | 2010-03-30 | 2010-03-30 | Method for providing text relation information using a mobile terminal, and mobile terminal |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2011122717A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111651049A (en) * | 2020-06-08 | 2020-09-11 | 浙江商汤科技开发有限公司 | Interaction method and device, computer equipment and storage medium |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20040098148A (en) * | 2003-05-13 | 2004-11-20 | 엘지전자 주식회사 | A mobile phone for browsing dictionary data and an operating method thereof |
US20050037806A1 (en) * | 2003-08-12 | 2005-02-17 | Kyoung-Weon Na | Managing an address book in portable terminal having a camera |
KR100789100B1 (en) * | 2006-09-19 | 2007-12-26 | 에스케이 텔레콤주식회사 | Mobile augment reality service system and method |
KR20090029350A (en) * | 2007-09-18 | 2009-03-23 | 에스케이텔레콤 주식회사 | System and method for servicing virtual world in mobile terminal |
KR20100124952A (en) * | 2009-05-20 | 2010-11-30 | 삼성에스디에스 주식회사 | Ar contents providing system and method providing a portable terminal real-time by using letter recognition |
-
2010
- 2010-03-30 WO PCT/KR2010/001916 patent/WO2011122717A1/en active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20040098148A (en) * | 2003-05-13 | 2004-11-20 | 엘지전자 주식회사 | A mobile phone for browsing dictionary data and an operating method thereof |
US20050037806A1 (en) * | 2003-08-12 | 2005-02-17 | Kyoung-Weon Na | Managing an address book in portable terminal having a camera |
KR100789100B1 (en) * | 2006-09-19 | 2007-12-26 | 에스케이 텔레콤주식회사 | Mobile augment reality service system and method |
KR20090029350A (en) * | 2007-09-18 | 2009-03-23 | 에스케이텔레콤 주식회사 | System and method for servicing virtual world in mobile terminal |
KR20100124952A (en) * | 2009-05-20 | 2010-11-30 | 삼성에스디에스 주식회사 | Ar contents providing system and method providing a portable terminal real-time by using letter recognition |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111651049A (en) * | 2020-06-08 | 2020-09-11 | 浙江商汤科技开发有限公司 | Interaction method and device, computer equipment and storage medium |
CN111651049B (en) * | 2020-06-08 | 2024-01-09 | 浙江商汤科技开发有限公司 | Interaction method, device, computer equipment and storage medium |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111556278B (en) | Video processing method, video display device and storage medium | |
CN111445583B (en) | Augmented reality processing method and device, storage medium and electronic equipment | |
WO2019013517A1 (en) | Apparatus and method for voice command context | |
WO2018182321A1 (en) | Method and apparatus for rendering timed text and graphics in virtual reality video | |
WO2010147359A2 (en) | Method for creating content using a camera of a portable terminal and a portable terminal adapted therefor | |
WO2013077643A1 (en) | Apparatus and method for providing augmented reality service for mobile terminal | |
WO2011105671A1 (en) | System and method for providing a user manual using augmented reality | |
CN110572716B (en) | Multimedia data playing method, device and storage medium | |
WO2017217713A1 (en) | Method and apparatus for providing augmented reality services | |
CN112991553B (en) | Information display method and device, electronic equipment and storage medium | |
JP2005267146A (en) | Method and device for creating email by means of image recognition function | |
CN115918108B (en) | Method for determining function switching entrance and electronic equipment | |
CN115481597A (en) | Method, device and equipment for editing electronic document and storage medium | |
WO2022154270A1 (en) | Method for generating highlight video and electronic device thereof | |
WO2012108552A1 (en) | Display device and control method thereof | |
CN111222063A (en) | Rich text rendering method and device, electronic equipment and storage medium | |
CN113297843B (en) | Reference resolution method and device and electronic equipment | |
CN111079030A (en) | Group searching method and electronic device | |
WO2014003520A1 (en) | Led display for outdoor advertising, and interaction method | |
WO2018131852A1 (en) | Electronic device for performing video call and computer-readable recording medium | |
WO2018043923A1 (en) | Display device and control method therefor | |
CN109947988B (en) | Information processing method and device, terminal equipment and server | |
WO2021045552A1 (en) | Electronic device for image synthesis and operating method thereof | |
CN113190307A (en) | Control adding method, device, equipment and storage medium | |
KR20110116116A (en) | Method for providing text relation information using mobile terminal and mobile terminal |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
ENP | Entry into the national phase |
Ref document number: 20117010161 Country of ref document: KR Kind code of ref document: A |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 10849039 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIUGHTS PURSUANT TO RULE 112(1) EPC OF 050213 |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 10849039 Country of ref document: EP Kind code of ref document: A1 |