KR20080099385A - System and method for searching and output of media contents via character recognition - Google Patents

Abstract

A media content search/output system and a method thereof are provided to mutually associate printed matters with electronic media contents. A media content search/output system through character recognition comprise the followings: an image input device(20) which photographs the image of a specific character on a printed matter, and then provides the photographed image data to search information about media contents related to the specific word; and an information inquiry terminal(30) which performs the character recognition for the character image data provided from the image input device, and searches the media contents related to the recognized character data in a media content storage(40) to display the searched result or output it through voice.

Description

System and Method for Searching and Output of Media Contents via Character Recognition}

1 is a diagram schematically illustrating a technical concept of a media content search and output system through character recognition according to the present invention;

2 is a diagram illustrating a configuration of a system for retrieving and outputting media content through character recognition according to the present invention;

3 is a view showing a state in which media content associated with a recognized character of a printed document is retrieved and extracted from a media recording medium according to a preferred embodiment of the present invention;

4 is a flowchart illustrating an operation of a method for retrieving and outputting media content through character recognition according to the present invention.

<Explanation of symbols for the main parts of the drawings>

10: prints, 20: image input device,

22: photographing camera, 24, 32: communication interface,

26: control unit, 30: information inquiry terminal,

34: image preprocessor, 36: character recognizer,

38: Recognition post-processor, 40: Media recording medium,

42: external recording medium, 44: recording medium interface,

46: media search unit, 48: display unit,

50: audio output unit, 52: control unit.

The present invention relates to a system for retrieving and outputting media content through text recognition and a method for retrieving media content related to the contents of a printed matter such as a printed booklet and outputting the image for viewing by a user.

Conventionally, in the case of a printed matter which is printed in a book form or provided in a form of a book by printing the related content on paper for transferring knowledge or education, readability may be ensured according to the printing state, and the related printed content may be read. It is versatile that can be used by anyone with the purpose.

On the other hand, in the case of such printed matters, since the relevant contents are printed and provided on paper, it is not easy to be damaged when a long time passes or used by a large number of people, and in the case of printed matters, it is inconvenient to carry them. In addition, in order to identify a portion to be additionally referred to in the printed contents of the printed matter, it was inconvenient to find and read a separate printed matter related to the printed contents.

In order to alleviate the inconvenience in the conventional printed matter, recently, the printed contents of the printed matter are converted into electronic data in the form of text or image, and the user can use a multimedia information device such as a personal computer (PC), PDA, or PMP. E-books that can be read are becoming commercially available.

On the other hand, in the case of such an electronic book, the text or image content is read through the monitor screen of a personal computer or the small screen of a small multimedia information device such as a PDA or PMP. In conjunction, there is a need to use e-books for additional content or for auxiliary purposes.

However, at present, the linkage between printed materials and electronic books of multimedia information devices is not smoothly established, and even if the user wants to find multimedia information such as video, image, audio, etc. as additional contents check or supplementary material in addition to electronic books, There is a problem that it is difficult to retrieve and provide information quickly and accurately in connection with the printed matter.

Accordingly, the present invention has been made to solve the above-described problems, the object of which is to shoot a desired character from the printed matter, such as a printed booklet, and search for and output the media content associated with the character through the character recognition of the photographed character The present invention provides a system and method for retrieving and outputting media content through character recognition.

In order to achieve the above object, according to the system of the present invention, an image input device for photographing a specific word of the printed matter and providing the image data of the image taken for information information of the media content associated with the word, and the image input device Character information characterized in that it comprises a information retrieval terminal for performing character recognition on the text image data from the image content, and retrieves the media content associated with the text recognition character data from the pre-stored media content storage to display and output the video Provides a system for retrieving and outputting media content through recognition.

According to the method of the present invention for achieving the above object, the step of photographing a specific word position portion of the printed matter in the image input device, and transmitting the photographed character image data to the information inquiry terminal via short-range communication, the information Receiving a character image data at a querying terminal, performing character recognition processing, determining validity of the character recognized character data, and displaying media content having information content related to the recognized character data for which the character recognition and validity determination have been performed. Searching from a recording medium storing media content including general additional information in relation to the contents of the printed matter; and extracting media content having information content related to the recognized text data as a result of searching for the media content. And voice output steps To provide media content search method and output through the character recognition, characterized in that formed.

Hereinafter, the present invention configured as described above will be described in detail with reference to the accompanying drawings.

That is, FIG. 1 is a diagram schematically illustrating a technical concept of a media content search and output system through character recognition according to the present invention.

As shown in FIG. 1, in the media content search and output system through character recognition according to the present invention, a user wants to find additional information or supplementary data among contents of a printed book 10 in a book form or a book form. If there is a word that requires the content to be found, the printed portion of the printed matter 10 on which the letter of the word is located is photographed through an image input device 20 equipped with a photographing camera, and then the photographed portion is taken. The character image data is transmitted to the information inquiry terminal 30.

The information inquiry terminal 30 may be a text, image, audio, flash, video, or the like, such as a personal computer terminal (PC), a notebook computer, a mobile communication terminal, a smartphone, a personal digital assistant (PDA), a personal multimedia player (PMP), or the like. It consists of a device that can query the multimedia information.

The information inquiry terminal 30 performs a character recognition process on the text image data received from the image input apparatus 20, searches for media information related to the text according to the text recognition processing result, and then returns a search result. The resulting text, image, audio, flash, video data can be output through the screen or the speaker for the user to check.

Next, FIG. 2 is a diagram illustrating a configuration of a media content search and output system through character recognition according to the present invention.

As shown in FIG. 2, the media content retrieval and output system through text recognition according to the present invention comprises an image input device 20 and an information retrieval terminal 30, the image input device 20. Includes a photographing camera 22, a communication interface 24, and a control unit 26. The information inquiry terminal 30 includes a communication interface 32, an image preprocessor 34, and a character recognizer 36. ), Recognition post processor 38, media recording medium 40, external recording medium 42, recording medium interface 44, media searching unit 46, display unit 48, audio output unit 50, control unit And 52.

The photographing camera 22 of the image input apparatus 20 outputs the photographed character part in the form of digital image data by photographing a specific word position part of the printed matter 10 according to a user's operation.

The communication interface 24 is connected to the communication interface 32 of the information inquiry terminal 30 so as to transmit text image data and control communication for the data transmission, and to control the control unit 26. Accordingly, the character image data photographed from the photographing camera 22 is transmitted to the information inquiry terminal 30.

Here, the communication interface 24 of the image input device 20 and the communication interface 32 of the information inquiry terminal 30 may be formed in the form of a short range wireless communication module such as IrDA, Bluetooth, zigbee, or WirelessUSB, or a serial interface. As a wired communication module such as UART, USB, etc.

When the specific character portion of the printed matter 10 is photographed by the photographing camera 22, the controller 26 captures the text image data photographed from the photographing camera 22 according to a user's operation through the communication interface 24. Control to be transmitted to the information inquiry terminal 30.

In addition, the communication interface 32 of the information inquiry terminal 30 is interconnected to allow data reception and control communication with the communication interface 24 of the image input device 20, thereby photographing from the photographing camera 22. The received character image data through the communication interface 24.

The image preprocessor 34 receives text image data from the image input device 20 received through the communication interface 32, and converts the text image data into binary data suitable for text recognition. In addition, the noise component except the text image is removed.

The character recognizer 36 performs a character recognition process on the pre-processed binary image data through the image preprocessor 34 to convert the character data into character data.

The recognition post-processor 38 is for determining the validity of the character data recognized and converted by the character recognizer 36. For example, the recognition post-processor 38 is not valid for words composed of Korean characters or words or symbols that cannot be understood. The result of the recognition is processed, and the validity judgment result is provided to the controller 52.

The media recording medium 40 may be provided as additional supplementary or supplementary text, images, audio, flashes with respect to all the words or photographs or description words (or names) of pictures and pictures included in the printed matter 10. , Multimedia data such as video is generally stored, which corresponds to a hard disk drive when the information inquiry terminal 30 is a personal computer or a notebook computer, and is a mobile communication terminal, a smartphone, a PDA, and a PMP. It is desirable to have a large capacity semiconductor memory or a small hard disk drive.

The external recording medium 42 is formed in a removable form with the information inquiry terminal 30, and as an additional or updated multimedia data is stored for the multimedia data stored in the media recording medium 40, This can be a memory card such as SD card, Mini-SD card, CF card, MMC card, XD card, T-Flash card, etc., or a disk-type recording medium such as compact disk (CD), DVD (DVD), mini disk (MD), etc. , A memory stick or the like is preferable.

The recording medium interface 44 mounts the external recording medium 42 and recognizes the recording medium to read storage information.

The media retrieval unit 46 searches the media recording medium using the characters derived as a result of the character recognition processing of the character image data through the image preprocessor 34, the character recognizer 36, and the recognition postprocessor 38. Relevant information about the stored data of the 40 and the external recording medium 42 is searched, and media content related to the word of the text is extracted and output to the display unit 48 and the audio output unit 50.

Under the display control of the controller 52, the display unit 48 visually checks the media content related to the character searched through the media search unit 46, that is, text, image, flash, and video data. To be displayed as possible.

The voice output unit 50 includes a power amplifier and a speaker for power amplifying an audio signal, and the audio content associated with the text searched through the media search unit 46, that is, the audio data and the flash. The audio portion and the audio portion of the video are outputted so that the user can audibly check.

The control unit 52 provides the media search unit 46 with the text data according to the character recognition result of which the validity determination is made through the recognition postprocessor 38 to request the search for the media content related to the corresponding character. Regarding the character image determined to be an invalid character through the recognition postprocessor 38, an indication of an invalid result and a re-shooting request indication may be made through the display unit 48, and the communication interface 32 It provides a result of the invalidity determination and the re-shooting request information to the image input device 20 through.

When the control unit 26 of the image input device 20 receives the invalidity determination result and the re-shooting request information from the control unit 52 of the information inquiry terminal 30, a separate unit provided in the corresponding image input device 20 is provided. It is preferable to display the state through a display unit (not shown) or to generate an alarm sound through a speaker (not shown) to notify the user.

Meanwhile, when the media retrieval unit 46 searches for media content stored in the media recording medium 40 or the external recording medium 42, the media recording medium 40 includes text media. Data, image media data, audio media data, moving picture media data, and flash media data are stored separately, and an image index indicating each description word (or name) for the image, audio, video, and flash media data. , Audio index, video index, and flash index are assigned.

The media retrieval section 44 searches the relevant text related to the recognition character as a whole for the entire text recording portion of the text media data of the media recording medium 38, and extracts the related text contents according to the result. do.

In addition, the media search unit 44 searches for each image index, audio index, video index, and flash index with respect to the image, audio, video, and flash media data. The search for the index of the video and the flash content is performed separately, and as a result, the related image, the audio, the video, and the flash content connected to the searched index are extracted.

Next, the operation of the present invention made as described above will be described in detail with reference to the flowchart of FIG. 4.

First, the user reads the printed book 10 in booklet form or in document form, and then adds additional media material for "Tiger" as a descriptive word (or name), such as a specific word such as "Tiger" or "Tiger picture". If you want to find it, if the image input device 20 fixes a specific word portion of the printed matter 10 at the photographing position and captures the word through the photographing camera 22 (step S10), the image input device 20 Control unit 26 transmits the character image data of the photographed specific word to the information inquiry terminal 30 through the communication interface 24 (step S11).

The image preprocessor 34 of the information inquiry terminal 30 converts the text image data from the image input device 20 received through the communication interface 32 into black and white local binary data so as to be suitable for character recognition. It converts and removes the noise component which exists other than a character area (step S12).

The character recognizer 36 of the information inquiry terminal 30 performs character recognition on the binary data converted through the image preprocessor 34 to convert the recognition into character data (step S13). 38 determines the validity of the recognized converted character data and provides the validity determination result to the controller 52 (step S14).

The control unit 52 determines whether the recognition character is a valid character for retrieving media data as a result of the validity determination from the recognition postprocessor 38 (step S15). As a result of the determination, the recognition character is not valid. If it is determined that it is not, the invalid state is displayed on the display unit 48, and the image input device 20 is requested to retake the image via the communication interface 32 (step S16).

However, when it is determined that the recognition character is valid from the recognition postprocessor 38 and the character data such as "tiger" is received according to the determination result, the control unit 52 is the media search unit 46. The media search unit 46 requests a search for media content related to the recognized text data, and the media search unit 46 searches the media text 40 or the external recording medium 42 using the recognized text data such as "tiger" as a search word. In addition to performing the relevant text search on the text media data stored in the &lt; RTI ID = 0.0 &gt; 1), &lt; / RTI &gt; the image, audio, video, and flash media data are respectively searched for the connected image index, audio index, video index, and flash index (step S17 ).

The control unit 40 determines whether the search result is extracted from the media search unit 46 (step S18), and if it is determined that the search result of the related media content with respect to the recognition text data is extracted, Media content related to recognized character data, such as "Tiger" (ie text content with additional description of "Tiger", pictures of "Tiger", image content with photos, and description of "Tiger"). Narrator voice, audio content including "tiger" crying, video content related to "tiger", flash content with the theme of "tiger") may be output via display 48 or voice output 50. So that the user can confirm additional information visually or audibly (step S19).

While specific embodiments of the present invention have been described and illustrated above, it will be apparent that the present invention may be embodied in various modifications by those skilled in the art. Such modified embodiments should not be understood individually from the technical spirit or the prospect of the present invention, but should fall within the claims appended to the present invention.

As described above, according to the present invention, when a material to be additionally checked for a specific word is required in a printed matter such as a booklet or a document, a specific word of the printed matter is photographed, and the photographed character image is recognized by a character, so that the recognized character and By retrieving related media contents from the recording medium containing the media information, and then outputting related media contents extracted as a search result, the printed materials and the electronic media contents are interconnected so that the printability, friendliness and media contents of the printed materials are interconnected. By satisfying the convenience of at the same time, it has the effect that it is possible to maximize the learning and reading efficiency of the user.

Claims (9)

An image input device for photographing a specific word of a printed matter and providing photographed character image data for information inquiry of media content associated with the word; And an information inquiry terminal for performing character recognition on text image data from the image input apparatus, retrieving media contents related to the text recognized text data from a pre-stored media content storage, and displaying and outputting an image. System for retrieving and outputting media contents through character recognition. The method of claim 1, The information inquiry terminal is at least one of a personal computer terminal (PC), a notebook computer, a mobile communication terminal, a smart phone, a PDA, a PMP. The method of claim 1, The image input apparatus may include a photographing camera which photographs a specific word position portion of a printed matter and outputs the image in the form of text image data; A communication interface for transmitting the text image data photographed from the photographing camera to the information inquiry terminal; And a control unit for controlling the text image data photographed from the photographing camera to be transmitted to the information inquiry terminal through the communication interface. The method of claim 1, The information inquiry terminal includes a communication interface for receiving text image data transmitted from the image input device; An image preprocessor for converting text image data received through the communication interface into binary data suitable for text recognition, and removing noise components except text images; A character recognizer for performing character recognition on the preprocessed binary image data through the image preprocessor to convert the image data into character data; Recognition post-processor to determine the validity of the character data recognized and converted through the character recognizer, A media recording medium in which the media content which can be provided as additional supplementary material or supplementary material in relation to all words, photographs, pictures and description words in the printed matter, A media retrieval unit configured to perform retrieval of related information on the stored data of the media recording medium using the recognized character as a search word, and to extract media content associated with a word of the character; A display unit configured to display media content related to the recognized character searched through the media search unit; A voice output unit configured to output an audio portion of media content associated with the text searched through the media search unit; And providing a control unit for requesting a search for media content related to the corresponding character by providing the recognized character data whose validity is determined through the recognition post-processor to output the media content through character recognition. system. The method of claim 4, wherein An external recording medium which is detachably attached to the information retrieval terminal and which stores additional or updated multimedia data with respect to the media contents contained in the media recording medium; And a recording medium interface for mounting the external recording medium and recognizing the recording medium to read storage information. The method according to claim 3 or 4, The communication interface of the image input device and the communication interface of the information inquiry terminal are at least one of a short range wireless communication module type such as IrDA, Bluetooth, zigbee, or WirelessUSB, or a wired communication module type such as UART or USB. System for retrieving and outputting media content through character recognition. A first step of photographing a specific word position portion of the printed matter by the image input apparatus and transmitting the photographed character image data to the information inquiry terminal through short-range communication; A second step of receiving the text image data from the information inquiry terminal to perform text recognition processing and determining validity of the text recognized text data; A third step of retrieving media content having information content related to the recognized character data from which the character recognition and validity determination have been made, from a recording medium storing the media content including general additional information relating to the content of the printed matter; And And a fourth step of extracting the media content having the information content related to the recognized text data as a result of the search for the media content, and displaying and outputting an image. The method of claim 7, wherein The second step may include converting the character image data into binary data suitable for character recognition, and removing noise components except for the character image; Performing character recognition on the converted binary image data and converting the same into character data; And determining the validity of the recognized converted text data. The method of claim 8, The determining of the validity of the text data may further include requesting a re-shooting from the information inquiry terminal to the image input device when it is determined that the corresponding text is not valid as a result of the validity determination of the text data. A method for retrieving and outputting media content through character recognition.

Images