JP2012160023A - Character extracting device, display method, and character extracting method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a character extracting device capable of executing a favorable extraction process to a character image included in a video, and to provide a display method and a character extraction method.SOLUTION: The character extracting device according to the present embodiment includes: reproduction means for reproducing video data; output means that outputs the video data reproduced by the reproduction means to a display device; an acceptance means for accepting an external input that corresponds to either position of the video displayed by the display device using the video data; image extraction means for extracting a first-shape image that is present within a predetermined range from the position corresponded with the external input, in the video displayed by the display device; and character extraction means that extracts a character from the image extracted by the extraction means.

Description

  Embodiments described herein relate generally to a character extraction device, a display method, and a character extraction method for extracting characters from video data.

  In a main part or a commercial of a TV broadcast program, an image that prompts viewers to search on the Internet may be displayed. Here, a search button imitating a search page on the Internet and an image of a search window are arranged in the video. Then, by placing an image of the character to be searched in the search window, the viewer is prompted to search for the character.

JP 2009-44658 A

  Here, when the user performs a process such as search using the character, a device such as a television receiver or a display device extracts the character from the character image in the video so that the user can select the character. It is preferable that the trouble of memorizing and writing down can be suppressed. In extracting the character, it is preferable that the character extraction process by the apparatus can be suitably executed.

  Therefore, an object of the embodiment of the present invention is to provide a character extraction device, a display method, and a character extraction method that can execute a suitable extraction process on a character image included in a video.

  In order to solve the above-described problems, a character extraction device according to the present embodiment uses a playback unit that plays back video data, an output unit that outputs video data played back by the playback unit to a display device, and video data. An accepting unit that accepts an external input corresponding to any position of an image displayed by the display device, and an image of a first shape within a certain range from the position corresponding to the external input in the image displayed by the display device. Image extracting means for extracting, and character extracting means for extracting characters from the image extracted by the extracting means.

FIG. 3 is a diagram showing an example of how the display device according to the first embodiment is used. The figure which shows the system configuration example of the display apparatus which concerns on 1st Embodiment. The figure which shows the structural example of the extraction character history database which the display apparatus which concerns on 1st Embodiment produces | generates. The figure which shows the example of a character extraction process by the display apparatus which concerns on 1st Embodiment. The figure which shows the example of a transition of the screen which the display apparatus which concerns on 1st Embodiment displays. The figure which shows the process flow example of the character extraction process by the display apparatus which concerns on 1st Embodiment. The figure which shows the example of a usage form of the display apparatus and information terminal which concern on 2nd Embodiment. The system configuration example of the display apparatus and information terminal which concern on 2nd Embodiment. The figure which shows the example of a character extraction process sequence by the display apparatus and information terminal which concern on 2nd Embodiment. The figure which shows the example of a transition of the screen which the display apparatus and information terminal which concern on 3rd Embodiment display. The figure which shows the example of a character extraction process sequence by the display apparatus and information terminal which concern on 3rd Embodiment.

(First embodiment)
Hereinafter, a first embodiment will be described with reference to the drawings. The character extraction device according to the present embodiment is realized as the display device 100, for example. Here, the display device 100 has a function of receiving video data such as a television broadcast and displaying a video based on the received video data on the display unit 107. In addition, the display device 100 receives an operation signal from an operation device such as the pointing device 200 and executes processing according to the operation signal.

  The display device 100 displays a CM video screen P20 on the display unit 107, for example. On the screen P20, a search box image P11 and a search button image P12 included in the CM video, a cursor image P21 generated by the display device 100, and the like are displayed. Note that the position of the cursor image P <b> 21 changes according to a movement instruction operation input from the pointing device 200. For example, when receiving an operation input to the button 201 or 202 of the pointing device 200, the display device 100 according to the embodiment detects the search box image P11 from within a certain range of the position where the cursor image P12 is arranged. The characters included in the image P11 can be extracted. Details will be described later with reference to FIGS.

  FIG. 2 shows a system configuration example of the display device 100. The display device 100 includes a receiving unit 101, a storage unit 102, a recording / playback control unit 103, a decoding unit 104, a combining unit 105, a signal processing unit 106, a display unit 107, an operation receiving unit 108, a GUI processing unit 109, and an image extraction unit 110. A character extraction unit 111, an extracted character database 112, a search unit 113, a communication unit 114, and the like.

  The receiving unit 101 receives, for example, video data of satellite digital TV broadcast or terrestrial digital TV broadcast. Here, the receiving unit 101 receives video data of a channel corresponding to an instruction from the recording / playback control unit 103, and outputs the received video data to the recording / playback control unit 103. The receiving unit 101 may receive video data such as IPTV distributed via a network such as the Internet. Furthermore, the receiving unit 101 may receive program information superimposed on broadcast waves or program information stored in a server on the network. For example, a broadcast channel, a broadcast time, a program title name, a performer name, a program content description, etc. are associated with these program information for each broadcast program.

  The storage unit 102 has a function of storing (recording) a stream of video data input from the recording / playback control unit 103. When the storage unit 102 receives an instruction to reproduce video data from the recording / playback control unit 103, the storage unit 102 outputs video data corresponding to the instruction among the stored video data to the recording / playback control unit 103.

  The recording / playback control unit 103 controls playback and recording processing of video data. In the reproduction process, the recording / reproduction control unit 103 causes the decoding unit 104 to decode (reproduce) the video data received by the reception unit 101 or the video data stored in the storage unit 102. That is, for example, when the operation accepting unit 108 receives a playback instruction input of a broadcast program of a predetermined channel from the user, the recording / playback control unit 103 instructs the receiving unit 101 to receive and output the video data of the channel. . The recording / playback control unit 103 outputs program information about the video data being played back to the character extraction unit 111 during the playback.

  In addition, when the operation receiving unit 108 receives a playback instruction input of recorded video data, the recording / playback control unit 103 instructs the storage unit 102 to output video data according to the playback instruction. At this time, the recording / playback control unit 103 controls the playback and stop of the video data in accordance with the operation input received by the operation receiving unit 108.

  In the recording process, the recording / playback control unit 103 outputs the stream of video data received by the receiving unit 101 to the storage unit 102 and causes the storage unit 102 to store (record) the video data.

  The decoding unit 104 decodes the video data stream input from the recording / playback control unit 103 and outputs the decoded data to the synthesis unit 105 and the image extraction unit 110.

  The synthesizing unit 105 superimposes the video data input from the decoding unit 104 and the GUI video data input from the GUI processing unit 109, and outputs the superimposed video data to the signal processing unit 106. Next, the signal processing unit 106 converts the input video data into a video signal in a format that can be displayed on the display, and outputs the video signal to the display unit 107. The display unit 107 displays a video using the input video signal.

  The operation receiving unit 108 receives an operation input signal from an operating device such as a pointing device 200 or a remote controller. Here, the operation accepting unit 108 receives, for example, a movement instruction operation input for moving the position of the cursor image P21, an extraction instruction operation input for instructing character extraction, playback of recorded video data, and a stop operation input. Then, the operation reception unit 108 outputs the received operation input to the GUI processing unit 109 and the recording / playback control unit 103 (the output path to the recording / playback control unit 103 is not illustrated).

  The GUI processing unit 109 generates GUI (Graphical User Interface) video data to be displayed on the display device 100, and processes an operation input from the user to the GUI. Here, the GUI processing unit 109 generates data of the cursor image P21, for example, and outputs the generated data to the combining unit 105. Then, the GUI processing unit 109 generates video data so that the cursor image P21 is arranged at a position corresponding to the movement instruction operation input input from the operation receiving unit 108.

  In addition, when the GUI processing unit 109 receives an extraction instruction operation input, the GUI processing unit 109 instructs the image extraction unit 110 to extract an image. The extraction instruction operation input is, for example, an operation input to the button 201 or 202 of the pointing device 200.

  Video data is input from the decoding unit 104 to the image extraction unit 110. Then, the image extraction unit 110 extracts a search box image included in the input video data in accordance with an instruction from the GUI processing unit 109. Further, the image extraction unit 110 extracts a character image included in the extracted search box image, and outputs the extracted character image to the character extraction unit 111.

  The character extraction unit 111 analyzes the input character image, and detects and extracts characters included in the character image. That is, the character extraction unit 111 generates character data (text data) from the character image. Then, the character extraction unit 111 outputs the generated character data to the extracted character database 112 and the search unit 113. Here, program information (channel, broadcast time, program title name, performer name, program description, etc.) regarding the video data being reproduced is input from the recording / reproduction control unit 103 to the character extraction unit 111. The Then, the character extraction unit 111 outputs the extracted character data and the program information about the video data from which the character data is extracted to the extracted character database 112 and the search unit 113.

  The extracted character database 112 stores the character data input from the character extraction unit 111 in association with the program information of the video data from which the character data is extracted. The database will be described later with reference to FIG.

  The search unit 113 executes a search using the character data extracted by the character extraction unit 111 and a search engine stored in a server on the Internet, for example. When executing the search, the search unit 113 first activates browser software for using the Internet. Subsequently, the search unit 113 controls the browser software so as to transmit data corresponding to the extracted character data to the URI of the search engine via the communication unit 114.

  Next, the search part 113 receives the data of the web page regarding the search result transmitted from the server. Here, the search unit 113 configures a search result screen based on the received web page data using browser software. Then, the search unit 113 instructs the GUI processing unit 109 to generate video data of the configured search result screen.

  The search unit 113 can also instruct the GUI processing unit 109 to generate a history screen that displays a history of character data input from the character extraction unit 111 or the extracted character database 112 and program information related to the character data. .

  FIG. 3 is a diagram illustrating a data configuration example of the extracted character database 112. The database 112 stores the extracted character B1 in association with the extracted date B2 of the character, the program information B3 from which the character is extracted, and the like. Here, the program information B3 is information based on program information superimposed on broadcast waves, program information distributed by a program information distribution server on the network, etc., for example, program name, broadcast channel, broadcast time, The name of the performer and description of the contents of the program are included.

  FIG. 4 shows an example of character extraction processing by the display device 100. FIG. 4A is a diagram illustrating an example of a screen displayed on the display unit 107 when the display device 100 is reproducing video data. The screen 300 is an example of one screen in a commercial video or the like, and the screen 300 includes a search box image P11 and a search button image P12 based on the reproduced video data.

  Here, the search box image P11 and the search button image P12 are images imitating, for example, an image of a search word input box displayed on a search web page on the Internet and an image of a search execution button. It is arranged on the left side of the search button image P12. The search box image P11 includes an image of a character that the producer or distributor of the video data including the screen P10 wants the user to search.

  Here, when the operation reception unit 108 receives an operation input from the pointing device 200, for example, the display unit 107 displays a screen P20 in which the cursor image P21 is superimposed on the screen P10. Note that the position of the cursor image P <b> 21 on the screen P <b> 20 changes according to the movement operation input from the pointing device 200.

  When the operation accepting unit 108 accepts, for example, a button operation input from the pointing device 200, the GUI processing unit 109 instructs the image extracting unit 110 to extract an image. Note that the GUI processing unit 109 outputs information about the position (coordinates) of the cursor image P21 on the screen P20 to the image extraction unit 110 at the time of the instruction.

  When receiving the image extraction instruction, the image extraction unit 110 extracts, for example, one frame from the video data input from the decoding unit 104. Then, the image extraction unit 110 extracts an image of the search box image P11 within a certain distance of the position of the cursor image P21 from the extracted frame image based on the position information of the cursor image P21. That is, for example, the image extraction unit 110 includes, for example, a flat portion in which a change in luminance of the image is small from within a range A2 of a distance L1 upward, a distance L2 downward, and a distance L3 leftward from the position of the tip A1 of the cursor image P21. An edge portion having a large luminance change is discriminated, and an image having a predetermined size (resolution) having a substantially rectangular edge portion is extracted.

  Here, as described above, in a commercial video or the like, the search box image P11 is often arranged in the vicinity left of the search button image P12. In many cases, the search box image has a rectangular shape whose length in the horizontal direction is longer than that in the vertical direction. Furthermore, the search box image often has a size (resolution) within a predetermined range that can be visually recognized by the user. Therefore, when the cursor image P21 is at a position corresponding to the search button image P12, the operation receiving unit 108 receives an operation input to a position corresponding to the position of the image P12, and the image extracting unit 110 is related to the search button image P12. The search box image P11 can be detected with high accuracy by detecting a substantially rectangular image of a predetermined size from within a certain range of the position, and the character image included in the search box image P11 can be detected and extracted.

  However, since the shape of the search box image P11 is not necessarily a rectangle, the image extraction unit 110 is not limited to a rectangular image, for example, a predetermined image such as an image having a circumferential edge or a horizontally long image. An image of the shape may be extracted. Here, the image extraction unit 110 may detect and extract the search box image P11 when an operation is performed on a position overlapping the search box image P11 or a position near the image P11. Also in this case, the image extraction unit 110 can detect the search box image P11 by detecting an image having a predetermined shape within a predetermined range from the position where the operation is received.

  Then, when extracting the search box image P11 as shown in FIG. 4C, the image extraction unit 110 extracts the character string images P30 and P40 included in the extracted image as shown in FIG. 4D. . The character extraction unit 111 extracts characters T1 and T2 included in the image P30 and the image P40.

FIG. 5 is a diagram illustrating a transition example of screens displayed when a search is executed using characters extracted by the display device 100.
First, the display device 100 displays a screen P20 shown in FIG. A search box image P11 and a search button image P12 are arranged on the screen P20 based on the video received by the receiving unit 101 or the video stored in the storage unit 102, and the cursor image P21 is displayed in response to an operation input from the user. Be placed. When the operation receiving unit 108 receives a button operation input from the pointing device 200, for example, the image extracting unit 110 and the character extracting unit 111 convert the character image included in the search box image P11 into character data. And the search part 113 performs a search using the said character data. And the display apparatus 100 will display the screen P50 shown in FIG.5 (B), if the information of a search result is acquired.

  FIG. 5B is a screen configuration example when the display device 100 displays the search result. On the screen P50, a reproduction video P60, a search result video P70, a history video P80, and the like are displayed. The reproduced video P60 is a video of video data decoded (reproduced) by the decoding unit 104, that is, video data received by the receiving unit 101 or video data stored in the storage unit 102.

  The search result video P70 is, for example, a window of browser software, and the web page P71 searched for, for example, is displayed on the video P70 as a search result acquired from the Internet server. The history video P80 is a video based on information stored in the extracted character database 112, and the character extraction time P82, program information P83, and the like are associated with the character P81 extracted by the character extraction unit 111. Is displayed.

  Note that the screen configuration example in the case of displaying the search result is not limited to this. For example, when an extraction instruction operation input from the user is received on the screen P20, the search result video P70 is displayed on the entire display screen of the display unit 107. It may be displayed on the screen.

FIG. 6 is a diagram illustrating a processing flow example related to character extraction by the display device 100.
First, the display device 100 reproduces the video data and outputs the reproduced video to the display unit 107 (S301). When the image extraction unit 110 receives an input of an extraction operation from the user (Yes in S302), the image extraction unit 110 detects a search box image P11 from within a certain range around the position where the cursor image P21 is displayed in the reproduced video. And extraction (S303). Subsequently, the image extraction unit 110 extracts a character image included in the extracted search box image P11 (S304). Then, the character extraction unit 111 analyzes the character image and converts it into character data (S305), and stores the character data in the extracted character database 112 (S306).

  Then, the search unit 113 executes a search on the Internet using the extracted character data (S307), and displays the search result screen on the display unit 107 when receiving the search result (S308). In the processing flow, when the display device 100 extracts a character, the display device 100 performs a search using the extracted character. However, the timing for executing the search is not limited to this. For example, the search may be executed when an operation input instructing the search is received from the user after the extracted character is stored in S306.

  Note that the display device 100 may receive a video reproduction stop instruction operation input from the user before receiving the extraction operation input in S302. In this case, the display device 100 displays the paused video on the display unit 107 and receives an extraction operation input from the user for the stopped video. In addition, when the receiving unit 101 is reproducing the video data being received, the display device 100 pauses the reproduction of the video and starts recording the video data when receiving the stop instruction operation. Playback of the recorded video can be started after receiving an operation input.

  In the present embodiment, the display device 100 has been described as receiving an operation input from, for example, the pointing device 200 or a remote controller. However, the display device 100 may receive an operation input from an information terminal such as a smartphone or a PC. . In this case, the information terminal is connected to the display device 100 via a network such as a wireless / wired LAN, and the operation input input to the information terminal is transmitted to the display device 100 via the network. Then, the display device 100 executes processing according to the input operation input.

(Second Embodiment)
Next, a second embodiment will be described with reference to FIGS. FIG. 7A is a diagram showing an example of usage of the display device 100b and the information terminal 300 according to the present embodiment. Here, the display device 100b is connected to the information terminal 300 via a network such as a wireless / wired LAN. The information terminal 300 is provided with a touch screen on the display screen of the display unit 309 and receives an operation input from the user. The information terminal 300 receives the user's touch operation input on the display screen, and transmits the received touch operation input to the display device 100b via the network.

  The display device 100b displays the screen P20 on the display unit 107b, similarly to the display device 100 in the first embodiment. On the screen P20, a search box image P11, a search button image P12, a cursor image P21, and the like are displayed. The position of the cursor image P <b> 21 changes according to a movement instruction operation input from the pointing device 200 or the information terminal 300. When receiving the extraction instruction operation input from the pointing device 200 or the information terminal 300, the display device 100b according to the present embodiment detects the search box image P11 from within a certain range of the position where the cursor image P12 is arranged. Thus, the characters included in the image P11 can be extracted. Then, the information terminal 300 can execute an Internet search using the extracted characters and display the search result.

  7B and 7C are examples of display screens displayed on the information terminal 300. FIG. On the screen P90, a frame image P91, a seek bar image P92, a stop button image P93, a playback button image P94, and the like are displayed. Here, the frame image P91 is an area for receiving an operation input for the cursor image P21 displayed by the display device 100b. The information terminal 300 functions as a touch pad for the display device 100b by transmitting an instruction corresponding to the touch operation on the region in the image P91 to the display device 100b. That is, the information terminal 300 outputs, to the display device 100b, a movement instruction for moving the position of the cursor image P21 and an extraction instruction for instructing character extraction in response to a touch operation on a region in the image P91.

  The seek bar image P92, the stop button image P93, and the playback button image P94 are images for instructing the video playback position, playback stop, and playback start when the display device 100b plays back the recorded video. The information terminal 300 transmits an instruction according to the user's touch operation on these buttons to the display device 100b. The screen P100 is a screen for displaying a search result, and displays information on a web page received from a server on the Internet according to the search. Although not shown in FIG. 7, the information terminal 300 may display an image for instructing to change the channel of the display program regarding the video of the broadcast program displayed by the display device 100b.

  FIG. 8 is a diagram illustrating a system configuration example of the display device 100b and the information terminal 300 according to the second embodiment. Here, the display device 100 b and the information terminal 300 are connected by a wireless or wired network 400.

  First, the display device 100b will be described. The display device 100b includes a receiving unit 101b, a storage unit 102b, a recording / playback control unit 103b, a decoding unit 104b, a combining unit 105b, a signal processing unit 106b, a display unit 107b, an operation receiving unit 108b, a GUI processing unit 109b, and an image extracting unit 110b. A character extraction unit 111b, an extracted character database 112b, a search unit 113b, a communication unit 114b, and the like. Since each configuration of the display device 100b has the same function as that of the first embodiment, the description here will focus on functions different from those of the first embodiment.

  When receiving an extraction instruction operation input, the GUI processing unit 109b instructs the image extraction unit 110 to extract an image. The GUI processing unit 109 outputs the position information of the cursor image when receiving the extraction instruction operation input to the image extraction unit 110b.

  The image extraction unit 110b extracts a frame image corresponding to the extraction instruction operation input from the video data input from the decoding unit 104b. Then, the image extraction unit 110b extracts an image of a search box image that is within a certain distance from the position of the cursor image in the extracted frame image based on the position information of the cursor image. Further, the image extraction unit 110b extracts a character image included in the search box image. Then, the image extraction unit 110b outputs the extracted character image to the character extraction unit 111 or the communication unit 301 of the information terminal 300.

  The character extraction unit 111b generates character data (text data) from the input character image. Here, the program information about the video data being reproduced is input to the character extraction unit 111b from the recording / reproduction control unit 103b. Then, the character extraction unit 111b outputs the extracted character data and the program information about the video data from which the character data is extracted to the extracted character database 112b or the communication unit 301 of the information terminal 300.

  The extracted character database 112b stores character data input from the character extraction unit 111b and program information in association with each other. The search unit 113b performs a search using the character data extracted by the character extraction unit 111b.

  When the communication unit 114b receives, from the information terminal 300, a movement instruction operation input for moving the position of the cursor image P21, an extraction instruction operation input for instructing character extraction, a playback / stop operation input of recorded video data, These operation inputs are output to the operation receiving unit 108b. Then, the operation reception unit 108b outputs the received operation input to the GUI processing unit 109b and the recording / playback control unit 103b. The communication unit 114b also has a function of transmitting any of the frame image extracted by the image extraction unit 110b, the search box image, the character image, and the character data extracted by the character extraction unit 111b to the communication unit 301 of the information terminal 300. Have. In addition, when transmitting the frame image extracted by the image extraction unit 110b to the communication unit 301, the communication unit 114b also transmits the position information of the cursor image.

  Next, a system configuration example of the information terminal 300 will be described. The information terminal 300 includes a communication unit 301, an image extraction unit 302, a character extraction unit 303, a search unit 304, a GUI processing unit 305, a signal processing unit 306, a display unit 307, an operation reception unit 308, and the like.

  The communication unit 301 records, from the GUI processing unit 305, a movement instruction operation input for instructing movement of the position of the cursor image P21 displayed on the display device 100b, an extraction instruction operation input for instructing character extraction, and the display device 100b recorded. When an operation input for instructing reproduction and stop of video data is input, the operation input is transmitted to the communication unit 114b of the display device 100b via the network 400.

  Further, when any one of the frame image and the search box image is input from the display device 100b, the communication unit 301 outputs the image to the image extraction unit 302. Note that the communication unit 301 also outputs the information to the image extraction unit 302 when the position information of the cursor image is input. The communication unit 301 outputs the character image to the character extraction unit 303 when a character image is input from the display device 100b, and outputs the character data to the search unit 304 when character data is input.

  When the frame image and the cursor position information are input from the communication unit 301, the image extraction unit 302 detects and extracts a search box image from within a certain range of the position indicated by the cursor position information in the frame image. Further, when a search box image is input from the communication unit 301 or when the image extraction unit 302 extracts a search box image, the image extraction unit 302 extracts a character image included in the search box image, and extracts the character image. The data is output to the character extraction unit 303.

  When a character image is input from the communication unit 301 or the image extraction unit 302, the character extraction unit 303 detects characters included in the character image, generates character data, and outputs the character data to the search unit 304.

  When character data is input from the communication unit 301 or the character extraction unit 303, the search unit 304 performs an Internet search using the character data and acquires a search result from a server on the Internet. Then, the search unit 304 instructs the GUI processing unit 305 to generate a search result display screen.

  The GUI processing unit 305 generates video data of the screens P90 and P100 shown in FIGS. 7B and 7C. Further, position information of operation input on the screen of the display unit 307 is input from the operation reception unit 308 to the GUI processing unit 305. Here, the GUI processing unit 305 determines which image on the display screen has been operated according to the position information, and instructs the communication unit 301 to transmit an instruction corresponding to the received image.

The signal processing unit 306 converts the video data into a video signal for a display device, and the display unit 307 outputs a video using the video signal.
The operation accepting unit 308 is, for example, a touch panel provided on the screen of the display unit 307, and accepts an operation input from the user. Then, the operation reception unit 308 outputs information about the position (coordinates) of the screen that has received the operation to the GUI processing unit 305.

  Next, a processing sequence example by the display device 100b and the information terminal 300 will be described with reference to FIG. First, the display device 100b outputs an image of the screen P20 (S501). When the display device 100b receives an extraction instruction from the pointing device 200 or the information terminal 300 (Yes in S502), the display device 100b extracts a frame of video data being reproduced (S503). Then, the display device 100b extracts a search box image from the range of the extracted frame based on the position information of the cursor image P21 (S504).

  Subsequently, the display device 100b extracts a character image from the search box image, and detects characters included in the character image as character data (S506). Next, the display device 100b transmits the detected / extracted character data to the information terminal 300 (S507). The information terminal 300 executes a search using the received character data (S508), and displays a screen showing the search result (S509).

  In FIG. 9, the display device 100b has been described as transmitting character data to the information terminal 300, but the data that the display device 100b transmits to the information terminal 300 is not limited to this. That is, the display device 100b may transmit the frame image extracted in S503 to the information terminal 300, and the information terminal 300 may process the subsequent processes of S504 to S506. The display device 100b may transmit the search box image extracted in S504 to the information terminal 300, and the information terminal 300 may execute the processes of S505 and S506. Further, the display device 100b may transmit the character image extracted in S505 to the information terminal 300, and the information terminal 300 may execute the process of S506.

(Third embodiment)
Next, a third embodiment will be described with reference to FIGS. The third embodiment is realized by the display device 100b and the information terminal 300 according to the second embodiment.
FIG. 10 is a diagram illustrating a transition example of screens displayed on the display device 100b and the information terminal 300 according to the third embodiment. First, the information terminal 300 displays the screen P200. On the screen P200, an extraction instruction button P201 and the like are displayed. If the information terminal 300 accepts an operation on the extraction instruction button P201 when the display device 100b displays the screen P300, the information terminal 300 instructs the display device 100b to extract and transmit the screen P300. The display device 100b extracts the frame image of the screen 300 and transmits it to the information terminal 300. Note that the display device 100b continues to reproduce and display video even when extracting and transmitting frame images.

  When receiving the frame image, the information terminal 300 displays the frame image on the screen P210 and accepts a touch operation input from the user. When the information terminal 300 receives an operation on the screen P210, the information terminal 300 detects an image having a predetermined shape within a certain range of the position where the operation is received from the image on the screen P210, and includes the image in the predetermined shape image. Extract included characters as character data. Here, the display device 100b continues to reproduce and display the video, and displays the screen P310 of the video being reproduced.

  Subsequently, the information terminal 300 transmits a search request based on the extracted character data to the Internet server, and displays information received from the Internet server in response to the request on the screen P220. Here, the display device 100b continues to reproduce and display the video as described above, and displays the screen P320 of the video being reproduced.

  Next, an example of a processing sequence by the display device 100b and the information terminal 300 will be described with reference to FIG. First, the display device 100b reproduces and displays a video (S601), and the information terminal 300 displays an operation screen such as the screen P200 shown in FIG. 10 (S602). When the information terminal 300 accepts an operation on the extraction instruction button P201 (Yes in S603), the information terminal 300 transmits a frame extraction instruction to the display device 100b (S604). Subsequently, when receiving a frame extraction instruction, the display device 100b extracts a frame image of the video being reproduced (S605) and transmits the frame image to the information terminal 300 (S607).

  The information terminal 300 displays the received frame image (S607), and when receiving an operation input for the display image (Yes in S608), the information terminal 300 is within a certain range of the position corresponding to the operation input in the display image. An image having a predetermined shape is extracted (S609). Next, the information terminal 300 extracts a character image from the image having the predetermined shape (S610), and converts characters included in the character image into character data (S611). Then, the information terminal 300 performs a search using the character data (S612), and displays information obtained by the search (S613).

  In FIG. 11, the information terminal 300 has been described as executing the processes of S609 to S611, but these processes may be executed by the display device 100b. In this case, when receiving an operation input for the display image in S608, the information terminal 300 determines which position of the display image the operation input corresponds to and displays the position information of the operation input on the display device 100b. Send. Then, the display device 100b extracts an image having a predetermined shape within a certain range of the position indicated by the position information, from the frame image extracted in S605, similarly to the process in S609. Subsequently, the display device 100b extracts a character image and character data, and transmits the character data to the information terminal 300.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. For example, the character extraction device according to the embodiment may be realized as a reception device that includes a tuner and outputs an image on a display. Also, these embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 100 ... Display apparatus 101 ... Receiving part 102 ... Memory | storage part 103 ... Recording / reproducing control part 104 ... Decoding part 105 ... Synthesis | combination part 106 ... Signal processing part 107 ... Display part 108 ... Operation reception part 109 ... GUI processing unit, 110 ... image extraction unit, 111 ... character extraction unit, 112 ... extracted character database, 113 ... search unit, 114 ... communication unit, 200 ... pointing device, 300 ... information terminal, 301 ... communication unit, 302 ... Image extracting unit 303 ... Character extracting unit 304 ... Searching unit 305 ... GUI processing unit 306 ... Signal processing unit 307 ... Display unit 308 ... Operation receiving unit 400 ... Network

Claims (15)

Playback means for playing back video data;
Output means for outputting the video data reproduced by the reproduction means to a display device;
Accepting means for accepting an external input corresponding to any position of the video displayed by the display device using the video data;
Image extracting means for extracting an image of a first shape within a certain range from the position corresponding to the external input in the video displayed by the display device;
A character extraction device comprising: character extraction means for extracting characters from the image extracted by the extraction means.   The character extracting apparatus according to claim 1, wherein the image extracting unit extracts the image having a substantially rectangular shape.   The character extracting apparatus according to claim 1, wherein the image extracting unit extracts the image having a certain size.   The character extraction device according to claim 1, wherein the image extraction unit extracts the image in a left direction within a certain range from the position of the external input.   The character extraction device according to claim 1, wherein the output unit outputs video data of an image indicating a position corresponding to the external input and the reproduced video data to the display device. Further comprising search means for performing a search using the characters extracted by the extraction means;
The character extraction device according to claim 1, wherein the output unit outputs video data of search information obtained by the search to the display device.   The character extraction device according to claim 1, wherein the output unit outputs both the reproduced video data and video data for the search information to the display device.   The character extraction device according to claim 1, further comprising character storage means for storing the extracted character.   The character extraction device according to claim 8, wherein the output unit outputs video data indicating the one or more characters stored in the character storage unit to the display device. The reproducing means stops the reproduction of the video data when the receiving means receives a second external input different from the external input;
The character extracting device according to claim 1, wherein the accepting unit accepts the external input after the reproduction of the video data is stopped. Receiving means for receiving video data of a broadcast program;
Video storage means for storing the received video data in a storage device;
The reproduction means reproduces the received video data,
The character extraction device according to claim 1, wherein the video storage unit stores the received video data in the storage device when the reproduction unit stops the reproduction of the video data.   The character extracting device according to claim 1, wherein the accepting unit accepts the external input from an external device connected to the character extracting device via a network. A display method in a display system comprising a first device and a second device as information processing devices, wherein the first device and the second device are connected by a network,
The first device comprises:
Playing video data,
Displaying video using the played video data; and
The first device or the second device is
Receiving an external input corresponding to any position of the displayed video;
Extracting a first shape image within a certain range from a position corresponding to the external input in the displayed video;
Extracting characters from the extracted image of the first shape;
Performing a search using the extracted characters;
The second device comprises:
Displaying the information obtained by the search. The second device accepts the external input;
The said 1st apparatus is further provided with displaying the said image | video which does not contain the cursor image which shows the position corresponding to the said external input, when the said 2nd apparatus is accepting the said 1st external input. Display method of description. A character extraction method in a device for extracting characters from video displayed on a display device,
Playing video data,
Outputting the reproduced video data to the display device;
Receiving an external input corresponding to any position of the video displayed by the display device using the video data;
Extracting an image of a first shape within a certain range from a position corresponding to the external input in the video displayed by the display device;
A character extraction method comprising: extracting characters from the extracted first shape image.

Images