JP2014033257A - Information terminal device and program therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information terminal device capable of simplifying search operation of a user when an image and sound are output for urging the user to perform Internet searches in viewing of television broadcast.SOLUTION: A TV broadcast content data acquisition unit 6A acquires television broadcast content data and an identification information discrimination unit 71 discriminates whether identification information indicative of urging the user to perform Internet searches is included in the television broadcast content data. On the basis of the timing at which the identification information is discriminated, a keyword extraction unit 72 extracts a search keyword from the television broadcast content data. The search keyword extracted in this way is transmitted to a search site by a keyword transmission unit 81 and a search result received from the search site is displayed on a display 3 by a display control unit 9.

Description

  The present invention relates to an information terminal device and a program thereof, and more particularly to an improvement of an information terminal device that performs keyword search on the Internet in conjunction with television broadcasting.

  For example, in a commercial message (hereinafter referred to as CM) of a television broadcast, a short character string is presented to facilitate access to a website related to product information and the like, and this character string is searched on the Internet (hereinafter referred to as a search keyword). , Internet search) is widely promoted. In order to perform such a net search, the user himself / herself needs to input a search keyword to a search site on the Internet.

  Therefore, a television apparatus that can automatically extract a character string from television broadcast content data and perform a net search using the character string is conventionally known (for example, Patent Document 1). Patent Document 1 describes a television device that uses an OCR technique to extract a character string included in a television image and perform a net search using the character string. It also describes that a character string is extracted from television sound and a net search is performed using the character string as a search keyword.

  However, the television device of Patent Document 1 is configured such that a character string is extracted from television broadcast content data when a user presses a key on a remote control device. For this reason, there is a problem that the timing for extracting the search keyword has to be specified by the user himself and is complicated.

  Further, in the case of a CM that promotes access to a website, the search keyword is presented for several seconds, and the user must perform key operations on the remote control device within such a short period. For this reason, it is not easy to perform an operation at an appropriate timing, and there is a problem that it is not possible to relax and watch a television broadcast.

JP 2010-141868 A

  The present invention has been made in view of the above circumstances, and automatically extracts a search keyword at a timing when the television broadcast prompts a net search without the user instructing a timing for extracting the search keyword, An object of the present invention is to provide an information terminal device that displays the search result.

  It is another object of the present invention to provide an information terminal device that can easily access a website presented in a television broadcast while relaxing and watching the television broadcast.

  Furthermore, an object of the present invention is to provide an information terminal device capable of performing a process of extracting a search keyword with high accuracy at a timing when a net search is promoted in a television broadcast.

  In the information terminal device according to the first aspect of the present invention, there is provided content acquisition means for acquiring television broadcast content data comprising television image data or television audio data, and identification information indicating that keyword search on the Internet is promoted. Identification information determining means for determining whether or not it is included in broadcast content data; keyword extracting means for extracting a search keyword from the television broadcast content data based on the timing at which the identification information is determined; It comprises keyword transmitting means for transmitting the search keyword to the search site, and search result display means for displaying based on the search result received from the search site.

  According to such a configuration, the search keyword is automatically extracted based on the timing when the identification information indicating that the net search is promoted is determined without the user instructing the timing for extracting the search keyword. In addition, display based on the search result can be performed. In particular, since it is not necessary to perform key operations in a short period of time when the search keyword is presented, the user can easily access the website presented in the television broadcast while relaxing and watching the television broadcast. Can do.

  In addition to the above configuration, the information terminal device according to the second aspect of the present invention is configured such that the identification information determination means determines whether or not the identification information is included in the television audio data.

  According to such a configuration, the search keyword can be extracted from the television broadcast content data based on the timing at which it is determined that the keyword search on the Internet is prompted in the television audio data.

  In addition to the above configuration, the information terminal device according to the third aspect of the present invention is configured such that the keyword extraction means extracts the search keyword from the television image data.

  According to such a configuration, it is possible to acquire the timing for extracting the search keyword based on the television audio data, and to extract the search keyword from the television image data based on the timing. Therefore, compared to the case where the search keyword is automatically extracted based on only one of the TV audio data or the TV image data, the process of extracting the search keyword at a timing when the TV broadcast prompts the Internet search with high accuracy. It can be carried out.

  In addition to the above-described configuration, the information terminal device according to the fourth aspect of the present invention is configured such that the identification information includes a click sound. According to such a configuration, it is possible to acquire the timing for extracting the search keyword by determining whether or not the click sound is included in the TV sound data. Therefore, it is possible to accurately determine the timing for extracting the search keyword. In addition, it is possible to reduce the processing load for acquiring the timing for extracting the search keyword.

  In addition to the above configuration, the information terminal device according to the fifth aspect of the present invention is configured such that the keyword extraction means extracts a character string displayed inside a search frame as the search keyword. According to such a configuration, since the character string displayed inside the search frame is extracted as the search keyword, it is possible to suppress the search keyword from being erroneously extracted.

  In addition to the above configuration, the information terminal device according to the sixth aspect of the present invention is configured such that the search result display means displays the search result.

  In addition to the above configuration, the information terminal device according to the seventh aspect of the present invention is configured such that the search result display means displays a link destination website of link information included in the search result.

  An information terminal device according to an eighth aspect of the present invention includes, in addition to the above configuration, the content acquisition unit including a tuner that receives a television broadcast wave, and a decoder that converts the received television broadcast wave into the television broadcast content data. Configured as follows. According to such a configuration, a search keyword can be extracted based on a television broadcast wave, and a website presented as a CM can be easily accessed.

  In an information terminal device according to a ninth aspect of the present invention, in addition to the above-described configuration, the content acquisition unit includes an imaging unit for capturing a television image displayed on the display of the television receiver, and a television sound of the television receiver. Voice input means for inputting. According to such a configuration, the information terminal device extracts the search keyword based on the output of the television receiver without receiving the television broadcast wave, and easily accesses the website presented by the CM. Can do.

  In an information terminal device according to a tenth aspect of the present invention, in addition to the above configuration, the television broadcast content data includes television image data, and the identification information determination means includes the identification information in the television image data. It is configured to determine whether or not. According to such a configuration, it is possible to extract the search keyword from the television broadcast content data based on the timing determined that the search on the Internet is prompted by the television image data.

  A program for an information terminal device according to an eleventh aspect of the present invention is a content acquisition step for acquiring TV broadcast content data comprising TV image data or TV audio data, and identification information indicating that keyword search on the Internet is promoted An identification information determination step for determining whether or not the television broadcast content data is included, a keyword extraction step for extracting a search keyword from the television broadcast content data based on the timing at which the identification information is determined, This comprises a procedure for causing a computer to execute a keyword transmission step for transmitting the search keyword to a search site on the Internet and a search result display step for displaying a search result received from the search site.

  The information terminal device according to the present invention uses the search keyword from the television broadcast content data based on the timing when it is determined that the identification information indicating that the keyword search on the Internet is prompted is included in the television broadcast content data. To extract.

  Therefore, it is possible to automatically extract the search keyword at the timing when the television broadcast prompts the Internet search without displaying the timing for extracting the search keyword, and to display the search result. In addition, the user can easily access the website presented in the television broadcast while relaxing and watching the television broadcast.

  Moreover, the information terminal device according to the present invention extracts a search keyword from the television image data based on the timing when it is determined that the identification information is included in the television audio data.

  Therefore, compared to the case where the search keyword is automatically extracted based on only one of the TV audio data or the TV image data, the process of extracting the search keyword at a timing when the TV broadcast prompts the Internet search with high accuracy. It can be carried out.

1 is a system diagram showing an example of a TV-linked search system 100 including an information terminal device according to Embodiment 1 of the present invention. FIG. 2 is a block diagram showing an example of an internal configuration of the mobile phone 1A of FIG. FIG. 3 is a block diagram illustrating an example of a detailed configuration of a keyword specifying unit 7 in FIG. 2. It is the figure which showed an example of the television image which promotes the internet search displayed on the display 3 of FIG. It is the figure which showed an example of the search result display screen displayed on the display 3 of FIG. A search genre selection screen displayed on the display 3 of FIG. 1 is shown. 3 is a flowchart showing an example of processing at the time of receiving a television broadcast in the mobile phone 1A of FIG. It is the figure which showed the mode at the time of acquiring television broadcast content data using the mobile telephone 1B by Embodiment 2 of this invention. It is the block diagram which showed an example of the internal structure of the mobile telephone 1B of FIG. 9 is a flowchart showing an example of an operation in the mobile phone 1B of FIG. It is the block diagram which showed an example of the internal structure of the mobile telephone 1C by Embodiment 4 of this invention.

Embodiment 1 FIG.
FIG. 1 is a system diagram showing an example of a TV-linked search system 100 including an information terminal device according to Embodiment 1 of the present invention. A mobile phone 1A is shown as an example of the information terminal device. The TV-linked search system 100 includes a mobile phone 1A, a broadcast station 20, a cellular base station 21, a network 22, and a number of web servers.

  The mobile phone 1A can receive a television broadcast wave from the broadcast station 20, display a television image, and output a television sound. In addition, web servers such as the search server 23 and the product information server 24 can be accessed via the cellular base station 21 and the network 22, and page data provided by these web servers can be received and displayed.

  The search server 23 is a web server constituting the search site, receives a search request including a search keyword from a client terminal, and receives page data including link information to a website highly relevant to the search keyword. To provide. Therefore, if the mobile phone 1A transmits a search request designating a search keyword to the search server 23, the mobile phone 1A can receive page data of the search result and display the search result. As such a search site, for example, Google and Yahoo are known (both are registered trademarks).

  The merchandise information server 24 is a web server of a merchandise information site that provides page data including merchandise information, and provides page data including merchandise information to the client terminal based on a browsing request from the client terminal. For this reason, if the mobile phone 1A transmits a browsing request to the product information server 23, the mobile phone 1A can receive the page data of the product information and display the product information.

  The cellular phone 1A is formed of a substantially rectangular thin casing, and a display 3 is disposed on the front surface thereof. The display 3 is display means for displaying an image, and a touch sensor for detecting a user's touch operation is formed on the display surface.

  When the mobile phone 1A receives a TV broadcast that prompts the user to search the Internet while watching TV, and the search keyword is displayed on the display 3, the search keyword is automatically extracted and the search server 23 receives the search keyword. Perform a search request with a keyword specified.

  FIG. 2 is a block diagram showing an example of the internal configuration of the mobile phone 1A of FIG. The cellular phone 1A includes a display 3, a TV broadcast content data acquisition unit 6A, a keyword designation unit 7, a cellular communication unit 8, and a display control unit 9.

  The TV broadcast content data acquisition unit 6A is an acquisition unit that acquires television broadcast content data based on a television broadcast wave. The television broadcast content data is composed of television image data or television audio data included in the television broadcast wave. The TV broadcast content data acquisition unit 6A receives a TV broadcast wave transmitted from the broadcast station 20, and generates TV broadcast content data from the TV broadcast wave.

  The TV broadcast content data acquisition unit 6A includes a tuner 61 and a decoder 62. The tuner 61 performs a process of receiving a television broadcast wave from the broadcast station 20 via a television broadcast receiving antenna. The decoder 62 performs a process of converting the received signal output from the tuner 61 into television broadcast content data.

  The keyword specifying unit 7 is a keyword specifying means for specifying a search keyword for making a search request to the search site 23 based on the television broadcast content data. Specifically, a character string included in TV image data or TV audio data is extracted and specified as a search keyword.

  The cellular communication unit 8 is a communication unit that transmits and receives data to and from the search server 23 via the cellular base station 21 and the network 22. Here, it is assumed that the search keyword specified by the keyword specifying unit 7 is transmitted to the search server 23 and page data indicating the search result is received from the search server 23.

  The cellular communication unit 8 includes a keyword transmission unit 81 and a page data reception unit 82. The keyword transmitting unit 81 performs processing for transmitting the search keyword specified by the keyword specifying unit 7 to the search server 23. The page data receiving unit 82 performs a process of receiving page data indicating a search result from the search server 23.

  The display control unit 9 is a control unit that performs display control of the display 3. The display control unit 9 and the display 3 constitute search result display means for displaying the page data received by the page data receiving unit 82.

  FIG. 3 is a block diagram showing an example of a detailed configuration of the keyword specifying unit 7 of FIG. The keyword specifying unit 7 includes an identification information determining unit 71, a keyword extracting unit 72, a keyword determining unit 73, a genre information holding unit 74, a genre selecting unit 75, and an unselected genre holding unit 76.

  The identification information determination unit 71 is a determination unit that determines whether or not the identification information is included in the television broadcast content data. The identification information is information included in the television broadcast content data, and is information indicating that the television broadcast content prompts the Internet search, and is determined in advance. For this reason, if it can be determined that the identification information is included in the TV broadcast content data, it can be determined that the TV broadcast content at that time prompts the keyword search.

  This identification information may be information included in either television audio data or television image data. The identification information included in the TV sound data includes, for example, sounds such as “Continue with WEB” and “Search by”, and click sounds indicating a search operation by a search website. The identification information included in the TV image data includes, for example, a search button for performing a search.

  Here, it is assumed that the identification information determination unit 71 performs a process of determining whether or not the identification information is included in the television audio data. For example, a click sound output when a search button is clicked is determined in a television broadcast composed of a moving image simulating a screen display when a search request is made at a search site. The determination of the click sound can be performed, for example, by detecting sound in a predetermined frequency band. Alternatively, the TV voice data is analyzed by a well-known voice recognition process, a character string including a series of character codes is generated, and the generated character string is identified with identification information such as “continue by WEB” or “search by”. Determine whether they match.

  The keyword extraction unit 72 is a keyword extraction unit that extracts search keyword candidates from the television broadcast content data based on the determination result of the identification information determination unit 71. The identification information determination unit 71 always determines whether identification information is included in the television broadcast content data. As a result, when it is determined that the identification information is included in the television broadcast content data, the keyword extraction unit 72 extracts search keyword candidates from the television broadcast content data based on the timing at which the determination is performed.

  Here, it is assumed that the keyword extraction unit 72 extracts search keyword candidates from the television image data by OCR (Optical Character Recognition) processing. In the OCR process, a character string including a series of character codes is generated by recognizing characters included in the television image data. As a method for character recognition by analyzing image data, for example, a pattern matching method is used. Specifically, the character recognition is performed by comparing the image data with a predetermined character pattern and repeatedly specifying the character based on the comparison result. In order to suppress the extraction of an erroneous character string, it is desirable to extract a character string within a search frame for inputting a search keyword as a search keyword candidate.

  In this way, when the identification information determination unit 71 determines that the identification information is included in the TV sound data, the character string included in the TV image data corresponding to the sound at that time is used as a search keyword candidate as the keyword extraction unit. 72. For this reason, even if the user does not instruct the timing for extracting the search keyword, the search keyword can be automatically extracted at the timing when the TV broadcast prompts the Internet search, and the user can relax and broadcast the TV broadcast. Can watch.

  The keyword determination unit 73 performs a process of determining whether or not the search keyword candidate extracted by the keyword extraction unit 72 matches the character string held in the genre information holding unit 74. The genre information holding unit 74 holds a character string that can be specified as a search keyword.

  Specifically, when the search keyword candidate matches the character string held in the genre information holding unit 74, the keyword determination unit 73 designates the search keyword candidate as the search keyword. On the other hand, if the search keyword candidate does not match the character string held in the genre information holding unit 74, the search keyword candidate is not designated as the search keyword.

  The genre selection unit 75 is a selection unit that stores a character string that can be specified as a search keyword in the genre information holding unit 74 based on a user operation. Among the character strings held in the unselected genre holding unit 76, The character string selected by the user is stored in the genre information holding unit 74. The unselected genre holding unit 76 holds two or more character strings in association with genres. For example, a character string of “health” and “genki” is associated with the genre of “health”, and when the user selects a genre of “health”, the characters of “health” and “genki” The columns are stored in the genre information holding unit 74 as character strings that can be designated as search keywords.

  FIG. 4 is a diagram showing an example of a television image that prompts the user to search the net displayed on the display 3 of the mobile phone 1A shown in FIG. This image is displayed, for example, while viewing the CM.

  In this television image, a search frame 101, a search button 102, and a cursor 103 are displayed. These images are images imitating the display on the search site. That is, the search frame 101 is a horizontally long rectangular frame that can display only one line of character string, the search button 102 is a rectangular button arranged adjacent to the right side of the search frame 101, and the cursor 103 is a mouse cursor. It is made up of images. Here, a character string “XYZ company new product” is displayed in the search frame 101 as a search keyword. In this CM, an operation for moving the cursor 103 to the position of the search button 102 and pressing the search button 102 is shown by a moving image, and a sound and click sound “Search by new product of XYZ company” are heard from a speaker (not shown). Is output from.

  FIG. 5 is a diagram showing an example of a search result display screen displayed on the display 3 of the mobile phone 1A of FIG. This search result display screen is displayed when page data indicating a search result is received from the search server 23.

  Near the upper end of the search result display screen, a search frame 111 and a search button 112 are displayed. In the search frame 111, a character string “XYZ new product” designated as a search keyword in this search is displayed. Below that, a sponsor site associated with the search keyword “XYZ new product” is displayed. In this example, a website with the title “XYZ company” is displayed as a sponsor site.

  Also, below the sponsor site, a list of websites highly relevant to the search keyword “XYZ company new product” is displayed. These websites are displayed from the top in descending order of the relevance to the search keyword “XYZ new product”. In this example, the website with the title “XYZ company new product” is the search keyword “XYZ new product”. Is listed as the most relevant website. When an Internet search is prompted by a television broadcast commercial, a website intended to be accessed by viewers of the commercial creator is often displayed as the most relevant website.

  FIG. 6 shows a search genre selection screen displayed on the display 3 as an example of the operation of the mobile phone 1A of FIG. This search genre selection screen is displayed when a user operation is performed to select a character string that can be specified as a search keyword.

  On the search genre selection screen, genres that can be selected by the user are arranged. Two or more character strings are associated with this genre, and if the user selects any genre, the character string associated with the selected genre can be designated as a search keyword. In addition, the genre selected by the user is highlighted at the periphery. In this example, eight genres of “beverage”, “candy”, “health”, “car”, “real estate”, “insurance”, “medicine”, and “beauty” are displayed. And two genres of “sweets” are selected. For this reason, the character strings “water”, “coffee” and “tea” associated with the genre “beverage” and the character strings “insurance” and “mutual aid” associated with the genre “insurance” Can be specified as.

  Steps S101 to S112 in FIG. 7 are flowcharts showing an example of processing at the time of television broadcast reception in the mobile phone 1A in FIG. This flowchart is started when the user activates the tuner 61 and a television broadcast wave can be received.

  First, the identification information discriminating unit 71 starts a voice recognition process based on the received television voice data (step S101). By this voice recognition processing, character strings included in the TV voice data are sequentially extracted. Next, whether or not identification information is included in the extracted character string is determined by the identification information determination unit 71 (step S102). If the identification information is included, the keyword extraction unit 72 starts OCR processing of the television image data (step S103). Further, the keyword extraction unit 72 determines a search frame included in the TV image data, and extracts a character string in the search frame as a search keyword candidate (steps S104 and S105).

  When the search keyword candidate is extracted, the keyword determination unit 73 determines whether or not a search using the extracted search keyword candidate has already been performed (step S106). If a search using the extracted search keyword candidates has already been performed, the process returns to step S102. On the other hand, if the search using the extracted search keyword candidate has not been performed yet, the keyword determination unit 73 determines whether or not the search keyword candidate is a character string that can be designated as the search keyword (step S31). S107). If the extracted search keyword candidate is not a character string that can be specified as a search keyword, the process returns to step S102.

  On the other hand, if the search keyword candidate is not a searched character string and is a character string that can be specified as a search keyword, the keyword transmission unit 81 transmits this character string to the search server 23 as a search keyword. A website highly relevant to the search keyword is searched for by the search server 23 (step S108). Thereafter, when the page data receiving unit 82 receives page data indicating the search result from the search server 23, the display control unit 9 displays the page data on the display 3, and a speaker (not shown) searches. The completion is notified to the user (steps S109 to S111). Further, the touch sensor (not shown) determines whether or not the user has pressed the search process stop button displayed on the display 3 (step S112). If the stop button is pressed, the process ends. On the other hand, if the stop button is not pressed, the process returns to step S102.

  In the mobile phone 1 according to the present embodiment, the identification information determination unit 71 determines whether or not the Internet audio data is urged to be searched by determining whether or not the identification information is included in the television audio data. Yes. Based on the determination result, the keyword specifying unit 7 extracts a character string from the television image data and specifies it as a search keyword.

  Therefore, the search keyword can be automatically extracted even if the user does not instruct the timing for extracting the search keyword. In particular, in a television broadcast CM, a search keyword is displayed for a few seconds, and it is not necessary to perform a key operation quickly in such a short period. Therefore, the user can easily access the website presented by the CM while relaxing and watching the television broadcast.

  Further, it is determined based on the television audio data that the television broadcast prompts the Internet search, and the search keyword is extracted from the television image data based on the determination timing. For this reason, it can suppress extracting character strings other than a search keyword accidentally, and the automatic extraction process of a search keyword can be implement | achieved with high precision.

  In the present embodiment, the configuration has been described in which the identification information determination unit 71 determines whether or not the identification information is included in the television audio data by the voice recognition process. However, the present invention is not limited only to such a case. For example, it may be configured to determine whether or not a click sound is included as identification information in the TV sound data. By adopting such a configuration, it is possible to reduce the processing load due to voice recognition while accurately determining the TV voice data that prompts the user to perform keyword search.

  In the present embodiment, the configuration in which the keyword extraction unit 72 extracts a character string in the search frame as a search keyword candidate has been described. However, the present invention is not limited only to such a case. For example, the keyword extraction unit 72 may extract a character string displayed on the left side of the search button as a search keyword candidate.

Embodiment 2. FIG.
In the first embodiment, an example in which television broadcast content data is acquired by receiving a television broadcast wave using the antenna of the information terminal device has been described. In contrast, in the present embodiment, an example will be described in which television broadcast content data output from a television receiver different from the information terminal device is acquired using a camera and a microphone of the information terminal device.

  FIG. 8 is a diagram showing a situation when television broadcast content data is acquired using the information terminal device according to Embodiment 2 of the present invention, and a mobile phone 1B is shown as an example of the information terminal device. . The television receiver 120 receives a television broadcast wave, displays a television image on the display 121, and outputs television sound from the speaker 122.

  The mobile phone 1 </ b> B is arranged in front of the television receiver 120 with the back of the housing facing the television receiver 120. A microphone 63 and a camera 64 (not shown) are provided on the rear surface of the casing of the mobile phone 1B. For this reason, the mobile phone 1B receives the TV broadcast wave by collecting the TV sound output from the TV receiver 120 with the microphone 63 and capturing the TV image output from the TV receiver 120 with the camera 64. TV broadcast content data is acquired without doing so.

  The support device 130 is a photographing direction adjustment unit that supports the photographing direction of the camera of the mobile phone 1B so as to be adjustable, and the mobile phone 1B so that the TV image displayed on the display 121 is included in the photographing range of the camera. Adjust the orientation. Note that the support device 130 may be configured as a part of the mobile phone 1B.

  FIG. 9 is a block diagram showing an example of the internal configuration of the mobile phone 1B of FIG. The mobile phone 1B includes a display 3, a TV broadcast content data acquisition unit 6B, a keyword designation unit 7, a cellular communication unit 8, and a display control unit 9. The same components as those of the cellular phone 1A in FIG.

  The TV broadcast content data acquisition unit 6B captures a TV image displayed on the display 121 of the TV receiver 120 and inputs TV sound output from the speaker 122, thereby acquiring TV broadcast content data. It is.

  The TV broadcast content data acquisition unit 6B includes a microphone 63 and a camera 64. The microphone 63 is an audio input device for inputting television audio from the television receiver 120. Note that the microphone 63 can also share a call microphone provided on the front surface of the housing, but may be a microphone different from the call microphone.

  The camera 64 is an imaging unit for capturing a television image displayed on the display 121, and generates television image data including a moving image. Note that the camera 64 may capture a television image as a still image at a predetermined timing.

  Steps S201 to S215 in FIG. 10 are flowcharts showing an example of processing at the time of shooting a television image in the mobile phone 1B in FIG. This flowchart is started when the user activates the camera module and can capture a television image on the display 121.

  First, the camera 64 starts capturing a television image of the television receiver 120 (step S201). Next, the microphone 63 starts input of TV sound output from the speaker 122 (step S202). Next, the keyword extraction unit 72 determines the shooting state of the television image (step S203). If the size and angle of the TV image to be captured are out of the allowable range, an error display prompting the user to adjust the size and angle of the TV image is displayed, and the determination in step S203 is performed again (step S204).

  When the size and angle of the TV image to be photographed are within the allowable range, each process of steps S205 to S215 is performed. The processes in steps S205 to S215 are the same as those in steps S102 to S112 in FIG. In this flowchart, when the search by the extracted search keyword candidate has already been performed (step S209), when the search keyword candidate is not a character string that can be designated as the search keyword (step 210), or when a stop button is displayed. If not pressed (step S215), the process returns to the determination of step S203.

  According to the present embodiment, since mobile phone 1B does not need to receive television broadcast waves, television broadcast content data includes image data that prompts a keyword search or audio data that prompts a keyword search. In addition, it is possible to suppress the complexity of the mobile phone 1B while simplifying the user's search operation.

  In the present embodiment, a configuration has been described in which imaging of a television image and input of television audio is started based on a user operation that instructs imaging of a television image. However, the present invention is not limited only to such a case. For example, a configuration in which only TV sound input is started based on a user operation may be employed. In this configuration, when the identification information determination unit 71 determines that the identification information is included in the television audio data, the imaging of the television image and the OCR processing are started, and the keyword extraction unit 72 searches the candidate video keyword from the television image data. To extract. On the other hand, when the identification information determination unit 71 determines that the identification information is not included in the TV audio data, the imaging of the TV image and the OCR processing are not started.

  Further, in the present embodiment, a configuration has been described in which an error display that prompts the user to adjust the size and angle of the television image is performed when the size and angle of the television image to be captured are out of the allowable range. However, the present invention is not limited to a configuration that displays an error on the display 3. For example, when the size and angle of the TV image to be captured are out of the allowable range, a sound that prompts the user to adjust the size and angle of the TV image may be output from the speaker of the mobile phone 1B. .

Embodiment 3 FIG.
In the first and second embodiments, the configuration in which the genre selection unit 75 selects a character string that can be specified as a search keyword based on a user operation has been described. On the other hand, in the present embodiment, a configuration will be described in which a character string that can be specified as a search keyword is selected based on the number of times a character string is used in mail data in a user's mail folder.

  In the present embodiment, the genre selection unit 75 counts the number of uses in the mail data in the user's mail folder for the character string held in the unselected genre holding unit 76. Further, a character string that can be specified as a search keyword is selected by comparing the total number of times of use of the character string associated with any genre with a threshold value. For example, if the total number of uses in the mail folder of the character strings “water”, “coffee”, and “tea” associated with the genre “beverage” is 10 times or more, these character strings hold the genre information. It is stored in the section 74 and held as a character string that can be specified as a search keyword. On the other hand, if the total number of times these character strings are used in the mail folder is less than 10, these character strings are not stored in the genre information holding unit 74 and are not designated as search keywords.

  In the present embodiment, a character string associated with a genre that is frequently used in the mail data in the mail folder is selected as a character string that can be transmitted to the search site. For this reason, it is possible to select a character string of high user interest while simplifying a user operation when selecting a transmittable character string.

Embodiment 4 FIG.
In the first to third embodiments, the example in which the display control unit 9 displays the page data of the search result received from the search site 23 has been described. On the other hand, in the present embodiment, a case will be described in which page data is further received from the product information site 24 based on link information included in the page data of the search result and the page data of the product information is displayed.

  FIG. 11 is a block diagram showing an example of the internal configuration of an information terminal device according to Embodiment 4 of the present invention, in which a mobile phone 1C is shown as an example of the information terminal device. The cellular phone 1 </ b> C includes a display 3, a TV broadcast content data acquisition unit 6, a keyword designation unit 7, a cellular communication unit 8, and a display control unit 9. In addition, about the structure same as FIG. 2 (Embodiment 1) and FIG. 10 (Embodiment 2), the same code | symbol is attached | subjected and description is abbreviate | omitted.

  The TV broadcast content data acquisition unit 6 is means for acquiring TV image data and TV audio data as TV broadcast content data, and may be the same as the TV broadcast content data acquisition unit 6A of FIG. The same as the TV broadcast content data acquisition unit 6B. The cellular communication unit 8 includes a keyword transmission unit 81, a page data reception unit 82 ′, and a page data request unit 83.

  The page data request unit 83 specifies page data including product information based on the link information included in the search result of the search server 23, and requests the product information server 24 to browse the page data.

  When the page data receiving unit 82 ′ receives page data from the search site 23, the page data receiving unit 82 ′ extracts access information to the link destination included in the page data, for example, a URL (Uniform Resource Locator), and outputs it to the page data requesting unit 83. . The page data request unit 83 specifies page data including product information based on this access information, and transmits a browsing request to the product information server 24 that provides the page data.

  Further, the page data receiving unit 82 ′ outputs the page data received from the product information server 24 to the display control unit 9 without outputting the page data received from the search server 23 to the display control unit 9. For this reason, the display 3 displays the page data of the website corresponding to the link destination of the link information, not the search result including the link information.

  For example, when two or more websites are shown as search results in the page data received from the search server 23, link destination information of websites having the highest relevance to the search keyword among these websites. (URL) is extracted by the page data request unit 83, and a page data browsing request is transmitted to the website. That is, a browsing request is transmitted to the website with the highest search order. If it is a search result as shown in FIG. 5, the browsing request | requirement of the page data of "XYZ company new product" (www.---.com/new_product) will be transmitted.

  Alternatively, in the page data received from the search server 23, the link destination information (URL) of the website indicated as the advertisement is extracted by the page data request unit 83, and the page data browsing request is sent to the website. Is sent. If the search result is as shown in FIG. 5, a page data browsing request of “XYZ company” (www.---.com) is transmitted. Furthermore, if two or more websites are shown as advertisements, the link destination information (URL) of the website having the highest relevance to the search keyword among these websites is the page data request section. The page data browsing request is transmitted to the website. That is, a browsing request is transmitted to the website displayed as the first advertisement.

  In the first to fourth embodiments, the configuration in which the identification information determination unit 71 determines whether the identification information is included in the television audio data has been described. However, the present invention is not limited to this. For example, the identification information determination unit 71 may determine whether the television image data includes image data such as a search button as the identification information. By adopting such a configuration, it is possible to determine that image data that prompts a keyword search is included in the television image data.

  In the first to fourth embodiments, the configuration in which the keyword extraction unit 72 extracts search keyword candidates from television image data by OCR processing has been described. However, the present invention is not limited only to such a case. For example, the keyword extraction unit 72 may extract a character string included in the television audio data as a search keyword candidate by voice recognition processing.

  In Embodiments 1 to 4, an example in which the keyword extraction unit 72 extracts search keyword candidates when identification information is included in at least one of the television audio data and the television image data has been described. However, the present invention is not limited only to such a case. For example, when the identification information is included in both the TV audio data and the TV image data, the keyword extraction unit 72 extracts the search keyword candidate, and the identification information is included in one of the TV audio data and the TV image data. If not, a configuration may be adopted in which search keyword candidates are not extracted. With such a configuration, erroneous detection of identification information can be suppressed.

  In the first to fourth embodiments, the configuration in which the TV broadcast content data acquisition units 6A and 6B acquire TV image data and TV audio data as TV broadcast content data has been described. However, the present invention is not limited only to such a case. For example, the TV broadcast content data acquisition units 6A and 6B may be configured to acquire only one of television image data or television audio data.

  The portable information terminal according to the present invention incorporates a microprocessor such as a CPU (Central Processing Unit) and a DSP (Digital Signal Processor), that is, a computer that operates based on a program, and is described in the above embodiment. Each function, for example, each functional block shown in FIG. 2 and each step of the flowchart shown in FIG. 7 may be given as a program executed by the microprocessor.

1A, 1B, 1C Mobile phone 20 Broadcast station 21 Cellular base station 22 Network 23 Search server 24 Product information server 3 Display 6, 6A, 6B TV broadcast content data acquisition unit 61 Tuner 62 Decoder 63 Microphone 64 Camera 7 Keyword specification unit 71 Identification Information discriminating unit 72 Keyword extracting unit 73 Keyword determining unit 74 Genre information holding unit 75 Genre selecting unit 76 Unselected genre holding unit 8 Cellular communication unit 81 Keyword transmitting unit 82, 82 'Page data receiving unit 83 Page data requesting unit 9 Display control Unit 100 linked search system 120 TV receiver 121 display 122 speaker

Claims (11)

Content acquisition means for acquiring TV broadcast content data consisting of TV image data or TV audio data;
Identification information discriminating means for discriminating whether or not the TV broadcast content data includes identification information indicating that keyword search on the Internet is promoted;
A keyword extracting means for extracting a search keyword from the television broadcast content data based on the timing at which the identification information is determined;
A keyword transmitting means for transmitting the search keyword to a search site on the Internet;
An information terminal device comprising: search result display means for displaying based on a search result received from the search site.   2. The information terminal device according to claim 1, wherein the identification information determination unit determines whether or not the identification information is included in the television audio data.   3. The information terminal device according to claim 2, wherein the keyword extraction unit extracts the search keyword from the television image data.   The information terminal apparatus according to claim 3, wherein the identification information is a click sound.   4. The information terminal device according to claim 3, wherein the keyword extraction unit extracts a character string displayed in a search frame as the search keyword.   The information terminal apparatus according to claim 1, wherein the search result display unit displays the search result.   The information terminal device according to claim 1, wherein the search result display unit displays a link destination website of link information included in the search result.   8. The content acquisition unit according to claim 1, further comprising: a tuner that receives a television broadcast wave; and a decoder that converts the received television broadcast wave into the television broadcast content data. Information terminal device.   The content acquisition means includes imaging means for capturing a television image displayed on the display of the television receiver, and audio input means for inputting the television sound of the television receiver. The information terminal device according to claim 1.   2. The information terminal device according to claim 1, wherein the identification information determination unit determines whether or not the identification information is included in the television image data. A content acquisition step of acquiring TV broadcast content data consisting of TV image data or TV audio data;
An identification information determination step of determining whether or not identification information indicating that keyword search on the Internet is prompted is included in the television broadcast content data;
A keyword extraction step of extracting a search keyword from the television broadcast content data based on the timing at which the identification information is determined;
A keyword sending step for sending the search keyword to a search site on the Internet;
A program for an information terminal device comprising a procedure for causing a computer to execute a search result display step for displaying a search result received from the search site.

Images