WO2013080502A1

WO2013080502A1 - Terminal apparatus, server apparatus, information processing method, program, and linking application supply system

Info

Publication number: WO2013080502A1
Application number: PCT/JP2012/007506
Authority: WO
Inventors: Yasuaki Yamagishi
Original assignee: Sony Corporation
Priority date: 2011-11-29
Filing date: 2012-11-21
Publication date: 2013-06-06
Also published as: ES2650012T3; RU2632403C2; CN103947175A; US20150215673A1; US10616647B2; JP6219275B2; KR20140107199A; JP2015503249A; KR102058761B1; PL2786547T3; US20130145414A1; US9015785B2; EP2786547B1; BR112014012426A2; RU2014120748A; EP2786547A1; CN103947175B

Abstract

To execute applications in link with the progression of content being viewed at user's home. A terminal apparatus practiced as one embodiment of the present disclosure has an extracting block configured to extract signature data indicative of features of content to be reproduced, a query generating block configured to generate a query that at least includes the above-mentioned extracted signature, a communication block configured to transmit the above-mentioned generated query to a server apparatus and receive a response returned by this server in response to the above-mentioned response, and an application executing block configured to obtain an application on the basis of an application URL (Uniform Resource Locator) in which at least application identification information is written, this URL being included in the above-mentioned received response, and execute the obtained application. The above-mentioned terminal apparatus practiced as one embodiment of the invention is applicable to TV receivers.

Description

TERMINAL APPARATUS, SERVER APPARATUS, INFORMATION PROCESSING METHOD, PROGRAM, AND LINKING APPLICATION SUPPLY SYSTEM

The present disclosure relates to a terminal apparatus, a server apparatus, an information processing method, a program, and a linking application supply system and, more particularly, to a terminal apparatus, a server apparatus, an information processing method, a program, and a linking application supply system that are configured to execute application programs in operative association with the progress of not only television programs but also given content being viewed.

Take for example, the displaying, for TV (television) viewers, of information (cast, outline, preview of next installment, and the like) of a television program (here sometimes referred to simply as a program) being viewed and information (news, weather forecast, traffic information and the like) beneficial to viewers though not directly associated with a program itself onto the screen.

In order to realize the above-mentioned function, commands for obtaining and launching a dedicated application program (hereafter sometimes referred to simply as an application program) may be transmitted beforehand to the TV receiver in link with the progress of the program. Actually, this function has already been realized in Japan and Europe, for example, by transmitting these commands and applications by use of a band for data broadcasting in a TV broadcast signal (refer to Patent Document 1 below for example).

On the other hand, in the U.S.A., the band for data broadcasting is not arranged in the TV broadcast signal and no alternative method has been established, so that the execution of an application in link with the progress of a program has not been realized.

It should be noted that, in the U.S.A., more families view programs via CATV and IPTV than those directly receiving and viewing programs through TV receivers. With CATV and IPTV, only video and audio may be extracted for delivery from TV broadcasting, so that, even if the band for data broadcasting is arranged in the TV broadcast signal in the U.S.A., the transmission of the data broadcast signal containing application-associated data up to viewers cannot be assured.

Japanese Patent Laid-open NO. 2006-50237

Further, recently, in the case where given content, such as recorded programs and programs reproduced from package media based on DVD and BD is viewed, there is a demand for executing applications in link with the progress of the content. However, this demand has not been satisfied.

Therefore, the present disclosure addresses the above-identified and other problems and solves the addressed problems by providing technologies for executing applications in link with the progress of content being viewed at user homes.

In carrying out the invention and according to a first aspect thereof, there is provided a terminal apparatus. This terminal apparatus has an extracting block configured to extract signature data indicative of a feature of content from the content to be reproduced; a query generating block configured to generate a query that at least includes the extracted signature data; a communication block configured to transmit the generated query to a server apparatus and receive a response returned by the server apparatus in response to the query; and an application executing block configured to obtain an application on the basis of an application URL (Uniform Resource Locator) in which at least application identification information is written, the application URL being included in the response, and execute the application.

In the above-mentioned terminal apparatus, execution time information indicative of an execution time of the application is further written to the application URL and the application executing block obtains the application with a timing which it is ready in time for the execution time.

In the abovementioned terminal apparatus, at least one of extraction local time information indicative of a time at which the signature data was extracted by the extracting block, content identification information indicative of the corresponding content, and extraction reproduction time information indicative of an extraction time at a content reproduction timing of the corresponding signature data is further written to the application URL.

In the above-mentioned terminal apparatus, the terminal apparatus is incorporated in a reproducing apparatus configured to reproduce content. The above-mentioned terminal apparatus further has a content supply block configured to supply all pieces of content to be reproduced by the reproducing apparatus to the extracting block.

In carrying out the invention and according to the first aspect of the invention, there is provided an information processing method for a terminal apparatus. The above-mentioned information processing method has the steps of extracting signature data indicative of a feature of content from the content to be reproduced; generating a query that at least includes the extracted signature data; transmitting the generated query to a server apparatus and receiving a response returned by the server apparatus in response to the query; and obtaining an application on the basis of an application URL (Uniform Resource Locator) in which at least application identification information is written, the application URL being included in the response, and executing the application.

In carrying out the invention and according the first aspect thereof, there is provided a program for making a computer function as: an extracting block configured to extract signature data indicative of a feature of content from the content to be reproduced; a query generating block configured to generate a query that at least includes the extracted signature data; a communication block configured to transmit the generated query to a server apparatus and receive a response returned by the server apparatus in response to the query; and an application executing block configured to obtain an application on the basis of an application URL (Uniform Resource Locator) in which at least application identification information is written, the application URL being included in the response, and execute the application.

In the first aspect of the present disclosure, signature data indicative of the feature of content is extracted from the content to be reproduced, a query at least including the extracted signature data is generated, the generated query is transmitted to a server apparatus, a response returned by the server apparatus in response to the query is received, and an application is obtained on the basis of an application URL in which at least application identification information is written, this application URL being included in the received response.

In carrying out the invention and according a second aspect thereof, there is provided a server apparatus. This server apparatus has a database configured to indicate a relation between signature data indicative of a feature of content extracted therefrom, content identification information indicative of the content of extraction source, and application identification information indicative of an application to be executed in link with the content; a response generating block configured to identify content from which signature data included in a query transmitted from a terminal apparatus was extracted by referencing the database and generate a response at least including an application URL (Uniform Resource Locator) in which the application identification information indicative of an application to be executed in link with the identified content is written; a transmitting block configured to transmit the generated response to the terminal apparatus; and an application supply block configured to supply the application to the terminal apparatus in response to a request from the terminal apparatus on the basis of the application URL included in the response.

In the above-mentioned server apparatus, the application supply block regenerates the application and supply the regenerated application to the terminal apparatus in response to a request from the terminal apparatus on the basis of the application URL included in the response.

In the above-mentioned server apparatus, the application supply block adjusts at least one of an execution timing of the application and an event fire timing in the application being executed as regeneration of the application.

In the above-mentioned server apparatus, at least one of extraction local time information indicative of a time at which the corresponding signature data was extracted in the terminal apparatus, content identification information indicative of the corresponding content, and extraction reproduction time information indicative of an extraction timing in a content reproduction timing of the corresponding signature data is further written to the application URL.

In carrying out the invention and according to the second aspect thereof, there is provided an information processing method for a server apparatus. This information processing method has the steps of: identifying content from which signature data included in a query transmitted from a terminal apparatus was extracted by referencing a database configured to indicate a relation between signature data indicative of a feature of content extracted therefrom, content identification information indicative of the content of extraction source, and application identification information indicative of an application to be executed in link with the content and generating a response at least including an application URL (Uniform Resource Locator) in which the application identification information indicative of an application to be executed in link with the identified content is written; transmitting the generated response to the terminal apparatus; and supplying the application to the terminal apparatus in response to a request from the terminal apparatus on the basis of the application URL included in the response.

In carrying out the invention and according to the second aspect thereof, there is provided a program for making a computer function as: a database configured to indicate a relation between signature data indicative of a feature of content extracted therefrom, content identification information indicative of the content of extraction source, and application identification information indicative of an application to be executed in link with the content; a response generating block configured to identify content from which signature data included in a query transmitted from a terminal apparatus was extracted by referencing the database and generate a response at least including an application URL (Uniform Resource Locator) in which the application identification information indicative of an application to be executed in link with the identified content is written; a transmitting block configured to transmit the generated response to the terminal apparatus; and an application supply block configured to supply the application to the terminal apparatus in response to a request from the terminal apparatus on the basis of the application URL included in the response.

In the second aspect of the present disclosure, content from which signature data included in a query returned from a terminal apparatus is identified by a database indicative of a relation between signature data indicative of a feature of content extracted therefrom, content identification information indicative of the content of extraction source, and application identification information indicative of an application to be executed in link with the content, a response at least including an application URL (Uniform Resource Locator) in which the application identification information indicative of the application to be executed in link with the identified content is generated, the generated response is transmitted to the terminal apparatus, and the above-mentioned application is supplied to the above-mentioned terminal apparatus in response to a request from the above-mentioned terminal apparatus on the basis of the application URL included in the above-mentioned response.

In carrying out the invention and according to a third aspect thereof, there is provided a linking application supply system. This linking application supply system has a terminal apparatus and a server apparatus. The terminal apparatus has an extracting block configured to extract signature data indicative of a feature of content from the content to be reproduced; a query generating block configured to generate a query that at least includes the extracted signature data; a communication block configured to transmit the generated query to a server apparatus and receive a response returned by the server apparatus in response to the query; and an application executing block configured to obtain an application on the basis of an application URL (Uniform Resource Locator) in which at least application identification information is written, the application URL being included in the response, and execute the application. The server apparatus has a database configured to indicate a relation between signature data indicative of a feature of content extracted therefrom, content identification information indicative of the content of extraction source, and application identification information indicative of an application to be executed in link with the content; a response generating block configured to identify content from which signature data included in a query transmitted from a terminal apparatus was extracted by referencing the database and generate a response at least including an application URL (Uniform Resource Locator) in which the application identification information indicative of an application to be executed in link with the identified content is written; a transmitting block configured to transmit the generated response to the terminal apparatus; and an application supply block configured to supply the application to the terminal apparatus in response to a request from the terminal apparatus on the basis of the application URL included in the response.

In the third aspect of the present disclosure, signature data indicative of a feature of content is extracted by the terminal apparatus from the content to be reproduced, a query at least including the extracted signature data is generated, the generated query is transmitted to the server apparatus, a response returned by the server apparatus in response to the request is received, and an application is obtained to be executed on the basis of the application URL in which at least application identification information is written, the application URL being included in the received response. At the same time, when the database indicative of a relation between signature data indicative of a feature of content extracted therefrom, content identification information indicative of the content of extraction source, and application identification information indicative of an application to be executed in link with the content is referenced by the server apparatus, the content from which the signature data included in the query transmitted from the terminal apparatus is identified, a response at least including the application URL (Uniform Resource Locator) in which the application identification information indicative of the application to be executed in link with the identified content is generated, the generated response is transmitted to the terminal apparatus, and the application is supplied to the terminal apparatus in response to a request from the terminal apparatus on the basis of the application URL included in the response.

According to the first aspect of the present disclosure, an application can be executed in link with the content being viewed at user's home.

According to the second aspect of the present disclosure, an application can be executed in link with the content being viewed at user's home.

According to the third aspect of the present disclosure, an application can be executed in link with the content being viewed at user's home.

FIG. 1 is a block diagram illustrating an exemplary configuration of a linking application supply system practiced as one embodiment of the invention. FIG. 2 is a block diagram illustrating an exemplary configuration of an ACR (Automatic Content Recognition) client. FIG. 3 is a block diagram illustrating an exemplary configuration of an ACR server. FIG. 4 is a block diagram illustrating an exemplary configuration of an application server. FIG. 5 is a diagram illustrating a data structure of an ACR query. FIG. 6 is a diagram illustrating a data structure of ACR reference data. FIG. 7 is a diagram illustrating a data structure of application schedule data. FIG. 8 is a diagram illustrating a relation between content and signature data and so on. FIGS. 9 are diagrams illustrating a data structure of an ACR response. FIGS. 10 are diagrams illustrating an exemplary description of an application URL. FIG. 11 is a flowchart indicative of a first exemplary operation of the linking application supply system. FIG. 12 is a diagram illustrating the first exemplary operation of the linking application supply system. FIG. 13 is a flowchart indicative of a second exemplary operation of the linking application supply system. FIG. 14 is a diagram illustrating the second exemplary operation of the linking application supply system. FIG. 15 is a block diagram illustrating an exemplary configuration of a computer.

<Mode for Carrying Out the Invention>
This invention will be described in further detail by way of best modes (hereafter referred to embodiments) thereof with reference to the accompanying drawings.

<Exemplary configuration of a linking application supply system>
Now, referring to FIG. 1, there is shown an exemplary configuration of a linking application supply system practiced as one embodiment of the invention.

This linking application supply system 10 is configured to execute an application in link with the progress of content being viewed by a user through TV receiver or the like.

Executing the application allows information (cast, outline, preview of next installment, and the like) associated with the content being viewed and user beneficial information (news, weather forecast, traffic information and the like) not directly associated with content to be displayed on the screen or the realization of viewer participating quiz and voting, for example.

It should be noted that content as used herein is not limited to on-air programs received by the TV receiver via a TV broadcasting network, a CATV network, or an IPTV network. For example, the content as used herein includes a program reproduced from the recording by a video recorder, a video reproduced from a package media by a multimedia player, a video distributed by a personal computer via the Internet, and so on, for example.

The linking application supply system 10 is configured by two or more terminal apparatuses 20-1 through 20-N and a supply apparatus 30. In the following, the terminal apparatuses 20-1 through 20-N are generically referred to simply as the terminal apparatus 20 unless otherwise noted. It is assumed that the terminal apparatus 20 be arranged at a user home for example in a state of being incorporated in a content reproduction apparatus typified by a TV receiver, for example.

<Description of the terminal apparatus 20>
The terminal apparatus 20 is connected to the supply apparatus 30 via a broadcasting network 11 including a TV broadcasting network, a CATV network, and an IPTV network and the Internet 12.

The terminal apparatus 20 has a content renderer 21, an ACR client 22, an application manager 23, and a user interface 24.

The content renderer 21 has a function of obtaining content from the outside and reproducing the obtained content. In addition, the content renderer 21 branches all items of content displayed on a display device such as a TV receiver in which the terminal apparatus 20 is incorporated and supplies the branched pieces of content to the ACR client 22. Namely, the content renderer 21 branches not only a program received by the tuner of the TV receiver in which the terminal apparatus 20 is incorporated but also the content entered via various input terminals (such as ah HDMI terminal and so on) of the TV receiver and supplies the branched pieces of content to the ACR client 22.

The ACR client 22 extracts signature data indicative of a feature of the content concerned in a predetermined sample period from the content entered from the content renderer 21. For a method of signature data extraction, any of existing methods can be applied. For example, a watermark embedded in content in advance may be extracted to provide signature data or a finger print may be computed to provide signature data.

Use of a finger print as signature data provides the same signature data as that before rendering even if content resolution, aspect ratio, bit rate, and coded format have been rendered. Therefore, the accuracy of identification in the case where content identification is executed on the basis of this signature data can be increased.

The ACR client 22 generates an ACR query that includes the extracted signature data and transmits the generated ACR query to an ACR server 31 of the supply apparatus 30 via the Internet 12. In addition, the ACR client 22 receives and holds an ACR response returned by the ACR server 31 in accordance with the transmitted ACR query. Further, the ACR client 22 analyzes the held ACR response to make the application manager 23 execute an application corresponding to the content being viewed, in link with the progress of the content.

Under the control of the ACR client 22, the application manager 23 obtains the application corresponding to the content being viewed from an application server 33 via the Internet 12 and executes the obtained application. In addition, the application manager 23 obtains associated meta data for use by the application being executed from the application server 33.

The user interface 24 displays, on a display device such as a TV receiver, various kinds of information to be displayed by the application executed in link with the progression of the content. In addition, the user interface 24 receives a user operation done on the application being executed.

<Description of the supply apparatus 30>
The supply apparatus 30 is configured by the ACR server 31, a content server 32, and the application server 33.

The ACR server 31 generates beforehand ACR reference data obtained by relating content identifier and application identifier with the signature data extracted in a predetermined sampling period of various items of content to be viewed on the terminal apparatus 20 and holds the generated ACR reference data. In addition, in response to an ACR query transmitted from the ACR client 22 of the terminal apparatus 20, the ACR server 31 generates an ACR response and transmits the generated ACR response to the ACR client 22 via the Internet 12.

The content server 32 distributes content via the broadcasting network 11 and, at the same time, supplies the content to be distributed to the ACR server 31.

The application server 33 supplies applications and associated meta data in response to a request from the application manager 23 of the terminal apparatus 20.

It should be noted that the ACR server 31, the content server 32, and the application server 33 that configure the supply apparatus 30 may be arranged at one location in a concentrated manner or at different places in a dispersed manner. Also, the ACR server 31, the content server 32, and the application server 33 may be configured in appropriate combinations.

<Detail exemplary configuration of the ACR client 22>
Referring to FIG. 2, there is shown a detail exemplary configuration of the ACR client 22 of the terminal apparatus 20.

The ACR client 22 is configured by a signature extracting block 41, an ACR query generating block 42, a communication block 43, a holding block 44, and an ACR response analyzing block 45.

The signature extracting block 41 extracts, by use of a predetermined extraction method, signature data indicative of a feature of content in a predetermined sampling period from the content entered from the content renderer 21 and outputs the extracted signature data to the ACR query generating block 42. It should be noted that signature data extraction timing and sample period may be set by the user as required.

The ACR query generating block 42 generates an ACR query including signature data every time the signature data is entered from the signature extracting block 41. A data structure of an ACR query will be described later with reference to FIG. 5.

The communication block 43 transmits the ACR query generated by the ACR query generating block 42 to the ACR server 31 of the supply apparatus 30 via the Internet 12. In addition, the communication block 43 receives an ACR response returned from the ACR server 31 and outputs the received ACR response to the holding block 44. The holding block 44 holds the ACR response received by the communication block 43.

The ACR response analyzing block 45 analyzes the ACR response held in the holding block 44 and identifies an application to be executed in link with the progress of the content being viewed, thereby notifying the application manager 23 thereof. To be more specific, an application included in the ACR response is notified.

<Detail exemplary configuration of the ACR server 31>
Referring to FIG. 3, there is shown a detail exemplary configuration of the ACR server 31 that configures the supply apparatus 30.

The ACR server 31 is configured by a content acquiring block 51, a signature extracting block 52, an ACR database 53, a communication block 54, and an ACR response generating block 55.

The content acquiring block 51 obtains various kinds of content to be viewed at the terminal apparatus 20 and meta data of these various kinds of content from the content server 32 for example and supplies the obtained content to the signature extracting block 52.

Like the signature extracting block 41 of the ACR client 22, the signature extracting block 52 extracts the signature data indicative of a feature of the content from the content acquiring block 51 in a predetermined sampling period by use of a predetermined method and outputs the extracted signature data to the ACR database 53.

The ACR database 53 generates ACR reference data by relating a content identifier indicative of the content that is the extraction source of each item of signature data entered from the signature extracting block 52, a content reproduction time indicative of a signature data extraction timing, and an application identifier indicative of an application to be executed in a linked manner with above-mentioned each item of signature data entered from the signature extracting block 52 and holds the generated ACR reference data. It should be noted that ACR reference data generated in advance may be supplied to the ACR database 53 and held therein. A data structure of ACR reference data will be described later with reference to FIG. 6.

The communication block 54 receives an ACR query transmitted from the ACR client 22 of the terminal apparatus 20 via the Internet 12 and outputs the received query to the ACR response generating block 55. In addition, the communication block 54 transmits an ACR response generated by the ACR response generating block 55 to the ACR client 22 of the terminal apparatus 20 via the Internet 12.

The ACR response generating block 55 references the ACR reference data of the ACR database 53 to identify the content that was the extraction source of the signature data included in the ACR query transmitted from the ACR client 22 of the terminal apparatus 20 and, at the same time, identifies an application to be executed in link with the progress of the identified content. In addition, the ACR response generating block 55 generates the content identifier as a result of the identification and an ACR response including an application URL for example for the terminal apparatus 20 to obtain the identified application and outputs the generated information to the communication block 54. A data structure of the ACR response will be described later with reference to FIG. 9.

It should be noted that, when the ACR response generating block 55 generates an ACR response, a communication delay of an ACR response in the Internet 12 is estimated on the basis an extraction local time stamp 75 (FIG. 5) included in the ACR query and a processing time of the ACR response generating block 55 itself is considered to select associated meta data to be stored in the ACR response.

<Exemplary detail configuration of the application server 33>
Referring to FIG. 4, there is shown an exemplary detail configuration of the application server 33 configuring the supply apparatus 30.

The application server 33 is configured by a communication block 61, an application regenerating block 62, and an application database 63.

The communication block 61 communicates with the application manager 23 of the terminal apparatus 20 via the Internet 12 to supply an application from the application manager 23 as required. In response to the request (to be more specific, the notification of an application URL, details thereof to be described later) from the application manager 23 received by the communication block 61, the application regenerating block 62 appropriately regenerates the application held in the application database 63 and outputs the regenerated application to the communication block 61.

The application database 63 stores application schedule data including a content identifier indicative of content, a reproduction time indicative of timing of the progression of the content, and an application (program data thereof) to be executed in link with the reproduction time. A data structure of the application schedule data will be described later with reference to FIG. 7.

<Data structure of an ACR query>
Referring to FIG. 5, there is shown a data structure of an ACR query generated in the ACR query generating block 42 of the ACR client 22.

An ACR query 70 includes an ACR query identifier 71, an extraction method identifier 72, a service identifier 73, a signature data 74, an extraction local time stamp 75, an originating address 76, and a signature 77.

The ACR query identifier 71 is information for identifying the ACR query 70 concerned. The extraction method identifier 72 is information for identifying an extraction method used to extract signature data 74. The service identifier 73 is information for selecting a service for processing the ACR query 70 concerned when there are two or more services for identifying content on the basis of the signature data 74.

The signature data 74 is extracted from the content by the signature extracting block 41. The extraction local time stamp 75 is indicative of a timing with which the signature data 74 is extracted from the content by the signature extracting block 41. This time stamp is indicated by a time indicated by a local system clock of the terminal apparatus 20.

The originating address 76 is an address in the Internet 12 of the terminal apparatus 20 that originates the ACR query 70 concerned and is used as information indicative of a return destination of an ACR response generated by the ACR server 31 in response to the ACR query 70 concerned. The signature 77 prevents the tampering of the ACR query 70 concerned along a communication route such as the Internet 12. It should be noted this tampering may also be prevented by encrypting the entire ACR query 70 concerned and transmitting the encrypted ACR query 70 concerned.

<Data structure of ACR reference data>
Referring to FIG. 6, there is shown a data structure of ACR reference data that is held in the ACR database 53 of the ACR server 31.

In an ACR reference data 80, a content identifier 82 a reproduction time 83, and an application identifier 84 are related with a signature data 81.

The signature data 81 is extracted from content by the signature extracting block 52. The content identifier 82 is indicative of content from which the signature data 81 concerned has been extracted. The reproduction time 83 is indicative of the progression timing of content when the signature data 81 concerned has been extracted from the content indicated by the content identifier 82. For example, the reproduction time 83 is indicated by an elapsed time from the head of the content indicated by the content identifier 82.

The application identifier 84 is indicative of an application to be executed in link with the progression timing indicated by the reproduction time 83 of the content indicated by the content identifier 82.

<Data structure of application schedule data>
Referring to FIG. 7, there is shown a data structure of application schedule data held in the application database 63 of the application server 33.

Application schedule data 90 includes a content identifier 91, a reproduction time 92, an application 93, and associated meta data 94.

The content identifier 91 is indicative of a particular piece of content. The reproduction time 92 is indicative of the progression timing of the content indicated by the content identifier 91 and is indicated by an elapsed time from the head of the content for example. The application 93 is an application (program data thereof) to be executed in link with the progression timing indicated by the reproduction time 92 of the content indicated by the content identifier 91. The associated meta data 94 includes various kinds of information (data to be displayed and so on) that are used by the application 93 being executed in the terminal apparatus 20.

Referring to FIG. 8, there is shown a relation between the content, the signature data, the application, and the associated meta data that are related with each other by the ACR reference data 80 and the application schedule data 90. As shown in the figure, the signature data is extracted from the content on a periodic manner and can change applications when the progression of the same content is continuing. In addition, the signature data can give different pieces of associated meta data to the application being executed.

It should be noted that the ACR reference data 80 held in the ACR database 53 and the application schedule data 90 held in the application database 63 may be integrated to be held at least in one of the ACR server 31 and the application server 33.

<Data structure of ACR response>
Referring to FIG. 9, there are shown data structures of an ACR response generated by the ACR response generating block 55 of the ACR server 31.

It should be noted that FIG. 9A shows the data structure of an ACR response when the content from which the signature data 74 of the ACR query 70 transmitted from the ACR client 22 has been identified, namely, the content being viewed on the terminal apparatus 20 has been identified. FIG. 9B shows the data structure of an ACR response when the content has not been identified.

An ACR response 100 in the case where the content being viewed on the terminal apparatus 20 shown in FIG. 9A includes an ACR query identifier 101, an extraction method identifier 102, a service identifier 103, an application URL 104, an originating address 105, and a signature 106.

An ACR response 100 in the case where the content being viewed on the terminal apparatus 20 shown in FIG. 9B has not been identified includes the ACR query identifier 101, a service identifier 103, an originating address 105, the signature 106, an identification fail flag 111.

The ACR query identifier 101, the extraction method identifier 102, and the service identifier 103 are used to identify the ACR query 70 that was a trigger for the generation of the ACR response 100 concerned. For these identifiers, the ACR query identifier 71, the extraction method identifier 72, and the service identifier 73 of the corresponding ACR query 70 can be used.

The application URL (Uniform Resource Locator) 104 is indicative of an application to be executed in link with the progression of the identified content and also indicative of the acquisition destination (the application server 33) of this application. The application URL 104 will be described later with reference to FIG. 10.

The application URL 105 is an address on the Internet 12 of the ACR server 31 that transmits the ACR response 100 concerned. The signature 106 is used to prevent the ACR response 100 concerned from being tampered on the communication route. It should be noted that the entire ACR response 100 may be encrypted to be transmitted, thereby preventing tampering.

The identification fail flag 111 is indicative that the content from which the signature data 74 included in the ACR query 70 has been extracted in the ACR server 31, namely, the content being viewed on the terminal apparatus 20 has not been identified.

<Example of the application URL 104>
FIG. 10 shows two types of description examples of the application URL 104 included in the ACR response 100.

As shown in the figure, the application URL 104 is written in the same manner as a URL indicative of the location of data or the like on the Internet 12. The first half of the application URL 104 "http://xxxx.com/applocation" is equivalent to the acquisition source and application identifier of each application. The last half of the application URL 104 describes a query character string.

In the description example of the application URL 104 shown in FIG. 10A, a query character string is made up of clientLocalTime, contentID, contentTime, and startTime.

On the other hand, in the description example of the application URL 104 shown in FIG. 10B, a query character string is made up of clientLocalTime, contentID, and contenTime.

clientLocalTime is indicative of a timing with which a corresponding signature data has been extracted from the content in a local system clock of the terminal apparatus 20, for which the extraction local time stamp 75 of the ACR query 70 is used. contentID is indicative of the content from which signature data has been extracted, for which the content identifier 82 of the ACR reference data 80 is used.

contentTime is indicative of a timing with which corresponding signature data has been extracted from the content, in a content reproduction time, for which the reproduction time 83 of the ACR reference data 80 is used. startTime describes the execution timing of the application indicated by the first half of the application URL 104 in content reproduction time.

It should be noted that, for the application URL 104, one of the description examples shown in FIG. 10A and FIG. 10B is employed. The following describes a difference in the operation of the linking application supply system 10 between the case where the description example of FIG. 10A is used and the case where the description example of FIG. 10B is used.

<Operation by the linking application supply system 10>
Referring to FIG. 11, there is shown a flowchart indicative of an operation (hereafter referred to as a first operation) of the linking application supply system 10 that is executed when the description example shown in FIG. 10A is used for the application URL 104.

It is assumed, for the premises of the first operation, that the already generated ACR reference data 80 be held in the ACR database 53 of the ACR server 31 configuring the supply apparatus 30 and the application schedule data 90 be held in the application database 63 of the application server 33.

Content is reproduced by a TV receiver in which the terminal apparatus 20 is incorporated. To be more specific, when the user views content on the TV receiver, the content renderer 21 of the terminal apparatus 20 branches the content being viewed and outputs the branched content to the ACR client 22 in step S1. In the ACR client 22, the signature extracting block 41 extracts signature data from the content being viewed with a predetermined sampling period and outputs the extracted content to the ACR query generating block 42.

In step S2, the ACR query generating block 42 generates an ACR query 70 including the signature data entered from the signature extracting block 41 and outputs the generated ACR query 70 to the communication block 43. The communication block 43 transmits the received generated ACR query 70 to the ACR server 31 of the supply apparatus 30 via the Internet 12.

On the other hand, in step S11, the ACR server 31 of the supply apparatus 30 waits until the ACR query 70 transmitted from the ACR client 22 of the terminal apparatus 20 is received by the communication block 54. When the ACR query 70 is received by the communication block 54, the procedure goes to step S12.

In step S12, the ACR response generating block 55 references the ACR reference data 80 of the ACR database 53 to identify content corresponding to the signature data 74 included in the ACR query 70.

In step S13, the ACR response generating block 55 generates an ACR response 100 in accordance with a content identification result. To be more specific, if the content corresponding to the signature data 74 included in the ACR query 70 has been identified, the ACR response 100 indicated in FIG. 9A is generated; otherwise, the ACR response 100 indicated in FIG. 9B is generated. The communication block 54 transmits the generated ACR response 100 to the ACR client 22 of the terminal apparatus 20 via the Internet 12.

When this ACR response 100 is received and held by the ACR client 22, then, in step S3, the ACR response analyzing block 45 analyzes the application URL 104 included in the held ACR response 100 to make the application manager 23 obtain the application to be ready in time for the execution start timing (startTime in the query character string of the application URL 104) of the application to be executed in link with the progression of the content being viewed.

To be more specific, as shown in FIG. 12, the reception time of the ACR response 100 in the client local time of the terminal apparatus 20 is obtained and clientLocalTime in the query character string of the application URL 104 is subtracted from the obtained reception time. A difference obtained by this subtraction is equivalent to a time from the extraction of signature data in the ACR client 22 to the reception of the ACR response 100, so that adding this difference to contenTime in the query character string allows the reception timing of the ACR response 100 on the content reproduction time axis to be recognized as a reference point. Then, because an interval between this reference point and the starTime in the query character string provides a temporal margin, the application manager 23 obtains the application by use of this temporal margin.

By transmitting the application URL 104 to the Internet 12, the application manager 23 obtains a corresponding application. To be more specific, in step S14, the application regenerating block 62 of the application server 33 reads an application corresponding to the application identifier indicated in the first half of the application URL 104 from the application database 63 and outputs the obtained application to the communication block 61. The communication block 61 supplies this application to the application manager 23. Namely, in this case, the regeneration of application is not executed by the application regenerating block 62.

It should be noted that the execution of the same operation as above by another terminal apparatus 20 in advance may have cached the corresponding application in a proxy server on the Internet 12. If this happens, because the application URL 104 is written by URL, the application already cached in the proxy server is supplied to the application manager 23. In this case, no access is executed to the application server 33, so that the processing load on the side of the supply apparatus 30 can be mitigated.

In step S4, the application manager 23 executes the obtained application when the progression of the content reaches startTime in the query character string.

The processing of each of the steps described above is repetitively executed while the content is being viewed. Consequently, the operation of the application in link with the progression of the content being viewed is enabled.

Then, depending on the type of an application to be linked, information associated with the content can be displayed, viewer participating quiz or voting can be realized, and other pieces of content can be recommended or downloaded.

Referring to FIG. 13, there is shown a flowchart indicative of an operation (hereafter referred to as a second operation) of the linking application supply system 10 that is executed when the description example shown in FIG. 10B is employed for the application URL 104.

It should be noted that the premises for the second operation concerned are the same as those for the first operation described above. The processing of steps S21 and S22 by the terminal apparatus 20 in the second operation is the same as the processing of steps S1 and S2 by the terminal apparatus 20 in the first operation. The operation of steps S31 through S33 by the supply apparatus 30 in the second operation is the same as the processing of steps S11 through S13 by the supply apparatus 30 in the first operation, so that the descriptions of these processing operations are skipped. Therefore, the second operation will be described starting with the processing of step S23.

When the ACR server 31 returns an ACR response 100, the ACR client 22 receives the ACR response 100 and holds the received ACR response 100 in the ACR database 53 in step S23. The ACR response analyzing block 45 analyzes the application URL 104 included in the held ACR response 100 and outputs the application URL 104 included therein to the application manager 23. The application manager 23 transmits the received application URL 104 to the application server 33 via the Internet 12.

On the other hand, in the application server 33, application regenerating block 62 reads the application corresponding to the application identifier indicated by the first half of the application URL 104 from the application database 63 in step S34. At the same time, the application regenerating block 62 reads, from the query character string of the application URL 104, contentTime, indicated by content reproduction time, indicative of the timing with which the signature data has been extracted from the content. Further, the application regenerating block 62 outputs, to the communication block 61, the application read from the application database 63 by executing application regeneration such as adjusting the execution timing of the application and the event fire timing in the executed application later than the time indicated in contentTime, for example. The communication block 61 supplies the regenerated application to the application manager 23.

In step S24, the application manager 23 receives the regenerated application and immediately executes the received application.

To be more specific, as shown in FIG. 14, the execution time of the regenerated application at the client local time of the terminal apparatus 20 is obtained and clientLocalTime in the query character string of the application URL 104 is subtracted from the obtained execution time. A difference obtained as a result of this subtraction is equivalent to the time from the extraction of the signature data in the ACR client 22 to the acquisition and execution of the corresponding application, so that a reference point is determined by adding this difference to contentTime in the query character string. Then, in the application being executed, an event is fired with a timing with the determine reference point set as reference.

Depending on the type of an application to be linked, information associated with the content can be displayed, viewer participating quiz or voting can be realized, and other pieces of content can be recommended or downloaded.

The terminal apparatus 20 and the supply apparatus 30 that execute the above-mentioned sequence of processing operations can be realized by executing software by a computer in addition to hardware configurations. When the above-mentioned sequence of processing operations is executed by software, the programs constituting the software are installed in a computer which is built in dedicated hardware equipment or installed, from a network or recording media, into a general-purpose personal computer for example in which various programs may be installed for the execution of various functions.

Referring to FIG. 15, there is shown a block diagram illustrating an exemplary hardware configuration of the above-mentioned computer.

In a computer 200, a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, and a RAM (Random Access Memory) 203 are interconnected by a bus 204.

The bus 204 is connected to an input/output interface 205. The input/output interface 205 is connected to an input block 206, an output block 207, a recording block 208, a communication block 209, and a drive 210.

The input block 206 is made up of a keyboard, a mouse, and a microphone, for example. The output block 207 is made up of a display and a speaker, for example. The recording block 208 is made up of a hard disk drive or a nonvolatile memory, for example. The communication block 209 is made up of a network interface for example. The drive 210 drives a removable media 211, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

In the computer 200 configured as described above, the CPU 201 loads a program from the recording block 208 into the RAM 203 through the input/output interface 205 and the bus 204 and executes the loaded program, thereby executing the sequence of processing operations described above.

Programs to be executed by the computer 200 (or the CPU 201) can be provided as recorded to the removable media 211 that is package media for example. At the same time, programs can be provided through wireless or wired transmission media, such as a local area network, the Internet, or digital satellite broadcasting.

In the computer 200, a program can be installed into the recording block 208 via the input/output interface 205 by loading the removable media 211 in which the program is stored onto the drive 210. Also, a program can be received at the communication block 209 via wireless or wired transmission media and the received program can be installed in the recording block 208. Besides, a program can be installed in the ROM 202 or the recording block 208 in advance.

It should be noted that a program to be executed by the computer 200 can be executed in a time-dependent manner along a sequence described herein or in parallel or on an on-demand basis.

While preferred embodiments of the present disclosure have been described using specific terms, such description is for illustrative purpose only, and it is to be understood that changes and variations may be made without departing from the spirit or scope of the following claims.

10: Linking application supply system, 20: Terminal apparatus, 21: Content renderer, 22: ACR client, 23: Application manager, 24: User interface, 30: Supply apparatus, 31: ACR server, 32: Content server, 33: Application server, 41: Signature extracting block, 42: ACR query generating block, 43: Communication block, 44: Holding block, 45: ACR response analyzing block, 51: Content acquiring block, 52: Signature extracting block, 53: ACR database, 54: Communication block, 55: ACR response generating block, 61: Communication block, 62: Application regenerating block, 63: Application database, 70: ACR query, 80: ACR reference data, 90: Application schedule data, 100: ACR response, 200: Computer, 201: CPU

Claims

A terminal apparatus comprising:
an extracting block configured to extract signature data indicative of a feature of content from said content to be reproduced;
a query generating block configured to generate a query that at least includes said extracted signature data;
a communication block configured to transmit said generated query to a server apparatus and receive a response returned by said server apparatus in response to said query; and
an application executing block configured to obtain an application on the basis of an application URL (Uniform Resource Locator) in which at least application identification information is written, said application URL being included in said response, and execute said application.
The terminal apparatus according to claim 1, wherein
execution time information indicative of an execution time of said application is further written to said application URL and
said application executing block obtains said application with a timing which it is ready in time for said execution time.
The terminal apparatus according to claim 2, wherein
at least one of extraction local time information indicative of a time at which said signature data was extracted by said extracting block, content identification information indicative of said corresponding content, and extraction reproduction time information indicative of an extraction time at a content reproduction timing of said corresponding signature data is further written to said application URL.
The terminal apparatus according to claim 2, wherein
said terminal apparatus is incorporated in a reproducing apparatus configured to reproduce content,
said terminal apparatus further comprising:
a content supply block configured to supply all pieces of content to be reproduced by said reproducing apparatus to said extracting block.
An information processing method for a terminal apparatus, comprising the steps of:
extracting signature data indicative of a feature of content from said content to be reproduced;
generating a query that at least includes said extracted signature data;
transmitting said generated query to a server apparatus and receiving a response returned by said server apparatus in response to said query; and
obtaining an application on the basis of an application URL (Uniform Resource Locator) in which at least application identification information is written, said application URL being included in said response, and executing said application.
A program for making a computer function as:
an extracting block configured to extract signature data indicative of a feature of content from said content to be reproduced;
a query generating block configured to generate a query that at least includes said extracted signature data;
a communication block configured to transmit said generated query to a server apparatus and receive a response returned by said server apparatus in response to said query; and
an application executing block configured to obtain an application on the basis of an application URL (Uniform Resource Locator) in which at least application identification information is written, said application URL being included in said response, and execute said application.
A server apparatus comprising:
a database configured to indicate a relation between signature data indicative of a feature of content extracted therefrom, content identification information indicative of said content of extraction source, and application identification information indicative of an application to be executed in link with said content;
a response generating block configured to identify content from which signature data included in a query transmitted from a terminal apparatus was extracted by referencing said database and generate a response at least including an application URL (Uniform Resource Locator) in which said application identification information indicative of an application to be executed in link with said identified content is written;
a transmitting block configured to transmit said generated response to said terminal apparatus; and
an application supply block configured to supply said application to said terminal apparatus in response to a request from said terminal apparatus on the basis of said application URL included in said response.
The server apparatus according to claim 7, wherein
said application supply block regenerates said application and supply the regenerated application to said terminal apparatus in response to a request from said terminal apparatus on the basis of said application URL included in said response.
The server apparatus according to claim 8, wherein
said application supply block adjusts at least one of an execution timing of said application and an event fire timing in said application being executed as regeneration of said application.
The server apparatus according to claim 8, wherein
at least one of extraction local time information indicative of a time at which said corresponding signature data was extracted in said terminal apparatus, content identification information indicative of said corresponding content, and extraction reproduction time information indicative of an extraction timing in a content reproduction timing of said corresponding signature data is further written to said application URL.
An information processing method for a server apparatus, comprising the steps of:
identifying content from which signature data included in a query transmitted from a terminal apparatus was extracted by referencing a database configured to indicate a relation between signature data indicative of a feature of content extracted therefrom, content identification information indicative of said content of extraction source, and application identification information indicative of an application to be executed in link with said content and generating a response at least including an application URL (Uniform Resource Locator) in which said application identification information indicative of an application to be executed in link with said identified content is written;
transmitting said generated response to said terminal apparatus; and
supplying said application to said terminal apparatus in response to a request from said terminal apparatus on the basis of said application URL included in said response.
A program for making a computer function as:
a database configured to indicate a relation between signature data indicative of a feature of content extracted therefrom, content identification information indicative of said content of extraction source, and application identification information indicative of an application to be executed in link with said content;
a response generating block configured to identify content from which signature data included in a query transmitted from a terminal apparatus was extracted by referencing said database and generate a response at least including an application URL (Uniform Resource Locator) in which said application identification information indicative of an application to be executed in link with said identified content is written;
a transmitting block configured to transmit said generated response to said terminal apparatus; and
an application supply block configured to supply said application to said terminal apparatus in response to a request from said terminal apparatus on the basis of said application URL included in said response.
A linking application supply system comprising:
a terminal apparatus; and
a server apparatus,
said terminal apparatus having
an extracting block configured to extract signature data indicative of a feature of content from said content to be reproduced;
a query generating block configured to generate a query that at least includes said extracted signature data;
a communication block configured to transmit said generated query to a server apparatus and receive a response returned by said server apparatus in response to said query; and
an application executing block configured to obtain an application on the basis of an application URL (Uniform Resource Locator) in which at least application identification information is written, said application URL being included in said response, and execute said application,
said server apparatus having
a database configured to indicate a relation between signature data indicative of a feature of content extracted therefrom, content identification information indicative of said content of extraction source, and application identification information indicative of an application to be executed in link with said content;
a response generating block configured to identify content from which signature data included in the query transmitted from the terminal apparatus was extracted by referencing said database and generate the response at least including an application URL (Uniform Resource Locator) in which said application identification information indicative of an application to be executed in link with said identified content is written;
a transmitting block configured to transmit said generated response to said terminal apparatus; and
an application supply block configured to supply said application to said terminal apparatus in response to a request from said terminal apparatus on the basis of said application URL included in said response.