JP2012244363A - Receiver and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible for a server to easily and adequately obtain the viewing state of a prescribed scene of content broadcast by a broadcast station in a system facilitating coordination between broadcast and communication.SOLUTION: A receiving system comprises a receiver 4 and a home server 40. The receiver 4 includes communication unit 14 which sends the home server 40 time information indicating relative reproduction start timing of a prescribed scene in content and a program ID identifying the content included in a broadcast signal. The home server 40 includes: an association unit 55 for generating association information associating the time information and the program ID transmitted by the receiver 4 with the viewing state of a viewer when the viewer watched the prescribed scene; and a communication unit 44 for transmitting the association information generated by the association unit 55 to the service server 3.

Description

  The present invention relates to a receiving system and a program having a function of coordinating broadcasting and communication.

  In recent years, with the progress of digitalization of broadcasting and broadbandization of communication, research and development for realizing a broadcasting / communication cooperation service has been performed.

  In this broadcasting / communication cooperation service, it is assumed that a plurality of contents are distributed using different transmission paths of broadcasting and communication, and the plurality of distributed contents are integrated and presented in a receiver such as a digital television. Is done.

  By the way, at present, in data broadcasting, various languages used for the broadcasting service called BML (Broadcast Markup Language) developed for multimedia encoding based on XML (extensible Markup Language) are used. Content is provided (see Patent Document 1).

JP 2003-60931 A

  By the way, in data broadcasting, since it is necessary to multiplex all information on a broadcast wave, there is a certain limitation on the data capacity that can be transmitted on the broadcast wave.

  Therefore, it is desired to realize a system that links broadcasting with communication with no limit on the data capacity that can be transmitted.

In particular, in such a system that links broadcasting and communication, a server that provides various services to the receiver by communication is constructed.
Therefore, for example, the viewer's viewing state of a predetermined scene in the content broadcast by the broadcast station, for example, the viewer's facial expression and sound when viewing the predetermined scene, is acquired from each of a number of receivers. Thus, it is assumed that servers will be used for services.

  In this case, if a conventional receiver such as a data broadcast is a target, the server sets the predetermined scene as the viewing state of a large number of viewers when viewing the predetermined scene of the predetermined content. What is necessary is just to acquire the viewing-and-listening state in the time (absolute time, such as 10:10) broadcast from the broadcasting station from each of many receivers.

  However, in a system that coordinates broadcasting and communication, even if the server acquires viewing states at the same time (absolute time) from each of a number of receivers, these viewing states are for the same scene. Not necessarily. This is because the receiver may synchronize the broadcast and communication internally, and the reproduction of content by the broadcast may be delayed. That is, the same scene is not always reproduced at the same time among a plurality of receivers.

  For this reason, there is a demand for a system that enables a server to easily and appropriately acquire a viewing state of a predetermined scene of content broadcast by a broadcast station in a system that links broadcasting and communication.

  The present invention has been made in view of such a situation, and in a system for coordinating broadcasting and communication, the server can easily and appropriately acquire the viewing state of a predetermined scene of content broadcast by a broadcasting station. One purpose is to do so.

  A receiving system according to the present invention is connected to a service server via a network and receives a broadcast signal including content broadcast by a broadcasting station, and is connected to the receiver and the service server via a network. And a home server, wherein the receiver receives time information indicating a relative reproduction start timing of a predetermined scene in the content and a program ID for specifying the content included in the broadcast signal. Receiver communication means for transmitting to a home server, wherein the home server includes the time information and the program ID transmitted from the receiver, and the viewing state of the viewer when viewing the predetermined scene. An associating unit that generates associating association information; and the association information generated by the associating unit And the home server communication means for transmitting the Bisusaba and configured to have.

According to this configuration, the receiver includes time information indicating the relative playback start timing of the predetermined scene in the content included in the broadcast signal broadcast by the broadcast station, and the viewing state of the viewer when viewing the predetermined scene. And the program ID for specifying the content included in the broadcast signal are transmitted to the home server by the receiver communication means.
The home server generates association information in which the time information and the program ID transmitted from the receiver are associated with the viewing state of the viewer when viewing the predetermined scene by the association means, and the generated association information is the home information. It is transmitted to the service server by the server communication means.

Thereby, in the home server, the predetermined scene of the content specified by the program ID reproduced by the receiver is associated with the viewing state of the viewer who has viewed the predetermined scene, and is transmitted to the service server.
Accordingly, in a system that coordinates broadcasting and communication, the server can easily and appropriately acquire the viewing state of a predetermined scene of content broadcast by a broadcasting station.

In this case, it is preferable that the home server includes an external device connection unit to which a viewing state acquisition unit that acquires a viewing state of the viewer when viewing the predetermined scene is connected.
According to such a configuration, even in a home server that does not include a viewing state acquisition unit, viewing state information can be acquired by connecting the viewing state acquisition unit as an external device to the external device connection unit.
As a result, in a system for linking broadcasting and communication, even if the home server does not include viewing state acquisition means, the server can easily and appropriately acquire the viewing state of the predetermined scene of the content broadcast by the broadcast station. Become.
In addition, since the viewing state acquisition means can be excluded from the essential items of the home server that can receive the service, the spread of the home server that can receive the service can be promoted.

In this case, the home server is connected to a mobile terminal via a network, and the external device connection means is provided in the mobile terminal or connected to the mobile terminal via the home server communication means. It is preferable that an acquisition means is connected.
According to such a configuration, even if the home server does not include the viewing state acquisition unit, the viewing state information can be acquired by connecting the portable terminal that can acquire the viewing state of the viewer to the external device connection unit.
As a result, in a system for linking broadcasting and communication, even if the home server does not include viewing state acquisition means, the server can easily and appropriately acquire the viewing state of the predetermined scene of the content broadcast by the broadcast station. Become.

  In this case, it is preferable that the home server includes viewing state acquisition means for acquiring the viewing state of the viewer when viewing the predetermined scene.

  According to such a configuration, since the home server includes the viewing state acquisition unit, in a system for cooperation between broadcasting and communication, the server does not separately prepare the viewing state acquisition unit, and the server can transmit the content broadcasted by the broadcasting station. The viewing state for the predetermined scene can be acquired easily and appropriately.

  In this case, the viewing state acquisition unit is a camera device that acquires an image, and the association unit is a viewer who views the predetermined scene as the viewing state information via the external device connection unit. Acquired by the image information acquisition means for acquiring the image taken from the camera device, the relative time information acquisition means for acquiring the time information, the ID acquisition means for acquiring the program ID, and the image information acquisition means It is preferable that the image association unit generates image association information in which the image, the time information acquired by the relative time information acquisition unit, and the program ID acquired by the ID acquisition unit are associated with each other.

  According to this configuration, the viewing state acquisition unit is a camera device that acquires an image, and the image information acquisition unit of the association unit views the predetermined scene as viewing state information via the external device connection unit. An image of a person is acquired from the camera device. The image association unit generates image association information in which the image acquired by the image information acquisition unit, the time information acquired by the relative time information acquisition unit, and the program ID acquired by the ID acquisition unit are associated with each other.

  As a result, since the viewer's image can be used as the viewing state information, the server can easily view the non-arbitrary viewing state of the predetermined scene of the content broadcast by the broadcast station in a system that links broadcasting and communication. And it becomes possible to acquire appropriately.

  Further, in this case, the viewing state acquisition unit is a microphone device that acquires audio information, and the association unit views the predetermined scene as the viewing state information via the external device connection unit. Acquired by the audio information acquisition means for acquiring the audio information emitted from the microphone device, the relative time information acquisition means for acquiring the time information, the ID acquisition means for acquiring the program ID, and the audio information acquisition means Voice association means for generating voice association information in which the voice information, the time information acquired by the relative time information acquisition means, and the program ID acquired by the ID acquisition means are associated with each other. preferable.

  According to such a configuration, the viewing state acquisition unit is a microphone device that acquires audio information, and the audio information acquisition unit of the association unit allows the viewer to view a predetermined scene as the viewing state information via the external device connection unit. The voice information that was emitted at the time of the acquisition is acquired from the microphone device. The audio association unit generates audio association information in which the audio information acquired by the audio information acquisition unit, the time information acquired by the relative time information acquisition unit, and the program ID acquired by the ID acquisition unit are associated with each other.

  Accordingly, since the viewer's voice can be used as the viewing state information, in the system for coordinating broadcasting and communication, the server can easily view an arbitrary viewing state of the predetermined scene of the content broadcast by the broadcast station. And it becomes possible to acquire appropriately.

A program according to the present invention controls a receiver connected to a service server via a network and receiving a broadcast signal including content broadcast by a broadcasting station, and a home server connected to the service server via the network. When the computer views time information indicating the relative playback start timing of a predetermined scene in the content transmitted from the receiver, a program ID that identifies the content included in the broadcast signal, and when viewing the predetermined scene It is made to function as an association unit that generates association information that associates the viewing state of the viewer, and a communication unit that executes control to transmit the association information generated by the association unit to the service server.
According to this configuration, the same operational effects as the receiver according to the present invention can be obtained.

  According to the present invention, in a system for coordinating broadcasting and communication, a server can easily and appropriately acquire a viewing state of a predetermined scene of content broadcast by a broadcasting station.

1 is an overall configuration diagram of a broadcasting / communication cooperation system according to an embodiment of the present invention. It is a figure which shows the stack | stuck of the transmission protocol of the broadcast signal which concerns on digital broadcasting. It is a figure which shows the outline | summary of a broadcast communication cooperation system. It is a block diagram which shows the function structure of a receiver. It is a block diagram which shows the function structure of a home server. It is a figure which shows an example of AIT described in the XML format. It is a figure explaining operation | movement regarding discovery of a device, Comprising: It is a figure which shows the relationship of the layer of Hybridcast function. It is a figure which shows the concept of a device connection part. FIG. 10 is a sequence diagram when a predetermined external device is connected as a device when a service of a Hybridcast application that uses the device for operation recognition or the like is started. FIG. 10 is a sequence diagram of a description and a control portion by a device instance when the power state of the device is an OFF state. FIG. 10 is a sequence diagram when a built-in device of a receiver is used as a device when a service of a Hybridcast application that uses the device for operation recognition or the like is started. It is a block diagram which shows an example of a function structure of an correlation part. It is a flowchart explaining the flow of a process of the correlation part of FIG. FIG. 13 is an example of a functional configuration of an associating unit, and is a block diagram illustrating an example different from FIG. 12. It is a flowchart explaining the flow of a process of the correlation part of FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is an overall configuration diagram of a broadcasting / communication cooperation system 100 according to an embodiment of the present invention. The broadcasting / communication cooperation system 100 includes a broadcasting station 1, a broadcasting antenna 2, a service server 3, a receiver 4, and a home server 40. In the broadcasting / communication cooperation system 100, a receiver 4 uses a broadcasting service provided from a broadcasting station 1 by an ISDB (Integrated Services Digital Broadcasting) system and a communication network N configured by the Internet or the like. A communication service provided from the service server 3 is linked. In the broadcasting / communication cooperation system 100, a communication service is provided from the service server 3 to the home server 40 that cooperates with the receiver 4 via the communication network N. Thereby, the broadcast communication cooperation service is provided to the user of the receiver 4 and the home server 40.
In the present embodiment, the broadcasting / communication cooperation system 100 is a Hybridcast (registered trademark) system. Hybridcast is a mechanism for constructing a system that takes advantage of broadcast characteristics such as broadcastability, high quality, and high reliability and communication characteristics that can meet the individual demands of viewers. The Hybridcast system can realize a hybrid service of broadcasting and communication from the viewpoint of strengthening the service by utilizing communication while placing the broadcasting service at the center.

The broadcasting station 1 includes a general digital broadcasting broadcasting facility (not shown) configured to include a program organization facility, a program transmission facility, and a transmission facility.
The broadcasting station 1 produces contents, event information (Event Information Table: EIT), AIT (hereinafter also referred to as application management information as appropriate), and the like, using broadcasting equipment. Then, the broadcast station 1 generates a broadcast signal in which these contents, event information, AIT, and the like are multiplexed by a broadcast facility, modulates the broadcast signal into a broadcast wave, and transmits the broadcast signal from the broadcast antenna 2.

  Content included in such broadcast waves is generated asynchronously with program content in addition to content including video and audio data constituting a program broadcast according to a broadcast schedule (hereinafter referred to as program content). There is content (hereinafter referred to as emergency content) including content, for example, earthquake early warning.

The event information includes meta information about the content such as the name of the program content, the broadcast date and time of the program content, and the description of the program content. Hereinafter, the event information in the table format is referred to as EIT.
The AIT (application management information) is information for managing one or more applications that can be executed by the receiver 4. The AIT includes individual management information corresponding to each of the one or more applications and managing each of the one or more applications. The individual management information includes an application ID for identifying the application to be managed, life cycle control information for controlling the life cycle of the application, location information indicating the location of the application (see FIG. 6 described later), and the like. .

  The content provided by the application includes content linked to the program content and content not linked to the program content. Hereinafter, content provided by an application is referred to as application content.

  The broadcast signal is the same as the broadcast signal of the conventional digital broadcast, and is defined by ARIB (registered trademark) (Association of Radio Industries and Broadcast) standard.

  FIG. 2 is a diagram illustrating a stack of broadcast signal transmission protocols related to digital broadcasting. As shown in FIG. 2, various types of data such as video and audio provided by digital broadcasting are stored in TS packets (transport stream packets) defined by the international standard MPEG-2 Systems and multiplexed in a time division manner. Is transmitted.

  In the TS packet, as shown in FIG. 2, PSI (Program Specific Information) / SI (Service Information) is defined for a section (Section). PSI / SI includes information indicating the type of data stored in the TS packet and information indicating the type of content. The above-mentioned EIT is included in SI.

  Data transmission using TS packets is classified into a method of transmitting data using sections and a method of transmitting data using PES (Packetized Elementary Stream) packets (data stream transmission method).

  Methods for transmitting data using sections include a data carousel transmission method and an event message transmission method.

  The data carousel transmission system is a transmission system that repeatedly transmits one or more data at a constant period. Each piece of data (module) transmitted by the data carousel transmission method is attached with identification information for uniquely identifying them. The data carousel transmission method is used for causing the receiver 4 to acquire individual data at an arbitrary timing.

  The event message transmission method is a method for sending a trigger signal from the broadcast station 1 to the receiver 4. The event message transmission method is used when a message with a small amount of data is transmitted from the broadcasting station 1 to the receiver 4.

  The data stream transmission method is a transmission method in which data to be transmitted is accommodated in a PES packet and transmitted as a stream. The data stream transmission method is used for transmitting real-time data such as video, audio, and caption data, and data that needs to be synchronized with other streams.

Here, AIT can be transmitted by various methods using TS packets.
That is, the AIT can be transmitted by describing the EIT included in the SI of the TS packet.

  The AIT can be transmitted by a data carousel transmission system using sections. When the AIT is transmitted by the data carousel transmission method, identification information is attached to the module so that the receiver 4 can recognize the AIT.

  The AIT can be transmitted as PES multiplexed with content video and audio.

The AIT can be described in binary representation or text representation by XML (Extensible Markup Language) and stored in the TS packet.
In this embodiment, the AIT transmission method is defined in advance in at least one of the above transmission methods.

  BML developed for multimedia coding based on XML is based on xHTML defined by W3C (Registered Trademark) (World Wide Web Consortium), and Java Script (Registered Trademark) is used as a procedural language. This is a standard defined in consideration of consistency with international standards using ECMAScript based on it.

  Although not shown, the service server 3 includes a content distribution server that distributes broadcasted program content and application content, and an application distribution server that distributes various applications that operate on the receiver 4 and the home server 40. Prepare.

The service server 3 is communicably connected to the equipment of the broadcast station 1 and receives program content and metadata of the program content from the broadcast station 1.
The service server 3 transmits the program content to the receiver 4 and the home server 40 in response to receiving the program content acquisition request from the receiver 4 and the home server 40.

  Further, the service server 3 transmits an application to the receiver 4 and the home server 40 in response to receiving an application acquisition request from the receiver 4 and the home server 40. Furthermore, the service server 3 transmits application content including data such as video and audio to the receiver 4 and the home server 40 when an application is executed in the receiver 4 and the home server 40.

  Further, the service server 3 multiplexes the AIT with the program content as necessary, and transmits the multiplexed content to the receiver 4 and the home server 40. Further, the service server 3 transmits an AIT to the receiver 4 and the home server 40 in response to receiving an AIT acquisition request from the receiver 4 and the home server 40.

In addition, the service server 3 is a program for specifying program content included in information indicating a viewer's viewing state (hereinafter referred to as viewing state information) for a predetermined scene of the program content and a broadcast signal broadcast from the broadcast station 1. The association information associated with the ID is received from the home server 40.
Here, the association information includes image association information, which is an image of the viewer when the viewing state information views the predetermined scene, and audio association, which is the audio information of the viewer when the viewing state information views the predetermined scene. Information.

  Further, the service server 3 performs viewer motion recognition and facial expression recognition based on the image association information received from the home server 40, and uses this to calculate the degree of interest related to the viewer's program and the like. A program recommended to the user and information related to the program and a predetermined scene are generated, and the information is transmitted to the receiver 4, the home server 40, a mobile terminal connected to the receiver 4, and the like.

  Further, the service server 3 extracts the utterance content of the viewer and the surrounding background noise section based on the voice association information received from the home server 40, and the degree of interest regarding the viewer's program using the extraction result. And the like, a program recommended for the viewer, information related to the program and a predetermined scene are generated, and the information is transmitted to the receiver 4, the home server 40, a portable terminal connected to the receiver 4, and the like.

  The receiver 4 performs predetermined processing on the program content received from the broadcast station 1 or through communication with the service server 3 to output the video and audio constituting the program content synchronously. .

The receiver 4 and the home server 40 as a receiving system appropriately acquire an application based on the AIT, and start and execute the acquired application. Further, for example, the receiver 4 can acquire application content from the service server 3 by an application being executed, and output the content in cooperation with the program content. Note that the output of application content does not necessarily have to be linked to program content.
Hereinafter, functions of the receiver 4 and the home server 40 will be described in detail.

First, for easy understanding, an outline of a part of the functions of the receiver 4 and the home server 40 will be described with reference to FIG.
FIG. 3 is a diagram showing an outline of the broadcasting / communication cooperation system 100.

In the example of FIG. 3, as viewing state information, data of an image showing the facial expression of the viewer and audio data uttered by the viewer are employed. For this reason, a camera device 5 for capturing the image and a microphone device 6 (hereinafter abbreviated as a microphone device 6) for inputting a viewer's voice are connected to the home server 40. In FIG. 3, the camera device 5 and the microphone device 6 are external devices as viewed from the home server 40, but this is merely an example, and as will be described later, the camera device 5 and the microphone device 6 are It may be built in the home server 40.
3, the camera device 5 and the microphone device 6 which are external devices when viewed from the home server 40 are also a camera device and a microphone device built in the mobile terminal 10 connected to the home server 40. Alternatively, a camera device and a microphone device connected to the mobile terminal 10 connected to the home server 40 may be used.
Further, the receiver 4 and the home server 40 can also operate in conjunction with the mobile terminal 10 configured with a smartphone, a mobile phone, or the like.

The receiver 4 receives a broadcast signal including program content broadcast by the broadcast station 1 and reproduces the program content.
Further, the receiver 4 acquires time information indicating the relative playback start timing of a predetermined scene in the program content.
Furthermore, the receiver 4 acquires a program ID that identifies the program content.
Then, the receiver 4 provides the acquired time information and program ID to the home server 40 by communication.
The home server 40 causes the camera device 5 to shoot a viewer when the receiver 4 is playing a predetermined scene in the program content, and acquires image data obtained as a result as viewing state information, The viewer's voice data input from the microphone device 6 is acquired as viewing state information.
Further, the home server 40 acquires time information and a program ID from the receiver 4.
Next, the home server 40 generates information associated with the acquired information (hereinafter referred to as association information). That is, the viewing state information and time information about the predetermined scene in the program content, and association information in which the program ID is associated are generated.
Then, the home server 40 provides the association information to the service server 3 by communication.

  Upon receiving the association information, the service server 3 receives various information related to a predetermined scene of the program content and information on another program content recommended for the viewer as service feedback information for the receiver 4 and the home. It transmits to the server 40 and the portable terminal 10.

Hereinafter, the receiver 4 will be described in detail with reference to FIG.
FIG. 4 is a block diagram illustrating a functional configuration of the receiver 4.
The receiver 4 includes a broadcast wave receiver 11, a first separator 12, a broadcast AIT acquisition unit 13, a communication unit 14 as a receiver communication means, a second separator 15, and a communication AIT acquisition unit 16. , An application execution control unit 17, a voice control unit 18, a display control unit 19, a speaker 20, a display 21, a memory 22, and an AIT storage unit 23.

  The broadcast wave receiving unit 11 receives a broadcast wave transmitted from the broadcast station 1 via the broadcast antenna 2.

  The first separator 12 extracts a broadcast signal, that is, a TS packet, by demodulating the broadcast wave received by the broadcast wave receiver 11. The first separation unit 12 refers to the PSI / SI of the TS packet, determines the type of data included in the TS packet, and separates various data such as video, audio, EIT, and program ID. Extract. Further, the first separation unit 12 separates and extracts the AIT by referring to the section and the PES according to a predetermined AIT transmission method.

  Subsequently, when the data included in the PES of the TS packet is audio data, the first separation unit 12 outputs the audio data to the audio control unit 18. In addition, when the data included in the PES of the TS packet is video data, the first separation unit 12 outputs the video data to the display control unit 19.

  In addition, the first separation unit 12 stores the extracted EIT, program ID, and other various data in the memory 22. Further, when the first separation unit 12 extracts the AIT, the first separation unit 12 outputs the extracted AIT to the broadcast AIT acquisition unit 13.

  The broadcast AIT acquisition unit 13 acquires the AIT output from the first separation unit 12 and stores it in the AIT storage unit 23.

  The communication unit 14 transmits / receives various types of information to / from the server group 3 and the home server 40 via the communication network N, receives an operation signal from a remote controller R (hereinafter abbreviated as remote controller R), Various information is transmitted to and received from the terminal 10.

  The second separation unit 15 determines the type of data transmitted from the server group 3 via the communication network N and received by the communication unit 14, and separates the data output destination based on the determination result. For example, if the second separation unit 15 determines that the received data is an AIT, the second separation unit 15 outputs the AIT to the communication AIT acquisition unit 16. When the second separation unit 15 determines that the received data is an application, the second separation unit 15 outputs the application to the application execution control unit 17.

  Further, when the second separation unit 15 determines that the received data is a TS packet, the second separation unit 15 separates and extracts various data such as video, audio, AIT, and program ID from the TS packet. The second separation unit 15 outputs this AIT to the communication AIT acquisition unit 16 when the AIT is extracted, and outputs this data to the application execution control unit 17 when data other than the AIT is extracted.

  The communication AIT acquisition unit 16 acquires the AIT output from the second separation unit 15 and stores it in the AIT storage unit 23.

  The application execution control unit 17 acquires an application based on a user's execution instruction via the AIT or the remote controller R, and controls execution of the acquired application. For example, when an application that generates and outputs association information is activated, the application execution control unit 17 executes NPT or PTS as time information indicating the relative playback start timing of a predetermined scene in the program content by executing the application. Is transmitted from the display control unit 19 to the home server 40 via the communication unit 14. Further, the application execution control unit 17 transmits the program ID stored in the memory 22 and the user ID for identifying the user to the home server 40 via the communication unit 14 by executing the application.

The memory 22 stores meta information of program contents such as EIT and program ID and other various information.
The AIT storage unit 23 stores the AIT acquired by the broadcast AIT acquisition unit 13 and the communication AIT acquisition unit 16.

Further, the home server 40 will be described in detail below with reference to FIG.
FIG. 5 is a block diagram showing a functional configuration of the home server 40.
The home server 40 includes a communication unit 44 as a home server communication unit, a separation unit 45, a communication AIT acquisition unit 46, an application execution control unit 47, an AIT storage unit 53, and a device connection unit 54 as an external device connection unit. And an associating unit 55.

  The communication unit 44 transmits / receives various information to / from the server group 3 and the receiver 4 via the communication network N, receives an operation signal from a remote controller R (hereinafter abbreviated as a remote controller R), Various information is transmitted to and received from the terminal 10.

  The separation unit 45 determines the type of data transmitted from the server group 3 or the receiver 4 via the communication network N and received by the communication unit 44, and separates the data output destination based on the determination result. . For example, when the separation unit 45 determines that the received data is an AIT, the separation unit 45 outputs the AIT to the communication AIT acquisition unit 46. If the separating unit 45 determines that the received data is an application, the separating unit 45 outputs the application to the application execution control unit 17. If the separation unit 45 determines that the data received from the receiver 4 is NPT, PTS, program ID, or user ID, the separation unit 45 sends the NPT, PTS, program ID, and user ID to the application execution control unit 47. Output.

  Further, when the separation unit 45 determines that the received data is a TS packet, the separation unit 45 separates and extracts various data such as video, audio, AIT, and program ID from the TS packet. When the AIT is extracted, the separation unit 45 outputs the AIT to the communication AIT acquisition unit 46. When the separation unit 45 extracts data other than the AIT, the separation unit 45 outputs the data to the application execution control unit 47.

  The communication AIT acquisition unit 46 acquires the AIT output from the separation unit 45 and stores it in the AIT storage unit 53.

The application execution control unit 47 acquires an application based on a user's execution instruction via the AIT or the remote controller R, and controls execution of the acquired application. For example, when an application that generates and outputs association information is activated, the application execution control unit 47 outputs the NPT, PTS, program ID, and user ID received from the receiver 4 to the association unit 55 by executing the application. To do.
The AIT storage unit 53 stores the AIT acquired by the communication AIT acquisition unit 46.

The device connection unit 54 connects devices such as the camera device 5 and the microphone device 6 and supplies input information from the devices to the association unit 55 as viewing state information.
Specifically, the device connection unit 54 views data of an image obtained as a result of the viewer taking a picture with the camera device 5 by an application executed by the application execution control unit 47 (hereinafter also referred to as “Hybridcast application”). Obtained as state information and output to the associating unit 55. Further, the device connection unit 54 acquires the audio data of the viewer input from the microphone device 6 as the viewing state information by the application executed by the application execution control unit 47 and outputs the acquired data to the associating unit 55.
The camera device 5 and the microphone device 6 connected to the device connection unit 54 may be a camera device and a microphone device built in the mobile terminal 10 connected to the home server 40, or connected to the mobile terminal 10. It may be a camera device and a microphone device.
The device connected to the device connection unit 54 is not limited to the camera device 5 and the microphone device 6. For example, a gyro sensor that can acquire the viewer's position information or an optical sensor that can detect the viewer's operation. It may be.
Furthermore, the connection to the device connection unit 54 does not only mean connecting to an external device but also means connecting to a built-in device. Accordingly, the camera device 5 and the microphone device 6 are not particularly required to be external devices, and may be built in the home server 40.
Further details of the device connection unit 54 will be described later with reference to FIGS.

The associating unit 55 acquires NPT or PTS from the application execution control unit 47 as time information indicating the relative playback start timing of the predetermined scene in the program content. The associating unit 55 acquires image or audio data from the device connecting unit 54 as viewing state information indicating the viewing state of the viewer when viewing the predetermined scene. The associating unit 55 acquires ID information such as a program ID and a user ID from the application execution control unit 47.
The association unit 55 generates association information by associating the acquired information, that is, NPT or PTS, viewing state information as image or audio data, and ID information, via the communication unit 44. To the service server 3.
Further details of the associating unit 55 will be described later with reference to FIGS.

  Further, the device connection unit 54 of the home server 40 having such a functional configuration will be described below with reference to FIGS.

  As described above, devices such as the camera device 5 and the microphone device 6 are connected to the device connection unit 54. However, even if the device is simply physically connected, the device connection unit 54 cannot output information from the device as viewing state information to the associating unit 55 unless the device is discovered and recognized. .

FIG. 7 is a diagram for explaining an operation related to discovery of such a device, and is a diagram illustrating a relationship of layers of the Hybridcast function.
As shown in FIG. 7, as the layer of the Hybridcast function, there are hardware, system software, a virtual machine, a Hybridcast basic library and an extension library, and a Hybridcast application in order from the lower layer to the upper layer.
In the present embodiment, the device discovery operation in the device connection unit 54 is performed by a device discovery library which is one of extended libraries (software programs).

FIG. 8 is a diagram illustrating the concept of the device connection unit 54.
As shown in FIG. 8, the Hybridcast application executed by the application execution control unit 47 communicates with a device manager provided in the device connection unit 54, an HID (Human Interface Device), and the like.
Here, as for the device manager implementation and the HID implementation in the general physical network layer, the existing ones are used in this embodiment. For this reason, various listeners as shown in the sequence diagrams of FIGS. 9 to 11 are implemented, and an implementation including a device manager and an HID is generated as a class for generating and processing various events as shown in the sequence diagrams. The device connection unit 54 is realized by implementing the annotation class.

FIG. 9 shows a sequence diagram when a predetermined external device is connected as a device when a service of a Hybridcast application that uses the device for operation recognition or the like is started.
In step S <b> 1, the associating unit 55 acquires ID information such as a program ID and a user ID from the memory 22. However, acquisition of ID information is performed not only at the processing timing of step S1, but also at any time at the change of the program content from the broadcast station 1.

Next, the following steps S2 to S9 are executed as a sequence of device discovery parts by the device manager.
In step S2, the Hybridcast application (abbreviated as an application in the description of the sequence diagrams of FIGS. 9 to 11) executed by the application execution control unit 47 registers a device manager instance in the device connection unit 54.
In step S <b> 3, the device manager notifies the association unit 55 of the application.
In step S4, the associating unit 55 notifies the device manager of the associating unit physical information. Here, the association unit physical information is information indicating where the viewer state information (image or sound) should be physically flowed.
In step S5, the application generates a device discovery listener. That is, a device discovery event is registered.
In step S6, the device manager attempts to find a device for the HID.
In step S7, the HID returns the response to the device manager.
In step S8, the device manager generates and registers a new device instance based on the device information obtained from the response in step S7 and the association unit physical information notified in step S4.
In step S9, the device manager notifies the app of a new device by sending device information to the app.

In this way, when the processing of steps S2 to S9 is executed as a device discovery part sequence by the device manager, the following steps S10 to S14 are subsequently executed as a description and control part sequence by the device instance. The process is executed.
In step S10, the application registers a device status listener for the device instance.
In step S <b> 11, the device instance confirms the device status (for example, the status of whether the power state is ON or OFF) with respect to the HID.
In step S12, the HID notifies the device instance of the status of the device, for example, the power supply status and device specific information. When the power state of the device is OFF, the processing shown in FIG. 10 is further required, and this processing will be described later.
In step S13, the device instance notifies the app of the device status notified from the HID.
In step S <b> 14, the device instance sends the notification of the association unit 55 to the HID so as to execute control to output the input data of the device (data that is image or audio data and becomes viewer state information) to the association unit 55. To do.

Here, the description of the device instance and the sequence of the control part when the power state of the device is OFF will be described.
First, the process of step S11 is executed regardless of the power state of the device. However, when the power state of the device is OFF, the process according to the sequence diagram of FIG. 10 is used instead of the process of step S12. Is executed.
FIG. 10 is a sequence diagram of the description by the device instance and the control part when the power state of the device is OFF.
The process of step S12-1 is substantially the same as step S12. However, in step S12-1, the device instance is notified that the power state is OFF.
In step S33, the device instance makes a request to the device manager to change the power state of the device to the ON state (hereinafter referred to as a device power ON request).
In step S34, the device manager changes the power state of the device to the ON state.
In step S12-2, the HID notifies the device instance of the device status, for example, the power supply status and device-specific information. In this case, the device instance is notified that the power state is ON.
Thereafter, the processes of steps S13 and S14 of FIG. 9 described above are executed.

In this way, when the process of steps S10 to S14 in FIG. 9 (the process of FIG. 10 when the device is in the power-off state) is executed as the description of the device instance and the control part sequence, it continues next. The following steps S15 to S19 are executed.
In step S15, the application requests an input event from the device instance.
In step S16, the device instance requests an input event from the HID.
In step S <b> 17, the device, that is, the camera device 5 or the microphone device 6, which is an external device here, inputs image or audio data to the HID.
In step S18, the HID provides the association unit 55 with the image or audio data input in the process of step S17 as input data. The associating unit 55 acquires the input data as viewer state information.
In step S19, the application provides NPT / PTS at the time of processing in step S18 to the associating unit 55.
Thereafter, each time an input event is requested from the application, the processes of steps S15 to S19 are repeatedly executed.

In summary, the device manager functions in the device connection unit 54. The main function of this device manager is to discover an HID (device) and generate a device instance using the device information of the discovered device and the association unit information together.
The function of the device instance generated by the device manager in this way is to receive an input event from the application and to control the HID and the associating unit 55.
The function of the associating unit 55 includes input data from the device (HID) (image and audio data, which is viewer status information), and internal information of the receiver 4 connected to the home server 40. (Time information such as NPPTTS and ID information) is associated with each other.

  By the way, the device connected to the device connection unit 54 is not particularly limited to an external device as described above, and may be a built-in device of the home server 40.

FIG. 11 shows a sequence diagram when the built-in device of the home server 40 is used as a device when a service of a Hybridcast application that uses the device for operation recognition or the like is started.
Comparing FIG. 11 and FIG. 9, steps S51 to S54 in FIG. 11 are the same processes as steps S1 to S4 in FIG.
However, in FIG. 11, there is no processing corresponding to steps S5 to S7 of FIG. This is because there is no need to discover the internal device. For this reason, in step S55 of FIG. 11, the device manager generates and registers a new device instance based on the known information of the built-in device and the association unit physical information notified in step S54.
In FIG. 11, there is no description of the device instance and the sequence of the control part. That is, in FIG. 11, there is no processing corresponding to steps S10 to S14 of FIG. 9 (FIG. 10 when the power supply state of the device is OFF). This is because a built-in device does not require a status listener. That is, the situation can be changed from the device manager.
Note that steps S56 to S60 in FIG. 11 are the same processes as steps S15 to S19 in FIG. That is, every time an input event is requested from the application, the processes of steps S56 to S60 are repeatedly executed.

  The details of the device connection unit 54 in the home server 40 having the functional configuration of FIG. 4 have been described above with reference to FIGS. Next, details of the associating unit 55 in the home server 40 will be described with reference to FIGS. 12 to 15.

FIG. 12 is a functional block diagram illustrating an example of a functional configuration of the associating unit 55.
The association unit 55 generates image association information using the image information acquisition unit 551, the relative time information acquisition unit 552, the ID (program / user ID) acquisition unit 553, and the image association unit 554.
The image information acquisition unit 551 views data (see step S18 in FIG. 9) of the image (moving image frame or still image) output as input data from the device connection unit 54, that is, viewing when a predetermined scene of the program content is viewed. Data of an image in which the viewer is shown is acquired as viewer state information.
The relative time information acquisition unit 552 acquires NPT or PTS (see step S19 in FIG. 9) as time information indicating the relative playback start timing of the predetermined scene output from the application execution control unit 47.
The ID (program / user ID) acquisition unit 553 acquires the program ID and user ID (ID information in step S1 of FIG. 9) output from the application execution control unit 47.
The image association unit 554 includes the image data acquired by the image information acquisition unit 551, the NPT or PTS acquired by the relative time information acquisition unit 552, and the program acquired by the ID (program / user ID) acquisition unit 553. The association information is generated by associating the ID and the user ID. Hereinafter, the association information generated by the image association unit 554 is particularly referred to as image association information. The image association unit 554 outputs image association information to the communication unit 44 at any time until the application execution control unit 47 terminates the application.

FIG. 13 is a flowchart for explaining the processing flow of the associating unit 55 in the example of FIG.
In step S101, the associating unit 55 acquires image (moving image frame or still image) data with the image information acquiring unit 551, acquires time information (NPT or PTS) with the relative time information acquiring unit 552, and acquires an ID. In part 553, the program ID and the user ID are acquired.
In step S102, the image association unit 554 generates image association information by associating various information acquired in step S101, that is, image data, time information, a program ID, and a user ID.
In step S <b> 103, the image association unit 554 outputs the image association information generated in step S <b> 102 to the communication unit 44.
In step S104, the associating unit 55 determines whether or not the application execution control unit 47 has ended the application. If the application execution control unit 47 has not ended the process, the process returns to step S101.

The image association information output from the association unit 55 is transmitted to the service server 3 (see FIG. 4) via the communication unit 44.
The service server 3 performs image processing based on the received image association information. As a result, the service server 3 estimates information such as “in which scene the viewer laughed” and “in which scene he / she lost interest”, and the viewer can watch the viewer without performing a special action. The state can be estimated.

FIG. 14 is a functional block diagram illustrating another example of the functional configuration of the associating unit 55.
The associating unit 55 includes an audio information acquiring unit 555, a relative time information acquiring unit 552, an ID (program / user ID) acquiring unit 553, and an audio associating unit 556. Information).
The audio information acquisition unit 555 uses, as audio information, audio data (see step S18 in FIG. 9) output as input data from the device connection unit 54, that is, audio data generated when a predetermined scene of the program content is viewed. get.
The audio association unit 556 includes the audio information acquired by the audio information acquisition unit 555, the NPT or PTS acquired by the relative time information acquisition unit 552, and the program ID acquired by the ID (program / user ID) acquisition unit 553. And association information is generated by associating the user ID with the user ID. Hereinafter, the association information generated by the audio association unit 556 is particularly referred to as audio association information. The voice association unit 556 outputs the voice association information to the communication unit 44 at any time until the application execution control unit 47 ends the application.

FIG. 15 is a flowchart for explaining the processing flow of the associating unit 55 in the example of FIG.
In step S <b> 201, the associating unit 55 acquires audio information with the audio information acquisition unit 555, acquires time information (NPT or PTS) with the relative time information acquisition unit 552, and acquires a program ID and user ID with the ID acquisition unit 553. get.
In step S202, the audio association unit 556 generates audio association information by associating various information acquired in step S201, that is, audio information, time information, a program ID, and a user ID.
In step S203, the voice association unit 556 outputs the voice association information generated in step S202 to the communication unit 44.
In step S204, the associating unit 55 determines whether or not the application execution control unit 47 has ended the application. If the application execution control unit 47 has not ended the process, the process returns to step S201.

According to this embodiment, there are the following effects.
(1) The receiver 4 includes time information indicating a relative reproduction start timing of a predetermined scene in content included in a broadcast signal broadcast by the broadcast station 1, and a viewing state of a viewer when the predetermined scene is viewed. The program ID for specifying the content included in the broadcast signal is transmitted to the home server 40 by the communication unit 14.
The home server 40 generates association information in which the time information and the program ID transmitted from the receiver 4 are associated with the viewing state of the viewer when viewing the predetermined scene by the association unit 55, and the generated association Information is transmitted to the service server 3 by the communication unit 44.

Thereby, in the home server 40, the predetermined scene of the content specified by the program ID reproduced by the receiver 4 is associated with the viewing state of the viewer who has viewed the predetermined scene, and is transmitted to the service server 3. .
Accordingly, in a system that coordinates broadcasting and communication, the server can easily and appropriately acquire the viewing state of a predetermined scene of content broadcast by a broadcasting station.

(2) Even if the home server 40 does not include a camera device or a microphone device, the viewing state information can be acquired by connecting the camera device or the microphone device as an external device to the device connection unit 54.
As a result, in a system for linking broadcasting and communication, even if the home server does not include viewing state acquisition means, the server can easily and appropriately acquire the viewing state of the predetermined scene of the content broadcast by the broadcast station. Become.
In addition, since the viewing state acquisition means can be excluded from the essential items of the home server that can receive the service, the spread of the home server that can receive the service can be promoted.

(3) Even if the home server 40 is not equipped with a camera device or a microphone device, it is possible to acquire viewing state information by connecting the mobile terminal 10 that can acquire the viewing state of the viewer to the device connection unit 54.
As a result, in a system for linking broadcasting and communication, even if the home server does not include viewing state acquisition means, the server can easily and appropriately acquire the viewing state of the predetermined scene of the content broadcast by the broadcast station. Become.

  (4) Since the home server 40 includes a camera device and a microphone device, in a system for coordinating broadcasting and communication, even if a camera device and a microphone device are not separately prepared, the server broadcasts content broadcasted by a broadcasting station. The viewing state of the predetermined scene can be acquired easily and appropriately.

  (5) The image information acquisition unit 551 of the associating unit 55 of the receiver 4 transmits, from the camera device 5, image data obtained by photographing a viewer when viewing a predetermined scene as viewing state information via the device connection unit 54. get. The image association unit 554 includes image association information in which the image data acquired by the image information acquisition unit 551, the time information acquired by the relative time information acquisition unit 552, and the program ID acquired by the ID acquisition unit 553 are associated with each other. Generate.

  As a result, the viewer's image data can be used as the viewing state information. Therefore, in the system for coordinating the broadcasting and the communication, the server is not arbitrarily viewing the predetermined scene of the content broadcast by the broadcasting station. Can be acquired easily and appropriately.

  (6) The audio information acquisition unit 555 of the associating unit 55 of the receiver 4 acquires, from the microphone device 6, audio data generated when the viewer views a predetermined scene as viewing state information via the device connection unit 54. To do. The audio association unit 556 includes audio association information in which the audio data acquired by the audio information acquisition unit 555, the time information acquired by the relative time information acquisition unit 552, and the program ID acquired by the ID acquisition unit 553 are associated with each other. Generate.

  As a result, since the audio data of the viewer can be used as the viewing state information, in the system for coordinating broadcasting and communication, the server is not arbitrarily viewing the predetermined scene of the content broadcast by the broadcasting station. Can be acquired easily and appropriately.

It should be noted that the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within a scope that can achieve the object of the present invention are included in the present invention.
In this embodiment, for example, a camera device and a microphone device are provided as devices. However, the present invention is not limited to this, and any camera device or microphone device can be provided as long as the viewer's viewing state information can be acquired. That's fine.

1 Broadcasting station 3 Service server 4 Receiver 14 Communication unit 40 Home server 44 Communication unit 55 Association unit N Network

Claims (7)

A receiver connected to a service server via a network and receiving a broadcast signal including content broadcast by a broadcasting station, and a home server connected via the network to the receiver and the service server A system,
The receiver includes receiver communication means for transmitting time information indicating a relative reproduction start timing of a predetermined scene in the content and a program ID for specifying the content included in the broadcast signal to the home server,
The home server is
Association means for generating association information in which the time information and the program ID transmitted from the receiver are associated with the viewing state of the viewer when viewing the predetermined scene;
And a home server communication unit that transmits the association information generated by the association unit to the service server.   The receiving system according to claim 1, wherein the home server includes an external device connection unit to which a viewing state acquisition unit that acquires a viewing state of a viewer when viewing the predetermined scene is connected. The home server is connected to a mobile terminal via a network,
The external device connection means is connected to the viewing state acquisition means included in or connected to the mobile terminal via the home server communication means.
The receiving system according to claim 2. The home server includes a viewing state acquisition unit that acquires a viewing state of a viewer when viewing the predetermined scene;
The receiving system according to claim 1. The viewing state acquisition means is a camera device that acquires an image,
The association means includes
Image information acquisition means for acquiring, from the camera device, the image obtained by photographing the viewer when viewing the predetermined scene via the external device connection means;
Relative time information acquisition means for acquiring the time information;
ID acquisition means for acquiring the program ID;
Image association means for generating image association information associating the image acquired by the image information acquisition means, the time information acquired by the relative time information acquisition means, and the program ID acquired by the ID acquisition means. When,
The receiving system according to claim 2 or 3. The viewing state acquisition means is a microphone device that acquires audio information,
The association means includes
Audio information acquisition means for acquiring, from the microphone device, the audio information emitted when a viewer views the predetermined scene via the external device connection means;
Relative time information acquisition means for acquiring the time information;
ID acquisition means for acquiring the program ID;
Audio association for generating audio association information in which the audio information acquired by the audio information acquisition unit, the time information acquired by the relative time information acquisition unit, and the program ID acquired by the ID acquisition unit are associated with each other. Means,
The receiving system according to claim 2 or 3. A receiver connected to a service server via a network and receiving a broadcast signal including content broadcast by a broadcasting station, and a computer for controlling a home server connected to the service server via the network;
Time information indicating the relative playback start timing of a predetermined scene in the content transmitted from the receiver, a program ID for specifying the content included in the broadcast signal, and the viewer when viewing the predetermined scene An association means for generating association information associating a viewing state with
Communication means for executing control for transmitting the association information generated by the association means to the service server;
Program to function as.

Images