JP5859223B2 - Receiver and program - Google Patents

Description

  The present invention relates to a receiver and a program having a function of coordinating broadcasting and communication.

  In recent years, with the progress of digitalization of broadcasting and broadbandization of communication, research and development for realizing a broadcasting / communication cooperation service has been performed.

  In this broadcasting / communication cooperation service, it is assumed that a plurality of contents are distributed using different transmission paths of broadcasting and communication, and the plurality of distributed contents are integrated and presented in a receiver such as a digital television. Is done.

  By the way, at present, in data broadcasting, various languages used for the broadcasting service called BML (Broadcast Markup Language) developed for multimedia encoding based on XML (extensible Markup Language) are used. Content is provided (see Patent Document 1).

JP 2003-60931 A

  By the way, in data broadcasting, since it is necessary to multiplex all information on a broadcast wave, there is a certain limitation on the data capacity that can be transmitted on the broadcast wave.

  Therefore, it is desired to realize a system that links broadcasting with communication with no limit on the data capacity that can be transmitted.

In particular, in such a system that links broadcasting and communication, a server that provides various services to the receiver by communication is constructed.
Therefore, for example, the viewer's viewing state of a predetermined scene in the content broadcast by the broadcast station, for example, the viewer's facial expression and sound when viewing the predetermined scene, is acquired from each of a number of receivers. Thus, it is assumed that servers will be used for services.

  In this case, it is required to implement an appropriate mechanism for connecting a device such as a camera that captures the viewer's facial expression or a microphone that inputs the viewer's voice to the receiver.

  The present invention has been made in view of such a situation, and in a system for coordinating between broadcasting and communication, it is one to realize an appropriate mechanism for connecting a device to a receiver so as to function. Objective.

  A receiver according to the present invention is a receiver that is connected to a server via a network and receives a broadcast signal that is broadcast from a broadcast station and is configured by multiplexing a plurality of pieces of information together with program content. Associates execution control means for controlling execution of an application managed by application management information multiplexed on a broadcast signal or transmitted via the network, information relating to the program content, and input information from the device An association unit; a device manager that generates a device instance based on the information of the association unit and the information of the device; and the device and the association unit that are generated by the device manager, and controls the application. When receiving an input event request from And a configuration and a device instance be supplied to the association means acquires the input information from the chair.

According to this configuration, a device instance is generated by the device manager based on the association unit information and the device information. The device and the association unit are controlled by the device instance. When there is an input event request from the application, input information is acquired from the device and supplied to the association unit.
In this way, an appropriate mechanism for connecting a device to a receiver and causing it to function in a system that links broadcasting and communication is realized.

In this case, it is preferable that the device is an external device connected from the outside to the receiver, and the device manager further discovers the device as the external device.
According to this configuration, a device as an external device is further discovered by the device manager.
Thereby, in a system for coordinating broadcasting and communication, an appropriate mechanism for connecting and functioning a receiver to a device as an external device is realized.

A program according to the present invention is a receiver that is connected to a server via a network and receives a broadcast signal that is broadcast from a broadcasting station and is configured by multiplexing a plurality of pieces of information together with program content. A computer that controls a receiver including an execution control unit that controls execution of an application managed by application management information multiplexed on a signal or transmitted via the network, information about the program content, and a device Associating means for associating input information from the device, a device manager for generating a device instance based on the information of the associating means and the information of the device, and after the device manager generates the device instance, the device and the associating means Control and from the application Upon receiving the power event request was configured to function as a device instance, supplied to the association means acquires the input information from the device.
According to this configuration, the same operational effects as the receiver according to the present invention can be obtained.

  According to the present invention, it is possible to realize an appropriate mechanism for connecting a device to a receiver so as to function in a system that links broadcasting and communication.

1 is an overall configuration diagram of a broadcasting / communication cooperation system according to an embodiment of the present invention. It is a figure which shows the stack | stuck of the transmission protocol of the broadcast signal which concerns on digital broadcasting. It is a figure which shows the outline | summary of a broadcast communication cooperation system. It is a block diagram which shows the function structure of a receiver. It is a figure which shows an example of AIT described in the XML format. It is a figure explaining operation | movement regarding discovery of a device, Comprising: It is a figure which shows the relationship of the layer of Hybridcast function. It is a figure which shows the concept of a device connection part. FIG. 10 is a sequence diagram when a predetermined external device is connected as a device when a service of a Hybridcast application that uses the device for operation recognition or the like is started. FIG. 10 is a sequence diagram of a description and a control portion by a device instance when the power state of the device is an OFF state. FIG. 10 is a sequence diagram when a built-in device of a receiver is used as a device when a service of a Hybridcast application that uses the device for operation recognition or the like is started. It is a block diagram which shows an example of a function structure of an correlation part. It is a flowchart explaining the flow of a process of the correlation part of FIG. FIG. 12 is a block diagram illustrating an example of a functional configuration of an associating unit that is different from FIG. 11. It is a flowchart explaining the flow of a process of the correlation part of FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is an overall configuration diagram of a broadcasting / communication cooperation system 100 according to an embodiment of the present invention. The broadcasting / communication cooperation system 100 includes a broadcasting station 1, a broadcasting antenna 2, a service server 3, and a receiver 4. In the broadcasting / communication cooperation system 100, a receiver 4 uses a broadcasting service provided from a broadcasting station 1 by an ISDB (Integrated Services Digital Broadcasting) system and a communication network N configured by the Internet or the like. A communication service provided from the service server 3 is linked. Thereby, the broadcasting / communication cooperation service is provided to the user of the receiver 4.
In the present embodiment, the broadcasting / communication cooperation system 100 is a Hybridcast (registered trademark) system. Hybridcast is a mechanism for constructing a system that takes advantage of broadcast characteristics such as broadcastability, high quality, and high reliability and communication characteristics that can meet the individual demands of viewers. The Hybridcast system can realize a hybrid service of broadcasting and communication from the viewpoint of strengthening the service by utilizing communication while placing the broadcasting service at the center.

The broadcasting station 1 includes a general digital broadcasting broadcasting facility (not shown) configured to include a program organization facility, a program transmission facility, and a transmission facility.
The broadcasting station 1 produces contents, event information (Event Information Table: EIT), AIT (hereinafter also referred to as application management information as appropriate), and the like, using broadcasting equipment. Then, the broadcast station 1 generates a broadcast signal in which these contents, event information, AIT, and the like are multiplexed by a broadcast facility, modulates the broadcast signal into a broadcast wave, and transmits the broadcast signal from the broadcast antenna 2.

  Content included in such broadcast waves is generated asynchronously with program content in addition to content including video and audio data constituting a program broadcast according to a broadcast schedule (hereinafter referred to as program content). There is content (hereinafter referred to as emergency content) including content, for example, earthquake early warning.

The event information includes meta information related to the content such as the name of the program content, the broadcast date and time of the program content, and the description of the program content. Hereinafter, the event information in the table format is referred to as EIT.
The AIT (application management information) is information for managing one or more applications that can be executed by the receiver 4. The AIT includes individual management information corresponding to each of the one or more applications and managing each of the one or more applications. The individual management information includes an application ID for identifying an application to be managed, life cycle control information for controlling the life cycle of the application, location information indicating the location of the application (see FIG. 5 described later), and the like. .

  The content provided by the application includes content linked to the program content and content not linked to the program content. Hereinafter, content provided by an application is referred to as application content.

  The broadcast signal is the same as the broadcast signal of the conventional digital broadcast, and is defined by ARIB (registered trademark) (Association of Radio Industries and Broadcast) standard.

  FIG. 2 is a diagram illustrating a stack of broadcast signal transmission protocols related to digital broadcasting. As shown in FIG. 2, various types of data such as video and audio provided by digital broadcasting are stored in TS packets (transport stream packets) defined by the international standard MPEG-2 Systems and multiplexed in a time division manner. Is transmitted.

  In the TS packet, as shown in FIG. 2, PSI (Program Specific Information) / SI (Service Information) is defined for a section (Section). PSI / SI includes information indicating the type of data stored in the TS packet and information indicating the type of content. The above-mentioned EIT is included in SI.

  Data transmission using TS packets is classified into a method of transmitting data using sections and a method of transmitting data using PES (Packetized Elementary Stream) packets (data stream transmission method).

  Methods for transmitting data using sections include a data carousel transmission method and an event message transmission method.

  The data carousel transmission system is a transmission system that repeatedly transmits one or more data at a constant period. Each piece of data (module) transmitted by the data carousel transmission method is attached with identification information for uniquely identifying them. The data carousel transmission method is used for causing the receiver 4 to acquire individual data at an arbitrary timing.

  The event message transmission method is a method for sending a trigger signal from the broadcast station 1 to the receiver 4. The event message transmission method is used when a message with a small amount of data is transmitted from the broadcasting station 1 to the receiver 4.

  The data stream transmission method is a transmission method in which data to be transmitted is contained in a PES packet and transmitted as a stream. The data stream transmission method is used for transmitting real-time data such as video, audio, and caption data, and data that needs to be synchronized with other streams.

Here, AIT can be transmitted by various methods using TS packets.
That is, the AIT can be transmitted by describing the EIT included in the SI of the TS packet.

  The AIT can be transmitted by a data carousel transmission system using sections. When the AIT is transmitted by the data carousel transmission method, identification information is attached to the module so that the receiver 4 can recognize the AIT.

  The AIT can be transmitted as PES multiplexed with content video and audio.

The AIT can be described in binary representation or text representation by XML (Extensible Markup Language) and stored in the TS packet.
In this embodiment, the AIT transmission method is defined in advance in at least one of the above transmission methods.

  BML developed for multimedia coding based on XML is based on xHTML defined by W3C (Registered Trademark) (World Wide Web Consortium), and Java Script (Registered Trademark) is used as a procedural language. This is a standard defined in consideration of consistency with international standards using ECMAScript based on it.

  Although not shown, the service server 3 includes a content distribution server that distributes broadcasted program content and application content, and an application distribution server that distributes various applications that operate on the receiver 4.

The service server 3 is communicably connected to the equipment of the broadcast station 1 and receives program content and metadata of the program content from the broadcast station 1.
The service server 3 transmits the program content to the receiver 4 in response to receiving the program content acquisition request from the receiver 4.

  Further, the service server 3 transmits the application to the receiver 4 in response to receiving the application acquisition request from the receiver 4. Furthermore, the service server 3 transmits application content including data such as video and audio to the receiver 4 when an application is executed in the receiver 4.

  Further, the service server 3 multiplexes the AIT with the program content as necessary, and transmits the multiplexed AIT to the receiver 4. Further, the service server 3 transmits an AIT to the receiver 4 in response to receiving an AIT acquisition request from the receiver 4.

In addition, the service server 3 is a program for specifying program content included in information indicating a viewer's viewing state (hereinafter referred to as viewing state information) for a predetermined scene of the program content and a broadcast signal broadcast from the broadcast station 1. The association information in which the ID is associated is received from the receiver 4.
Here, the association information includes image association information, which is an image of the viewer when the viewing state information views the predetermined scene, and audio association, which is the audio information of the viewer when the viewing state information views the predetermined scene. Information.

  Further, the service server 3 performs viewer motion recognition and facial expression recognition based on the image association information received from the receiver 4, and uses this to calculate the degree of interest related to the viewer's program and the like. A program recommended to the user or information related to the program or a predetermined scene is generated, and the information is transmitted to the receiver 4 or a portable terminal connected to the receiver 4.

  Further, the service server 3 extracts the utterance content of the viewer and the surrounding background noise section based on the voice association information received from the receiver 4, and uses this extraction result to show the degree of interest regarding the viewer's program. And the like, and a program recommended for the viewer and information related to the program and a predetermined scene are generated, and the information is transmitted to the receiver 4 and a mobile terminal connected to the receiver 4.

The receiver 4 performs predetermined processing on the program content received from the broadcast station 1 or through communication with the service server 3 to output the video and audio constituting the program content synchronously. . Further, the receiver 4 appropriately acquires an application based on the AIT, and activates and executes the acquired application. Further, for example, the receiver 4 can acquire application content from the service server 3 by an application being executed, and output the content in cooperation with the program content. Note that the output of application content does not necessarily have to be linked to program content.
Hereinafter, the function of the receiver 4 will be described in detail.

First, in order to facilitate understanding, an outline of some of the functions of the receiver 4 will be described with reference to FIG.
FIG. 3 is a diagram showing an outline of the broadcasting / communication cooperation system 100.

  In the example of FIG. 3, as viewing state information, data of an image showing the facial expression of the viewer and audio data uttered by the viewer are employed. For this reason, a camera device 5 for capturing the image and a microphone device 6 (hereinafter abbreviated as a microphone device 6) for inputting a viewer's voice are connected to the receiver 4. In FIG. 3, the camera device 5 and the microphone device 6 are external devices when viewed from the receiver 4, but this is only an example, and as will be described later, the camera device 5 and the microphone device 6 are It may be built in the receiver 4. The receiver 4 can also operate in conjunction with the mobile terminal 10 configured with a smartphone, a mobile phone, or the like.

The receiver 4 receives a broadcast signal including program content broadcast by the broadcast station 1 and reproduces the program content.
At this time, the receiver 4 causes the camera device 5 to photograph the viewer while reproducing a predetermined scene in the program content, and obtains image data obtained as a result as viewing state information, The audio data of the viewer input from the device 6 is acquired as viewing state information.
Further, the receiver 4 acquires time information indicating the relative playback start timing of a predetermined scene in the program content.
Furthermore, the receiver 4 acquires a program ID that identifies the program content.
Next, the receiver 4 generates information associated with the acquired information (hereinafter referred to as association information). That is, the viewing state information and time information about the predetermined scene in the program content, and association information in which the program ID is associated are generated.
Then, the receiver 4 provides the association information to the service server 3 by communication.

  Upon receiving the association information, the service server 3 receives various information related to a predetermined scene of the program content and information on another program content recommended to the viewer as service feedback information for the receiver 4 or the mobile phone. Transmit to the terminal 10.

Hereinafter, the receiver 4 will be described in detail with reference to FIG.
FIG. 4 is a block diagram illustrating a functional configuration of the receiver 4.
The receiver 4 includes a broadcast wave reception unit 11, a first separation unit 12, a broadcast AIT acquisition unit 13, a communication unit 14, a second separation unit 15, a communication AIT acquisition unit 16, and an application execution control unit 17. A voice control unit 18, a display control unit 19, a speaker 20, a display 21, a memory 22, an AIT storage unit 23, a device connection unit 24 as an external device connection unit, and an association unit 25.

  The broadcast wave receiving unit 11 receives a broadcast wave transmitted from the broadcast station 1 via the broadcast antenna 2.

  The first separator 12 extracts a broadcast signal, that is, a TS packet, by demodulating the broadcast wave received by the broadcast wave receiver 11. The first separation unit 12 refers to the PSI / SI of the TS packet, determines the type of data included in the TS packet, and separates various data such as video, audio, EIT, and program ID. Extract. Further, the first separation unit 12 separates and extracts the AIT by referring to the section and the PES according to a predetermined AIT transmission method.

  Subsequently, when the data included in the PES of the TS packet is audio data, the first separation unit 12 outputs the audio data to the audio control unit 18. In addition, when the data included in the PES of the TS packet is video data, the first separation unit 12 outputs the video data to the display control unit 19.

  In addition, the first separation unit 12 stores the extracted EIT, program ID, and other various data in the memory 22. Further, when the first separation unit 12 extracts the AIT, the first separation unit 12 outputs the extracted AIT to the broadcast AIT acquisition unit 13.

  The broadcast AIT acquisition unit 13 acquires the AIT output from the first separation unit 12 and stores it in the AIT storage unit 23.

  The communication unit 14 transmits / receives various information to / from the server group 3 via the communication network N, receives an operation signal from a remote controller R (hereinafter abbreviated as a remote controller R), Send and receive various information between them.

  The second separation unit 15 determines the type of data transmitted from the server group 3 via the communication network N and received by the communication unit 14, and separates the data output destination based on the determination result. For example, if the second separation unit 15 determines that the received data is an AIT, the second separation unit 15 outputs the AIT to the communication AIT acquisition unit 16. When the second separation unit 15 determines that the received data is an application, the second separation unit 15 outputs the application to the application execution control unit 17.

  Further, when the second separation unit 15 determines that the received data is a TS packet, the second separation unit 15 separates and extracts various data such as video, audio, AIT, and program ID from the TS packet. The second separation unit 15 outputs this AIT to the communication AIT acquisition unit 16 when the AIT is extracted, and outputs this data to the application execution control unit 17 when data other than the AIT is extracted.

  The communication AIT acquisition unit 16 acquires the AIT output from the second separation unit 15 and stores it in the AIT storage unit 23.

  The application execution control unit 17 acquires an application based on a user's execution instruction via the AIT or the remote controller R, and controls execution of the acquired application. For example, when an application that generates and outputs association information is activated, the application execution control unit 17 outputs a program ID stored in the memory 22 and a user ID that identifies the user to the association unit 25 by executing the application. To do.

The memory 22 stores meta information of program contents such as EIT and program ID and other various information.
The AIT storage unit 23 stores the AIT acquired by the broadcast AIT acquisition unit 13 and the communication AIT acquisition unit 16.

The device connection unit 24 connects devices such as the camera device 5 and the microphone device 6 and supplies input information from the devices to the association unit 25 as viewing state information.
Specifically, the device connection unit 24 uses the application executed by the application execution control unit 17 (hereinafter also referred to as “Hybridcast application”) to view the data of the image obtained as a result of the viewer taking a picture with the camera device 5. Obtained as state information and output to the associating unit 25. In addition, the device connection unit 24 acquires the audio data of the viewer input from the microphone device 6 as the viewing state information by the application executed by the application execution control unit 17, and outputs it to the association unit 25.
Note that the device connected to the device connection unit 24 is not limited to the camera device 5 and the microphone device 6. For example, a gyro sensor capable of acquiring viewer position information and an optical sensor capable of detecting viewer operation. It may be.
Furthermore, the connection to the device connection unit 24 does not only mean connecting to an external device but also means connecting to a built-in device. Accordingly, the camera device 5 and the microphone device 6 are not particularly required to be external devices, and may be built in the receiver 4.
Further details of the device connection unit 24 will be described later with reference to FIGS.

The associating unit 25 acquires NPT or PTS from the display control unit 19 as time information indicating the relative playback start timing of the predetermined scene in the program content. The associating unit 25 acquires image or audio data from the device connecting unit 24 as viewing state information indicating the viewing state of the viewer when viewing the predetermined scene. The associating unit 25 acquires ID information such as a program ID and a user ID from the memory 22.
The associating unit 25 generates associating information by associating the acquired information, that is, NPT or PTS, viewing state information as image or audio data, and ID information, via the communication unit 14. To the service server 3.
Further details of the associating unit 25 will be described later with reference to FIGS. 11 to 14.

  Further, the device connection unit 24 of the receiver 4 having such a functional configuration will be described below with reference to FIGS.

  As described above, devices such as the camera device 5 and the microphone device 6 are connected to the device connection unit 24. However, even if the device is simply physically connected, the device connection unit 24 cannot output information from the device as viewing state information to the association unit 25 unless the device is discovered and recognized. .

FIG. 6 is a diagram for explaining an operation related to discovery of such a device, and is a diagram illustrating a relationship between layers of the Hybridcast function.
As shown in FIG. 6, as a layer of the Hybridcast function, there are hardware, system software, a virtual machine, a Hybridcast basic library and an extension library, and a Hybridcast application in order from the lower layer to the upper layer.
In the present embodiment, the device discovery operation in the device connection unit 24 is performed by a device discovery library that is one of extended libraries (software programs).

FIG. 7 is a diagram illustrating the concept of the device connection unit 24.
As shown in FIG. 7, the Hybridcast application executed by the application execution control unit 17 communicates with a device manager provided in the device connection unit 24, an HID (Human Interface Device), and the like.
Here, the device manager implementation and the HID implementation in a general physical network layer are used in the present embodiment. For this reason, various listeners as shown in the sequence diagrams of FIGS. 8 to 10 are implemented, and an implementation including a device manager and an HID is generated as a class for generating and processing various events as shown in the sequence diagrams. By implementing the entertainment class, the device connection unit 24 is realized.

FIG. 8 shows a sequence diagram when a predetermined external device is connected as a device when a service of a Hybridcast application that uses the device for operation recognition or the like is started.
In step S <b> 1, the associating unit 25 acquires ID information such as a program ID and a user ID from the memory 22. However, acquisition of ID information is performed not only at the processing timing of step S1, but also at any time at the change of the program content from the broadcast station 1.

Next, the following steps S2 to S9 are executed as a sequence of device discovery parts by the device manager.
In step S2, the Hybridcast application (abbreviated as “app” in the description of the sequence diagrams of FIGS. 8 to 10) executed by the application execution control unit 17 registers a device manager instance in the device connection unit 24.
In step S <b> 3, the device manager notifies the association unit 25 of the application.
In step S4, the associating unit 25 notifies the device manager of the associating unit physical information. Here, the association unit physical information is information indicating where the viewer state information (image or sound) should be physically flowed.
In step S5, the application generates a device discovery listener. That is, a device discovery event is registered.
In step S6, the device manager attempts to find a device for the HID.
In step S7, the HID returns the response to the device manager.
In step S8, the device manager generates and registers a new device instance based on the device information obtained from the response in step S7 and the association unit physical information notified in step S4.
In step S9, the device manager notifies the app of a new device by sending device information to the app.

In this way, when the processing of steps S2 to S9 is executed as a device discovery part sequence by the device manager, the following steps S10 to S14 are subsequently executed as a description and control part sequence by the device instance. The process is executed.
In step S10, the application registers a device status listener for the device instance.
In step S <b> 11, the device instance confirms the device status (for example, the status of whether the power state is ON or OFF) with respect to the HID.
In step S12, the HID notifies the device instance of the status of the device, for example, the power supply status and device specific information. If the power state of the device is OFF, the process shown in FIG. 9 is further required. This process will be described later.
In step S13, the device instance notifies the app of the device status notified from the HID.
In step S <b> 14, the device instance sends the notification of the association unit 25 to the HID so as to execute control to output the input data of the device (data that is image or audio data and becomes viewer state information) to the association unit 25. To do.

Here, the description of the device instance and the sequence of the control part when the power state of the device is OFF will be described.
First, the process of step S11 is executed regardless of the power state of the device, but when the power state of the device is OFF, the process according to the sequence diagram of FIG. 9 is used instead of the process of step S12. Is executed.
FIG. 9 is a sequence diagram of the description by the device instance and the control part when the power state of the device is OFF.
The process of step S12-1 is substantially the same as step S12. However, in step S12-1, the device instance is notified that the power state is OFF.
In step S33, the device instance makes a request to the device manager to change the power state of the device to the ON state (hereinafter referred to as a device power ON request).
In step S34, the device manager changes the power state of the device to the ON state.
In step S12-2, the HID notifies the device instance of the device status, for example, the power supply status and device-specific information. In this case, the device instance is notified that the power state is ON.
Thereafter, the processes of steps S13 and S14 of FIG. 8 described above are executed.

In this way, when the process of steps S10 to S14 of FIG. 8 (the process of FIG. 9 when the power state of the device is OFF) is executed as the description of the device instance and the sequence of the control part, it continues next. The following steps S15 to S19 are executed.
In step S15, the application requests an input event from the device instance.
In step S16, the device instance requests an input event from the HID.
In step S <b> 17, the device, that is, the camera device 5 or the microphone device 6, which is an external device here, inputs image or audio data to the HID.
In step S18, the HID provides the association unit 25 with the image or audio data input in the process of step S17 as input data. The associating unit 25 acquires the input data as viewer state information.
In step S19, the display control unit 19 provides the association unit 25 with the NPT / PTS at the time of processing in step S18.
Thereafter, each time an input event is requested from the application, the processes of steps S15 to S19 are repeatedly executed.

In summary, the device manager functions in the device connection unit 24. The main function of this device manager is to discover an HID (device) and generate a device instance using the device information of the discovered device and the association unit information together.
The function of the device instance generated by the device manager in this way is to receive an input event from the application and to control the HID and the association unit 25.
The function of the associating unit 25 includes input data from the device (HID) (image and audio data and data serving as viewer state information), and internal information of the receiver 4 (time information such as NPT / PTS). Or ID information).

  By the way, the device connected to the device connection unit 24 is not particularly limited to the external device as described above, and may be a built-in device of the receiver 4.

FIG. 10 shows a sequence diagram when the built-in device of the receiver 4 is used as a device when a service of a Hybridcast application that uses the device for operation recognition or the like is started.
Comparing FIG. 10 and FIG. 8, steps S51 to S54 in FIG. 10 are the same processes as steps S1 to S4 in FIG.
However, in FIG. 10, there is no processing corresponding to steps S5 to S7 of FIG. This is because there is no need to discover the internal device. Therefore, in step S55 of FIG. 10, the device manager generates and registers a new device instance based on the known information of the built-in device and the association unit physical information notified in step S54.
In FIG. 10, there is no description of the device instance and the sequence of the control part. That is, in FIG. 10, there is no processing corresponding to steps S10 to S14 of FIG. 8 (FIG. 9 when the power supply state of the device is OFF). This is because a built-in device does not require a status listener. That is, the situation can be changed from the device manager.
Note that steps S56 to S60 in FIG. 10 are the same processes as steps S15 to S19 in FIG. That is, every time an input event is requested from the application, the processes of steps S56 to S60 are repeatedly executed.

  The details of the device connection unit 24 in the receiver 4 having the functional configuration of FIG. 4 have been described above with reference to FIGS. Next, details of the associating unit 25 in the receiver 4 will be described with reference to FIGS. 11 to 14.

FIG. 11 is a functional block diagram illustrating an example of a functional configuration of the associating unit 25.
The association unit 25 generates image association information using the image information acquisition unit 251, the relative time information acquisition unit 252, the ID (program / user ID) acquisition unit 253, and the image association unit 254.
The image information acquisition unit 251 views data (see step S18 in FIG. 8) of images (moving image frames or still images) output as input data from the device connection unit 24, that is, viewing when a predetermined scene of program content is viewed. Data of an image in which the viewer is shown is acquired as viewer state information.
The relative time information acquisition unit 252 acquires NPT or PTS (see step S19 in FIG. 8) as time information indicating the relative playback start timing of the predetermined scene output from the display control unit 19.
The ID (program / user ID) acquisition unit 253 acquires the program ID and the user ID (ID information in step S1 in FIG. 8) output from the application execution control unit 17.
The image association unit 254 includes the image data acquired by the image information acquisition unit 251, the NPT or PTS acquired by the relative time information acquisition unit 252, and the program acquired by the ID (program / user ID) acquisition unit 253. The association information is generated by associating the ID and the user ID. Hereinafter, the association information generated by the image association unit 254 is particularly referred to as image association information. The image association unit 254 outputs image association information to the communication unit 14 at any time until the application execution control unit 17 ends the application.

FIG. 12 is a flowchart for explaining the processing flow of the associating unit 25 in the example of FIG.
In step S101, the associating unit 25 acquires image (moving image frame or still image) data by the image information acquiring unit 251, acquires time information (NPT or PTS) by the relative time information acquiring unit 252, and acquires ID. The unit 253 acquires a program ID and a user ID.
In step S102, the image association unit 254 generates image association information by associating various information acquired in step S101, that is, image data, time information, a program ID, and a user ID.
In step S103, the image association unit 254 outputs the image association information generated in step S102 to the communication unit 14.
In step S104, the associating unit 25 determines whether or not the application execution control unit 17 has ended the application. If the application execution control unit 17 has not ended the process, the process returns to step S101.

The image association information output from the association unit 25 is transmitted to the service server 3 (see FIG. 4) via the communication unit 14.
The service server 3 performs image processing based on the received image association information. As a result, the service server 3 estimates information such as “in which scene the viewer laughed” and “in which scene he / she lost interest”, and the viewer can watch the viewer without performing a special action. The state can be estimated.

FIG. 13 is a functional block diagram illustrating another example of the functional configuration of the associating unit 25.
The associating unit 25 includes an audio information acquiring unit 255, a relative time information acquiring unit 252, an ID (program / user ID) acquiring unit 253, and an audio associating unit 256. Information).
The audio information acquisition unit 255 uses, as audio information, audio data output from the device connection unit 24 as input data (see step S18 in FIG. 8), that is, audio data generated when a predetermined scene of the program content is viewed. get.
The audio association unit 256 includes the audio information acquired by the audio information acquisition unit 255, the NPT or PTS acquired by the relative time information acquisition unit 252, and the program ID acquired by the ID (program / user ID) acquisition unit 253. And association information is generated by associating the user ID with the user ID. Hereinafter, the association information generated by the audio association unit 256 is particularly referred to as audio association information. The voice association unit 256 outputs the voice association information to the communication unit 14 as needed until the application execution control unit 17 ends the application.

FIG. 14 is a flowchart for explaining the processing flow of the associating unit 25 in the example of FIG.
In step S201, the associating unit 25 acquires audio information with the audio information acquiring unit 255, acquires time information (NPT or PTS) with the relative time information acquiring unit 252, and acquires a program ID and user ID with the ID acquiring unit 253. get.
In step S202, the audio association unit 256 generates audio association information by associating various information acquired in step S201, that is, audio information, time information, a program ID, and a user ID.
In step S <b> 203, the voice association unit 256 outputs the voice association information generated in step S <b> 202 to the communication unit 14.
In step S204, the associating unit 25 determines whether or not the application execution control unit 17 has ended the application. If the application execution control unit 17 has not ended, the process returns to step S201.

According to this embodiment, there are the following effects.
(1) In the device connection unit 24 of the receiver 4, a device instance is generated by the device manager based on information in the association unit 25 and information on devices (for example, the camera device 5 and the microphone device 6). Then, the device and the association unit 25 are controlled by the device instance, and when there is an input event request from the application, input information is acquired from the device and supplied to the association unit 25.
In this way, an appropriate mechanism for connecting the device to the receiver 4 to function in the system for coordinating broadcasting and communication is realized.

(2) The device manager further discovers a device as an external device.
Thereby, in a system for coordinating broadcasting and communication, an appropriate mechanism for connecting and functioning a receiver to a device as an external device is realized.

It should be noted that the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within a scope that can achieve the object of the present invention are included in the present invention.
In this embodiment, for example, a camera device and a microphone device are provided as devices. However, the present invention is not limited to this, and any camera device or microphone device can be provided as long as the viewer's viewing state information can be acquired. That's fine.

DESCRIPTION OF SYMBOLS 1 Broadcasting station 3 Service server 4 Receiver 17 Application execution part 24 Device connection part 25 Association part 14 Communication part N Network

Claims (3)

For applications that receive broadcast signals that are broadcast from broadcast stations and that are composed of multiple pieces of information multiplexed with program content, and that contain applications and video and audio data via communication from a server connected via a network A receiver capable of receiving content and linking the broadcast and the communication ,
Execution control means for controlling execution of an application managed by application management information multiplexed on the broadcast signal or transmitted via the network;
Time information indicating a relative reproduction start timing of the predetermined scene acquired from a display control unit that outputs the predetermined scene in the program content, and a viewing state of the viewer when the predetermined scene input from the device is viewed Association means for generating association information that associates the viewing state information indicating the program content ID that identifies the program content included in the broadcast signal ;
A device manager that generates a device instance based on information indicating where the viewer state information of the association unit should be physically flowed and device information;
After being generated by the device manager, the device and the associating means are controlled, and upon receiving an input event request from the application, data that is input data from the device and becomes the viewing state information is acquired. And a device instance that supplies the association means. The device is an external device connected to the receiver from the outside,
The device manager further discovers the device as the external device;
The receiver according to claim 1. For applications that receive broadcast signals that are broadcast from broadcast stations and that are composed of multiple pieces of information multiplexed with program content, and that contain applications and video and audio data via communication from a server connected via a network A receiver capable of receiving content and linking the broadcast and the communication, and is an application managed by application management information multiplexed on the broadcast signal or transmitted through the network A computer for controlling a receiver comprising execution control means for controlling execution;
Time information indicating a relative reproduction start timing of the predetermined scene acquired from a display control unit that outputs the predetermined scene in the program content, and a viewing state of the viewer when the predetermined scene input from the device is viewed Associating means for generating associating information associating viewing state information indicating a program ID identifying the program content included in the broadcast signal ;
A device manager that generates a device instance based on information indicating where the viewer state information of the association unit should be physically flowed and device information;
After being generated by the device manager, the device and the associating means are controlled, and upon receiving an input event request from the application, data that is input data from the device and becomes the viewing state information is acquired. A device instance to be supplied to the association means
Program to function as.

Images