JP4491902B2 - Interactive system with media contents using sliding graphic windows - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a media playback device and an interactive television device.
[0002]
[Prior art]
In recent years, a number of interactive television systems and protocols have been developed. OpenTV^TMInteractive television systems based on this specification have already gained millions of units in the market. Open TV^TMThen, an interactive logo appears on the screen, indicating that the currently viewed broadcast contains interactive content. When this logo is displayed, the viewer can display the interactive menu and scroll the interactive menu using the remote control device.
[0003]
Microsoft WebTV^RNAllows you to interact while you are on TV, and you can access e-mail web content on TV. Available types of interaction include playing at game shows, participating in poles, chatting with other viewers, etc. "i" to indicate that interactive content is included in the television broadcast A character icon is displayed on the TV screen.
[0004]
TiVo^TMAnd ReplayTV^RNIn other systems, such as set-top boxes made by, broadcast content can be captured and stored on a hard disk drive.
[0005]
The term “metadata” used in the description of the present invention means information and data related to media content. For example, information about a television program such as a program name is metadata about the program. In a video program, a series of time stamps attached to the beginning of a scene is metadata. A link display to local content or remote content such as the URL of the displayed web page is also metadata.
[0006]
The term “interactive element” used in the description of the present invention means an application related to human interaction via an interface device such as a remote control device, a keyboard or a mouse. Most interactive elements operate on metadata, media data, or both. The most basic interactive element is an application that allows the display of metadata such as the display of program information. Of the interactive elements that operate on metadata, the application for poles is an example belonging to a complex category. The application allows the user to vote for any issue related to media content. The provision of poles and the selection of candidates is metadata. An example of an interactive element that operates on media data is a link anchor for content such as related television programs that are media content. An application that starts playing the program when the button is pressed is an interactive element. An example of an interactive element that operates on both media data and metadata is an interactive time bar. Using this interactive time bar, the user can search for videos. That is, if the “next” button is pressed, it is possible to fast-forward to the next scene, and if the “previous” button is pressed, it is possible to fast-rewind to the previous scene. In the scene boundary list, the data is metadata, and the video on which it operates is media data.
[0007]
The term “semantic classification” used in the description of the present invention means a classification of interactive elements and metadata. In accordance with the present invention, all types of metadata and interactive elements are assigned a semantic classification. For example, prepare four semantic classifications in the system. Information about the current program, system menu and setup, related links, and additional information. Interactive elements and metadata relating to broadcast or stored media content are assigned a semantic classification and sometimes multiple semantic classifications.
[0008]
SUMMARY OF THE INVENTION
  The present inventiondisplayMedia contentdisplayA device for interacting with media content, metadata and interactive elements while. MeExamples of media content include television programs, movies, sports programs,etcCan be mentioned. Media content comes from a variety of different sources. For example, the media content may be broadcast, locally stored, or distributed over the Internet.
[0009]
  The system of the present inventionDisplay controller, Selection means andWindow generatorConsists of
[0010]
  The display controller controls the display to display media content.
[0012]
  Selection means, Which has an operation unit for selecting four directions of up, down, left and right, and in these four directions, detailed information regarding the currently displayed media content and a list of links to the content related to the currently displayed media content Each of the semantic classifications is divided into four categories: links, additional information on the currently displayed media content, and system menus. These four categories can be selected by selecting the direction using this operation unit. Select any one of the categories. A user interface having a remote control device, a mouse, a keyboard, a joystick, or a touch panel can be used as the selection means.
[0013]
  Window generatorIsMetadata and interactive elements that are associated with any media content and assigned at least one of four semantic categories according to the selected category by the selection means Generates a graphic window including metadata and interactive elements belonging to the selected category, and associates the four categories with one of the four sides of the display, and controls the display to control the display. Are slid from the side of the display corresponding to the selected category in the direction selected by the operation unit of the selection means and displayed over the media content.
[0018]
DETAILED DESCRIPTION OF THE INVENTION
A system according to an embodiment of the present invention will be described with reference to the accompanying drawings.
[0019]
First, an information providing entity related to this system will be described with reference to FIG. The media content provider 1 provides media content such as television programs, sports information, and weather forecasts to the broadcast station 2 through the media distribution channel 1b. The broadcast station 2 distributes the media content to the viewer system 3 via the broadcast distribution channel 2a. Examples of broadcast distribution channels include terrestrial, satellite broadcast, cable distribution, and other multiplexed television broadcasts in addition to distribution via the Internet.
[0020]
The additional information provider 4 provides additional information regarding the media content to the broadcasting station 2 via the distribution channel 4a in order to perform mixing with the media content provided by the broadcasting station 2. Examples of additional information include interactive elements such as text strings on TV programs or metadata such as advertisements, links for accessing text information on TV programs, and links to web pages related to TV programs. . In the present embodiment, XML content is expressed using XML, and Java is used.^TMRepresents interactive elements using a programming language. Metadata and interactive elements are described in further detail below.
[0021]
FIG. 2 shows an example of a broadcasting station 2 based on a digital terrestrial television broadcasting model by the International Telecommunications Union (ITU). This system includes a video subsystem 210, an audio subsystem 220, a service multiplexer / transport system 230, and an RF / transmission system 240.
[0022]
The video subsystem 210 receives a video signal provided from the media content provider 1 using the MPEG-2 encoding method or other methods defined in ISO / IEC IS 13818-1 and ISO / IEC IS 13818-2. Encode and compress. Similarly, the audio subsystem 220 encodes and compresses the audio signal provided by the media content provider 1 using an AC-3 audio encoding standard or other method.
[0023]
The service multiplexer / transport system 230 includes a service multiplexer 231 and a transport encoder 232, which combine various components and arrange for broadcasting. The service multiplexer 231 multiplexes the compressed video signal and audio signal, attached data, and control data into one data. The transport encoder 232 packetizes the signal using the MPEG-2 transport stream syntax.
[0024]
The RF / transmission system 240 includes a channel encoder 241, a modulator 242, and a transmitter 243. The transmitter 243 is for transmitting a packet signal from the service multiplexer / transport system 230. The channel encoder 241 transforms the signal to prevent errors. Modulator 242 modulates the digital signal onto the carrier frequency. The modulated signal is transmitted by the transmitter 243. The viewer system 3 receives a signal on the channel 2a, performs a process reverse to the encoding process, and outputs a digital audio signal and a digital video signal.
[0025]
FIG. 3 is a block diagram showing a hardware configuration of viewer system 3 according to the embodiment of the present invention. The viewer system 3 includes a set top box 320, a hard disk drive 301, a video display 302, a speaker 302a, and a remote control device 309 having a simple configuration.
[0026]
The set top box 320 includes a processor 321, an MPEG-2 decoder 322, a tuner 323, a graphic controller 324a, an audio processor 324b, an audio amplifier 324c, a hard disk controller 325, a network controller 326, a memory 327, and an infrared input / output (I / O) controller. 328. The processor 321 is connected to the MPEG-2 decoder 322, the tuner 323, and the graphic controller 324a via the graphic bus 329, and sends commands and data to the MPEG-2 decoder 322, the tuner 323, and the graphic controller 324a via the graphic bus 329. To do. The processor 321 is connected to the hard disk controller 325, the network controller 326, the memory 327, the audio processor 324b, and the infrared input / output controller 328 via the common bus 330, and reads and writes control commands and status commands via the common bus 330. These commands are sent to the hard disk controller 325, the network controller 326, the memory 327, the audio processor 324b, and the infrared input / output controller 328.
[0027]
The hard disk drive 301 is connected to the hard disk controller 325 by a bus 331. In this embodiment, the hard disk drive 301 is installed at a position physically separated from the set top box 320, but the hard disk drive 301 may be built in the set top box 320. The hard disk controller 325 controls the bus 331. The hard disk controller 325 can use an IDE or a SCSI bus controller.
[0028]
The network controller 326 is connected to a network 332 such as the Internet, sends data to the network 332, and receives data from the network 332. The network controller 326 also interprets control commands and status commands. The processor 321 and other modules use the memory 327 in order to temporarily store programs and data. The infrared input / output controller 328 receives a command from the remote control device 309 and stores the command in the memory 327 or directly sends it to the processor 321.
[0029]
The audio processor 324b is connected to the bus 330. The processor 321 sends a control request and a status request to the audio processor 324b so that data and commands are sent from the memory 327, the network controller 326, or the hard disk controller 325 to the audio processor 324b. The audio processor 324b is connected to the audio amplifier 324c via a cable 333b, and the audio amplifier 324c is connected to the speaker 320a via a cable 333c. The audio processor 324b controls the audio amplifier 324c and the speaker 320a to output sound such as radio broadcast, movie sound part, or television broadcast. Audio processor 324b and audio amplifier 324c may be implemented using a single physical device such as a computer sound card that functionally includes both an audio processor and an audio amplifier.
[0030]
The tuner 323 demodulates the television signal from the broadcasting station 2 sent from the channel 2a. The graphic controller 324a performs graphic command processing and executes functions such as video composition, drawing, and scanning line conversion. The graphic controller 324a is connected to the video display 302 via a video cable 333a. The MPEG-2 decoder 322 decodes the compressed video data from the hard disk drive 301 and the tuner 323.
[0031]
FIG. 4 is a front view of the remote control device 309. The remote control device 309 includes six navigation buttons, that is, an up button 310, a right button 311, a down button 312, a left button 313, a selection button 314, and a mode cycle button 315.
[0032]
Next, the operation of the system according to the present embodiment will be described with reference to the flowcharts shown in FIGS. The flowchart shown in FIG. 5 shows processing executed by the media content provider 1, the broadcast station 2, and the additional information provider 4. The flowchart shown in FIG. 6 shows processing executed by the viewer system 3.
[0033]
First, in S300, media content including video content and audio content is created by the media content provider 1. Also, individual identification data is given to the additional data for media content. The assignment of individual identification data, that is, a program ID, is a well-known process, and is a process required in digital broadcasting. For example, the Advanced Television System Committee (ATSC), which is a US digital television broadcasting system, has an agreement on standardization relating to program identification, and the contents thereof are described in the standard document A / 57.
[0034]
In S301, the additional information provider 4 creates metadata and interactive elements related to the broadcast media data as attached data. The digital broadcasting standard defines a standard way of representing certain types of metadata. For example, in the Advanced Television System Committee (ATSC), a standard document A / 65 defines a standard for encoding a program for terrestrial broadcasting and cable broadcasting and system information. Some of the standard interactive elements used in broadcasting are already widely used. For example, the Advanced Television Enhancement Forum (ATVEF) defines standards based on HTML 4.0, ECMA script and DOM. Other standards such as DVB-MHP and ATSC-DASE are JAVA^TMAlthough it is based on a programming language, it is not finished yet.
[0035]
In S301, the additional information provider 4 associates the metadata and the interactive element data with the program identification information of the program related to the metadata and the interactive element, and when the viewer system 3 combines the data, the correct program is displayed. So that you can get a correlation. Furthermore, in S301, the additional information provider 4 associates one metadata with each interactive element, and in some cases, with a plurality of semantic classifications. The semantic classification of each metadata or interactive element defines the final display format of the metadata or interactive element in the viewer system 3. An example of a method for associating metadata, interactive elements, and semantic classification will be described later.
[0036]
Next, in S302, the broadcasting station 2 multiplexes the media data and the attached data. The attached data includes a program identification information ID obtained from S300 and S301, metadata, and interactive elements. The broadcasting station 2 performs digital broadcasting using MPEG-2 as a transport stream. The service multiplexer / transport system 230 is an international standard for digital storage media command and control (DSM-CC) determined by ISO / IEC 13818-6, more specifically, a repetitive data transmission system (carousel protocol). ) Is used to perform this multiplexing process.
[0037]
In S303, the multiplexed signal is modulated on the other channel by the modulator 242, and the RF / transmission system 243 broadcasts the signal using a standard digital broadcasting protocol such as ATSC or DVB.
[0038]
In S <b> 304 of FIG. 6, the viewer system 3 receives a modulation signal from the broadcast station 2. The tuner 323 tunes the signal, that is, demodulates the signal to a single channel, and extracts a digital signal for one channel from the demodulated signal. This digital signal includes media data, metadata, and interactive elements.
[0039]
In S305, the MPEG-2 decoder 322 performs a demultiplexing process of the tuning signal using an existing TV application programming interface. During this demultiplexing process, the media data is separated from the metadata and interactive elements to generate a media content stream and a data content stream. The data content stream includes metadata and interactive elements.
[0040]
In S <b> 306, the processor 321 performs control processing so as to store each of metadata or interactive elements in a data content stream as a table in the memory 327 or as a part of a database local to the hard disk drive 301. The program identification data associated with the metadata and the interactive element in S301 is a key for performing access processing to the data executed in S310 described below.
[0041]
In S307, the processor 321 performs control processing for extracting the semantic classification from the metadata and the interactive element, and stores the extracted semantic classification in association with the metadata and the interactive element. This semantic classification is used as a second key for retrieving and reading the corresponding metadata and interactive elements in S310 described below.
[0042]
In S308, the graphic controller 324 and the processor 321 operate to display media content on the video display 302 based on the media content stream obtained in S305. In the content viewing mode, as a media content to be displayed on the video display 302, a soccer game 500 shown in FIG. 7 is taken as an example. The end user views this media content on the video display 302. Viewing the media content for the viewer is a central mode for interacting with the system, and viewing the media content is hereinafter referred to as a content viewing mode.
[0043]
In the content viewing mode, if metadata, interactive elements, or both of these are available, a message to that effect is displayed. An example of such a display is a graphic display. The graphic display is displayed overlaid on the media content display. The configuration required for such graphic display is disclosed in US Pat. No. 5,848,352. U.S. Pat. No. 5,848,352 is specified as a reference.
[0044]
In S309, an input for selecting one of the four semantic classifications is made from the remote control device 309, and the input is received. That is, to access metadata and interactive elements of various semantic classifications, the viewer needs to switch the system from a content viewing mode to a level 1 semantic classification mode. The viewer can use the remote control device 309 to perform this switching, that is, to select a semantic classification corresponding to the desired metadata and interactive elements. In this embodiment, there are four semantic classifications: program details, system menus, related links, and additional information. Note that not all of these semantic categories are assigned metadata and interactive elements associated with the currently displayed media content. For example, metadata and interactive elements irrelevant to the media content are assigned to the system menu classification.
[0045]
During the content viewing mode, each button 110-113 of the remote control device 309 is assigned one semantic classification. In other words, the detailed information related to the current program is assigned to the upper button 310, and the related link is assigned to the left button 313. Further, the right button 311 is assigned a classification of additional information, and the lower button 312 is assigned a classification of a system menu. The viewer can switch the system from content viewing mode to level 1 classification mode. To switch, press the button corresponding to the desired metadata and the semantic classification of the interactive element.
[0046]
In S310, if the selected semantic classification is an active classification and the selected semantic classification is related to the media content currently being displayed, the system switches from the content viewing mode to the level 1 classification mode. Can be switched to. Then, metadata and interactive elements that match both the current program identification data and the selected semantic classification are read out. Since the program identification data is embedded in the media content, the set top box 320 has information on the program identification data of the media content currently being viewed. If the selected semantic category is something like a system menu that includes only metadata or interactive elements that are not relevant to the currently displayed media content, the metadata that matches the selected semantic category And only interactive elements will be read.
[0047]
The read metadata and interactive elements are analyzed and displayed in a sliding window on the video display 302 corresponding to the selected semantic classification. If the selected semantic classification includes only metadata or interactive elements that are unrelated to the currently displayed media content, the corresponding window remains the same regardless of the displayed media content. Maintained in a state.
[0048]
In S310, the functions of the physical buttons 110 to 113 are reset by the system. That is, four logical buttons representing open, closed, forward, and reverse are set to function different physical buttons 110 to 113 depending on which semantic classification is selected. Each of the physical buttons 110 to 113 is represented by button identification data shown in FIG. The correspondence between logical buttons and physical buttons is set according to the currently selected semantic classification as shown in FIG.
[0049]
In S311 which is the last step, the user interacts with the metadata or the interactive element in a form corresponding to the specific interactive element.
[0050]
Here, a specific example of the operation executed in S308 to S311 will be described. As described above, in this embodiment, each semantic classification is associated with four sides of the video display 302. When a semantic category is selected, a window containing metadata and interactive elements belonging to that semantic category in the direction indicated by the arrow button on the remote control device 309 is displayed from the corresponding side of the video display 302. It is displayed slidingly on the inside.
[0051]
Detailed program classification
When the viewer is watching a soccer game in the content viewing mode shown in FIG. 7, if the viewer presses the up button 310 as shown in FIG. The mode is switched to the classification mode, and as shown in FIG. 11, the program detail window 510 is slid and displayed upward from the bottom of the screen of the video display 302. That is, the window 510 is displayed by sliding in the same direction as the arrow indicated by the up button 310. As shown in FIG. 12, the window 510 displays detailed information regarding the program currently being viewed. Detailed information includes a program name, a channel, a program broadcast date and time, and a program time length. The media content provider 1 can include detailed information regarding programs other than those described above using a standard markup language such as HTML or XML.
[0052]
Once the program details window is opened, the following assignments are made to the logical buttons based on the table shown in FIG.
Logical button              Physical button
Open button Up button 310
Close button Down button 312
Forward button Right button 311
Reverse button Left button 313
[0053]
As shown in FIG. 12, the three sides of the window 510 are surrounded by a boundary line 511, and the fourth side 512 is the lower side of the video display 302. There is no boundary line on this fourth side. The portion without the boundary line 511 indicates that the window 510 is further slid inward to display more information, that is, more metadata, interactive elements, or both. In this situation, when the user presses the up button 310 again as shown in FIG. 13, the display area of the window 510 is expanded and further information is displayed as shown in FIG. At this time, the viewer system 3 is in the level 2 classification mode. As shown in FIG. 15, a state in which boundary lines 511 appear on all four sides of the window 510 including the side 512 indicates that the window 510 is in a fully open state. There are no more metadata or interactive elements in this semantic classification, and the window 510 does not expand any further.
[0054]
FIGS. 16 to 20 show examples of interactive elements displayed in the fully opened window 510. To interact with the media content provider 1, a return channel and a network protocol are required. The network protocol is preferably the HTTP network protocol. FIG. 16 shows an initial screen displayed when the window 510 is fully opened. As an example of the interactive element, two buttons 513 and 514 are displayed on the screen, and the user can participate in a soccer game pole using the buttons displayed on these screens. Interactive elements can use logic forward, logic backward and logic selection buttons. That is, the viewer can change the input target by using the logical forward button assigned to the right button 311 that is a physical button and the logical backward button assigned to the left button 313 that is a physical button. . By pressing the right button 311 as shown in FIG. 17, the input target can be changed to the screen display button 514 as shown in FIG. As shown in FIG. 19, when a selection button 314 is pressed, an interaction can be executed and an action event can be sent to the interactive application. As shown in FIG. 20, the display of the window 510 is controlled accordingly.
[0055]
The type of interaction possible when selecting an item depends on the design determined by the provider of the metadata and interactive elements. For example, when a certain item is selected, a corresponding window can be slid and displayed to provide a wider display area, and a system design can be designed to execute the corresponding interactive application in the window. If additional information can be obtained as a result of the interaction, the classification window 510 may be further expanded by the viewer pressing the open button 310. The application is HTML, Java script^TM, Java^TMOr any other language supported by the hardware platform.
[0056]
As shown in FIG. 21, when the viewer presses the down button 312, the window 510 slides and becomes invisible, and the viewer system 3 returns to the content viewing mode as shown in FIG.
[0057]
Related link classification
In the state of the content viewing mode shown in FIG. 6, the viewer can select the related link classification by pressing the left button 313 on the remote control device as shown in FIG. As a result, as shown in FIG. 24, the related link window 520 slides and expands from the right side of the video display 302. While the related link window 520 is open, the following physical buttons are assigned to the logical buttons.
Logical button              Physical button
Open button Left button 313
Close button Right button 311
Forward button Down button 312
Reverse button Up button 310
[0058]
In the example illustrated in FIG. 25, the related link window 520 includes a plurality of link anchors 521 to 525 related to the currently displayed media content. In the example shown in FIG. 25, an anchor 521 of “Japan Team” is an input target. As shown in FIG. 26, when the viewer presses the selection button 314 to select the link anchor 521, the viewer moves to the link destination. Since the link anchor 521 is a link to a World Wide Web page, the web browser application is launched and the video display 302 is controlled to display the homepage 526 of the Japanese soccer team as shown in FIG. At this time, when the viewer presses the right button 311 as shown in FIG. 28, the related link window 520 is closed as shown in FIG. 29, and the cursor control is under the control of the web browser. This state is called a web browser mode. When the mode cycle button 315 on the remote control device 309 is pressed, the web browser mode can be exited. As a result, the viewer system 3 returns to the content viewing mode shown in FIG.
[0059]
While the associated link window 520 is open, navigating to the link anchors and other interactive elements 521-525 in the associated link window 520 by pressing the forward button 310 and the backward button 312 on the remote control device 309. Can do. For example, when the input object is at the anchor of “Japan Team” as shown in FIG. 25, by pressing the down button 312 three times as shown in FIG. It is possible to move to an interactive element 524 labeled “Record related event”.
[0060]
In this state, the viewer can use either the open button 313 or the select button 314 shown in FIG. 32 to expand the related link window 520 as shown in FIG. As shown in FIG. 34, the related link window 520 in the fully opened state has three selection portions 524a to 524c. The viewer can navigate to a desired item by executing the selection button 314 using the forward button 310 or the backward button 312. Also, the recording operation can be canceled by pressing the close button 311.
[0061]
Examples of links displayed in the related link window 520 include links to web pages, links to other television programs and radio programs, links to locally recorded content, and the like. Link information is preferably provided with media content using ATVEF enhanced television specifications, or via the Internet or other network. The additional information provider 4 can optionally include a link relating to the program when the program is distributed.
[0062]
A description of the link is made using a standard material identifier (URI). For example, a link to television content is described using the URI of the television station. The basic composition of the URI of a TV station is
tv: <broadcast>
And <broadcast> indicates a data source, and this data source can use an identifier as used in the domain name system (DNS). The identifier indicates a specific broadcasting station or television network. For example,
tv: wqed.org for the WQED network
tv: nbc.com for the NBC network
[0063]
These broadcast station identifiers are not the same as Internet addresses, and therefore do not represent IP addresses, but represent frequencies and broadcast protocols. The set-top box 320 includes a lookup table, in which a list of identifiers of all broadcast stations that can be received by the set-top box 320, and frequencies and broadcasting protocols corresponding thereto are written. An example of a television URI protocol that has been developed is a standard material identifier for television broadcasting.
[0064]
Future TV broadcasts are expected to be able to identify programs by adding start time and broadcast time information to the television URI. For example, in order to record a 30 minute program on the Fox network at 8 pm (local time) on March 10, 2001, the system for viewers of 2001, the tv url format is expanded as follows.
tv: fox.com/date=2001-0310/time=20: 00: 00 / duration = 0: 30: 00
[0065]
When the user enters a link to a program that has not yet been broadcast, a message is displayed to the user that the program has not yet been broadcast.
[0066]
If there is no metadata or interactive element for the media content currently displayed, the related link window 520 is not displayed, or the related link window 520 including the default link is displayed. The default link may be a link to the hardware manufacturer's web page, a link to the current media broadcaster's web page, or a link to the advertiser.
[0067]
System menu classification
In the content viewing mode of FIG. 6, the user can select the system menu category by pressing the down button 312 of the remote control device 309 as shown in FIG. As a result, as shown in FIG. 36, the system menu window 530 is slid downward from the upper side of the video display 302. While the system menu window 530 is open, the following physical buttons are assigned to the logical buttons.
Logical button              Physical button
Open button Down button 312
Close button Up button 310
Forward button Right button 311
Reverse button Left button 313
[0068]
The system menu window 530 makes it possible to set a system to meet the user's preference, for example, to start an application, and to browse content stored on the own device side. In the example shown in FIG. 37, the system menu window 530 includes options 531 to 535. As in the previous example, use the forward and reverse buttons to move between the listed options.
[0069]
Additional information classification
In the content viewing mode shown in FIG. 6, the user can select the additional information classification by pressing the right button 311 as shown in FIG. As a result, as shown in FIG. 39, the additional information window 540 is slid and displayed on the screen from the left side of the video display 302. While the additional information window 540 is open, the following physical buttons are assigned to the logical buttons.
Logical button              Physical button
Open button Right button 311
Close button Left button 313
Forward button Down button 312
Reverse button Up button 310
[0070]
In the example shown in FIG. 40, the additional information window 540 includes selectable items 541 to 546. The user can move between the items displayed here using the forward button and the reverse button of the logical buttons, and can select using the selection button 314.
[0071]
Here, the process in S301 for associating metadata and interactive elements with semantic classification will be described in more detail. Each metadata and interactive element includes a tag. This tag associates metadata or interactive elements with one of the semantic classifications. Such tags can be realized by encoding and distributing metadata and interactive elements as an extensible markup language (XML) document. XML is well known and is defined in a specification recommended by the World Wide Web Consortium (W3C).
[0072]
FIG. 41 conceptually shows the overall structure of an XML-encoded data stream 406 used to associate program identification information with metadata and interactive elements. An XML encoded data stream 406 associates semantic classification with metadata and interactive elements. Data stream 406 may include metadata and the interactive element itself.
[0073]
The data stream 406 encoded with XML includes fields 401 to 405. A field 401 includes a reference to program identification information. Fields 402-405 contain metadata and interactive elements for each semantic classification defined by the system. As an example, four semantic classifications are defined, and integers from 1 to 4 are assigned. The four semantic categories here are the same as the four semantic categories of program details, related links, system menus, and additional information described above.
[0074]
A data stream 406 encoded in XML includes a link to the cascade style sheet 400. This cascade style sheet 400 is stored in both the broadcasting station 2 and the viewer system 3. Cascading style sheet 400 includes rules that describe how to display metadata and interactive elements. For example, the cascading style sheet 400 takes into account the specifications of font, font size, color and other attributes. This type of cascading style sheet is well known and is defined in a specification recommended by the World Wide Web Consortium (W3C).
[0075]
FIG. 42 illustrates the data stream 406 encoded in XML in more detail. Line 00 describes a reference to a cascading style sheet named “default.css”. The style sheet may be stored locally, or may be broadcast via the data carousel (DSM-CC) as described above. The format for linking cascade style sheets to XML documents is well known.
[0076]
Line 02 includes a program identifier. This program identifier is used to associate all information in the XML stream with the broadcast program.
[0077]
Lines 04 and 05 contain data for semantic classification 1. In this example, semantic category 1 does not include metadata or interactive elements.
[0078]
Lines 07 to 12 contain data for semantic classification 2. Lines 08 and 09 are descriptions relating to links to web contents.
[0079]
Lines 14 to 21 contain data for semantic classification 3. Title attributes representing the titles of television programs are assigned to 15 lines. Other attributes of the television program are assigned to lines 16-18. Lines 19 and 20 show one method for defining interactive elements. Applet tags are classified as semantic^TMIndicates that an applet should be included. In this example, the applet code (ie compiled bytecode) is referenced as follows:
lid: //12abc554c3d3dd3f12abc554c3d3dd3f/code/timebar.jar
[0080]
This string is a local identifier, Java for executing interactive elements in the local file system^TMIndicates the storage location of the archive file. Java, also called a jar file^TMAn archive file is a single file, and several Java^TMA class file and related information files are bundled into a single archive file. Jar files usually contain class files and ancillary resources associated with applets and applications. The format of the local identifier is well known and is defined in a standard document indicated by the Advanced Television Enhancement Forum (ATVEF).
[0081]
When the user selects an interactive element in the jar archive, Java^TMThe virtual machine opens the contents of the jar archive and authenticates. Any Java in the manifest in the jar archive file^TMDescribes whether a class file should be executed. Java to execute^TMClass files are described as main class attributes. Java^TMExecution engine is specified Java^TMCalls the main function of the class file. Java^TMClass execution is well known by those skilled in the art, including those contained in jar files.
[0082]
Although the present invention has been described above with reference to specific embodiments, it goes without saying that various changes and modifications can be made within the scope of the technical idea of the present invention.
[0083]
For example, in the above embodiment, the case of broadcasting media content has been described, but the media content may be stored locally or distributed through the Internet.
[0084]
In the above embodiment, the media content provider 1, the broadcast station 2, and the additional information provider 4 are all described as independent information providing entities, but may be included in a single organization. Further, the functions of each of the media content provider 1, the broadcasting station 2, and the additional information provider 4 may be realized in a state where several different information providing entities are merged. For example, the media content provider 1 and the broadcast station 2 may be positioned as the same organization, and only the additional information provider 4 may be a separate organization.
[0085]
In the above-described embodiment, the additional information provider 4 embeds the additional information in the same MPEG-2 stream as the media content, thereby providing the additional information simultaneously with the media content and through the same distribution channel as the media content. However, as shown in FIG. 43, the additional information provider 4 ′ may distribute the additional information through a channel 4 ′ different from the distribution channel of the media content. For example, in the case of a television program on which media content is broadcast, another organization such as a “TV program commentary” service may distribute additional information regarding the content of the broadcast television program via the Internet. Such a method is a particularly useful method when handling media content stored in the local storage device of the viewer system 3.
[0086]
For simplicity, in the above embodiment, metadata and interactive elements are assigned to the entire program. However, in the present invention, the metadata and interactive elements are related to only a part of the program. It is also applicable to such cases. In such a case, the metadata and the interactive element include information on the start time of a part of the program to which the metadata and the interactive element are applied and the part of the broadcast time length in the metadata and the interactive element. Need to be.
[0087]
In the above embodiment, the remote control device 309 is used as an example of an interface for the user to select semantic classification. However, instead of the remote control device 309, other types of people such as a keyboard and a mouse are used. An interface connecting the machine and the machine may be used.
[0088]
In the above embodiment, the interactive element is Java.^TMIt was expressed using a programming language, but JavaScript^TMYou may make it describe using an interpreter language like this.
[0089]
In the embodiment according to the present invention, the case where the present invention is applied to media content that is output visually and acoustically has been described. However, the present invention can also be applied to media content that is output only visually or only acoustically. is there.
[0090]
In the above embodiment, an example in which metadata and interactive elements belonging to different semantic classifications are slid up and down and left and right from different sides of the video display has been shown, but metadata belonging to different semantic classifications and The interactive element may be displayed in any form as long as the user can distinguish between different semantic classifications. For example, metadata and interactive elements belonging to different semantic classifications may be displayed with different colors, or may be output with different sounds by using different voices.
[0091]
S306 and S37 in the flowchart may be a single step. In addition, the media content may have a hierarchical structure, and the remote structure may be used to navigate the hierarchical structure of the media content. A simple example of the hierarchical structure of media content is just the order in which programs are usually shown. In this case, the program can be fast forwarded and rewinded using the remote control device or skipped to another scene in the program. A more complex example of a media content hierarchy is the branching of games and dramas, which allows the user to enjoy a variety of different results for the same story. In this case, the selection of which branch to proceed next is performed using the remote control device.
[0092]
【The invention's effect】
  According to the configuration of the present invention, the user canBy selecting one of the four directions (up / down / left / right) with the operation unit of the selection means, the selection unit is divided into four categoriesSemantic classificationClassification of any one ofIf you selectClassificationAll metadata and interactive elements assigned toGraphic window containingTheFrom the side corresponding to the selected category among the four sides of the display, the image is slid in the direction selected by the operation unit and displayed over the media content.be able to.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a relationship between entities related to production, transmission, and reception of media content according to an embodiment of the present invention.
FIG. 2 is a block diagram showing details of the broadcasting station shown in FIG.
FIG. 3 is a block diagram showing details of the viewer shown in FIG. 1;
4 is a front view showing details of the remote control device shown in FIG. 3. FIG.
FIG. 5 is a flowchart illustrating a process for preparing and broadcasting media content, metadata, and interactive elements.
FIG. 6 is a flowchart showing processing performed in the viewer system.
FIG. 7 shows an example of media content displayed on a display of a viewer system.
8 is a table showing button identification numbers assigned to physical buttons of the remote control device shown in FIG.
FIG. 9 is a table showing the correspondence between logical buttons and physical buttons depending on the selected semantic classification.
FIG. 10 is a diagram showing a state where a button on the remote control device is pressed to select a program detailed semantic classification.
FIG. 11 is a diagram showing a program details window displayed on the display when the program detailed semantic classification is selected.
FIG. 12 is an enlarged view of a program details window.
FIG. 13 is a diagram showing a state in which a button on the remote control device is pressed to display the entire program details window.
FIG. 14 is a diagram showing a state in which the entire program details window is displayed on the display.
FIG. 15 is a diagram showing details of a program detail window displayed as a whole.
FIG. 16 is a diagram showing details of an interactive game displayed in a program details window displayed as a whole.
17 is a view showing a state where a button on the remote control device is pressed in order to change an input candidate in the window display of the interactive game shown in FIG.
FIG. 18 is a diagram showing an input target indicated by a window moved to a candidate selected by the operation shown in FIG. 17;
FIG. 19 is a diagram showing a state in which a button on the remote control device is pressed to select a candidate that is an input target.
FIG. 20 is a diagram showing an operation executed when a candidate is selected.
FIG. 21 is a diagram showing a state where a button on the remote control device is pressed to close the program details window.
FIG. 22 shows the display screen after the program details window is closed.
FIG. 23 shows a state where a button on a remote control device is pressed to select a related link semantic classification.
FIG. 24 is a diagram showing a related link window displayed on a display when a related link semantic classification is selected.
25 is an enlarged view of the related link window shown in FIG. 24. FIG.
FIG. 26 is a diagram showing a state in which a button on the remote control device is pressed to select a “Japan Team” link in the related link window.
FIG. 27 is a diagram showing a state in which the homepage of a Japanese soccer team is displayed by moving to the “Japan Team” link by the operation of FIG. 26;
FIG. 28 is a diagram showing a state where a button on a remote control device button is pressed to close a related link window.
FIG. 29 is a diagram showing a display state after closing a related link window and transferring display control to a web browsing application.
FIG. 30 is a diagram showing a state where a button on the remote control device is pressed to move an item in the related link window.
FIG. 31 is a diagram showing a state in which the input target has moved to the item “record related event” by pressing the button shown in FIG. 30;
FIG. 32 is a diagram showing a state where a button on the remote control device is pressed to select an item “record related event”.
FIG. 33 is a diagram in which the display state of the related link window is expanded to show an item in the “record related event” item.
FIG. 34 is an enlarged view when a related link window is opened as shown in FIG. 33;
FIG. 35 shows a button pressed on the remote control device to select a system menu semantic category.
FIG. 36 is a diagram showing a system menu window displayed on the display when system menu semantic classification is selected.
FIG. 37 is an enlarged view of a system menu window.
FIG. 38 shows a state where a button on the remote control device is pressed to select additional information semantic classification.
FIG. 39 is a diagram showing an additional information window displayed on the display when the additional information semantic classification is selected.
FIG. 40 is an enlarged view of an additional information window.
FIG. 41 is a diagram showing an overall structure of an XML encoded data stream used for associating program identification information with metadata and an interactive element.
FIG. 42 is a diagram showing a specific example of an XML encoded data stream.
FIG. 43 is a related diagram of the main organization when the additional information provider is an independent organization.
[Explanation of symbols]
1 Media content provider
2 broadcasting stations
230 Service Multiplexer / Transport System
240 RF / Transmission System
3 Viewer system
301 hard disk drive
302 Display
309 Remote control device
320 set top box
4 Additional information providers

Claims (12)

A display controller for controlling the display to display media content;
There are operation sections for selecting four directions (up, down, left, and right). In the four directions, detailed information regarding the media content currently displayed, and a related link that is a list of links to the content related to the media content currently displayed Each of the four semantic categories of additional information regarding the currently displayed media content and the system menu for starting the application is assigned, and by selecting the direction with the operation unit, Selecting means for selecting any one of the four classifications;
Metadata associated with each media content and assigned with at least one of the four semantic categories according to the category selected by the selection means; A graphic window including metadata and interactive elements belonging to the selected category is generated from the interactive elements, and the four categories are associated with each of the four sides of the display to control the display. And a window generator for displaying the graphic window over the media content by sliding the graphic window from the side of the display corresponding to the selected category in the direction selected by the operation unit of the selection unit; With
An interactive system for media content using a sliding graphic window. The interactive system according to claim 1, further comprising a storage device for storing the media content. A television receiver for receiving a television signal including the media content from a broadcasting station;
The interactive system according to claim 1, further comprising a media content extraction device for extracting the media content from the television signal. The television signal further includes the metadata and the interactive element,
The media content extraction device interactive system according to claim 3, wherein you further extracting the interactive element and the metadata from the television signal. The interactive system according to claim 3, further comprising a metadata / interactive element receiving device that receives the metadata and the interactive element through a distribution channel different from a broadcasting station that transmits the television signal. Interactive system according to any one of claims 1 to 5 that displays a graph hook window containing the interactive element and the metadata belonging to the class of only one of said window generator the four categories at a time. In the state where the graphic window is displayed, the selection means is set with a function of moving between metadata or interactive elements in the graphic window and a function of selecting the moved metadata or interactive element. The window generator expands and displays the graphic window so as to further include a selection for the selected metadata or interactive element in response to the selection of the metadata or interactive element by the selection means. The interactive system according to any one of claims 1 to 6 . The window generator
When all the metadata and interactive elements belonging to the classification selected by the selection means are in a fully open state displayed in the graphic window, the graphic window is surrounded by a boundary line and displayed.
Meta data belonging to the selected classified by the selecting means, in the case where both of the interactive elements or metadata and interactive elements there are more along the sides of the display corresponding to the selected classification of the graphene hook window portion interactive system according to any one of claims 1 to 7 was not to display borders. The interactive means according to any one of claims 1 to 8 , further comprising browsing means for moving to a link destination of the related link and controlling the display to display content related to the currently displayed media content. system. It further comprises notification means for graphically displaying the presence of metadata, interactive elements, or both metadata and interactive elements in one or more semantic classifications on the media content displayed on the display . The interactive system as described in any one of Claims 1 thru | or 9 . Interactive system according to any one of claims 1 to 10 further comprising a network means, for receiving the interactive element and the metadata through the network means for connecting to the network. Interactive system according to claim 11, wherein that will receive the media content through the network means.

Images