JP2014510425A - Method and system for providing additional content related to displayed content - Google Patents

Abstract

  The present disclosure relates to a method and system for providing content related to a displayed program. The methods and systems of the present disclosure include a second screen application of a second screen device, such as a tablet (eg, iPad), that provides a rich content experience with the displayed main content (eg, a movie from a Blu-ray disc). . The second screen displays deep metadata, alternative scenes, other accompanying video material, or other relevant information synchronized to the main display program without compromising the user's program viewing experience.

Description

  The present invention generally relates to providing additional content related to displayed content.

Conventionally, a displayed movie or program (such as supplemental material for a DVD or Blu-ray disc) must be viewed separately from the main movie or program. That is, the user must stop or interrupt playback of a major movie or program to access supplemental material. In many cases, there is no correlation between supplemental material and major movies and programs. In other cases, the picture-in-picture format is used to display supplemental materials, but it interferes with the display and viewing of major movies and programs.
[Cross-reference to related applications]
This application claims the benefit of US provisional application 61 / 460,373 filed December 30, 2010 and US provisional application 61 / 450,328 filed March 8, 2011. Yes, the above references are incorporated herein by reference in their entirety.

  The present disclosure relates to a method and system for providing content related to a displayed program. The methods and systems of the present disclosure include a second screen application of a second screen device, such as a tablet (eg, iPad), that provides a rich content experience with the displayed main content (eg, a movie from a Blu-ray disc). . The second screen displays deep metadata, alternative scenes, other accompanying video material, or other relevant information synchronized to the main display program without compromising the user's program viewing experience.

  In one embodiment, a method for displaying additional content related to the main content being viewed is disclosed. The method includes the steps of: displaying main content on a first screen device; providing a synchronization mechanism for synchronizing additional content in relation to displaying the main content on the first screen; Displaying on the two-screen device additional content related to the main content on the first screen synchronized with the content displayed on the first screen device by the synchronization mechanism.

  In another embodiment, a system for displaying additional content related to the main content being viewed is disclosed. The system includes a first screen device and a second screen device. The first screen device is configured to display main content. The second screen device is configured to display additional content related to the main content on the first screen synchronized with the content displayed on the first screen device by the synchronization mechanism.

1 is a system diagram illustrating an overview of video and audio content distribution to a home, according to one embodiment. FIG. FIG. 2 is a system diagram showing a representative set-top box in more detail. It is a figure which shows the touchscreen control apparatus by one Embodiment. It is a figure which shows the example of user interaction used with the touchscreen control apparatus by one Embodiment. 1 is a system diagram illustrating an embodiment of a system that implements the method of the present invention according to an embodiment. FIG. FIG. 5 is a flow diagram illustrating an example process according to one embodiment. FIG. 3 is a diagram illustrating an example of a method for synchronization between devices according to an embodiment. FIG. 3 is a diagram illustrating an example of a method for synchronization between devices according to an embodiment. It is a figure which shows the example of a skeletal screen which shows the characteristic by one Embodiment when used by passive mode. It is a figure which shows the example of a skeletal screen which shows the characteristic by one Embodiment when used by passive mode. It is a figure which shows the example of a skeletal screen which shows the function by one Embodiment when used by active mode. It is a figure which shows the example of a skeletal screen which shows the function by one Embodiment when used by active mode. It is a figure which shows the example of a skeletal screen which shows the function by one Embodiment when used by active mode. It is a figure which shows the example of a skeletal screen which shows the function by one Embodiment when used by active mode. It is a figure which shows the example of a skeletal view which shows the social media sharing function by one Embodiment. It is a figure which shows the example of a skeletal view which shows the social media sharing function by one Embodiment. It is a figure which shows the example of a skeletal view which shows the social media sharing function by one Embodiment. It is a figure which shows the example of a skeletal view which shows the content selection function by one Embodiment. It is a figure which shows the example of a skeletal view which shows the content selection function by one Embodiment. It is a figure which shows the example of a skeletal view which shows the additional function by one Embodiment. It is a figure which shows the example of a skeleton view which shows the additional function by one Embodiment. It is a figure which shows the example of a skeletal view which shows the additional function by one Embodiment. It is a figure which shows the example of a skeletal view which shows the additional function by one Embodiment. It is a figure which shows the example of a skeletal view which shows the additional function by one Embodiment. It is a figure which shows the example of the screen view which carried out the skin which shows how a certain function looks to a user. It is a figure which shows the example of the screen view which carried out the skin which shows how a certain function looks to a user. It is a figure which shows the example of the screen view which carried out the skin which shows how a certain function looks to a user. It is a figure which shows the example of the screen view which carried out the skin which shows how a certain function looks to a user. It is a figure which shows the example of the screen view which carried out the skin which shows how a certain function looks to a user. It is a figure which shows the example of the screen view which carried out the skin which shows how a certain function looks to a user. It is a figure which shows the example of the screen view which carried out the skin which shows how a certain function looks to a user. It is a figure which shows the example of the screen view which carried out the skin which shows how a certain function looks to a user. It is a figure which shows the example of the screen view which carried out the skin which shows how a certain function looks to a user. It is a figure which shows the example of the screen view which carried out the skin which shows how a certain function looks to a user. It is a figure which shows the example of the screen view which carried out the skin which shows how a certain function looks to a user. It is a figure which shows the example of the screen view which carried out the skin which shows how a certain function looks to a user.

  Referring now to FIG. 1, a block diagram of one embodiment of a system 100 for delivering content to a home or end user is shown. Content originates from content sources 102 such as movie studios and production houses. Content can be provided in at least one of two forms. One format is broadcast content. The broadcast content is supplied to the broadcast affiliate manager 104. The broadcast affiliate manager 104 is generally a national broadcasting service such as ABC (American Broadcasting Company), NBC (National Broadcasting Company), and CBS (Columbia Broadcasting System). The broadcast affiliate manager collects and stores the content and schedules delivery of the content via the delivery network shown as delivery network 1 (106). Delivery network 1 (106) includes satellite link transmissions from a national center to one or more regional or local centers. Delivery network 1 (106) may also include local content delivery using a local delivery system, such as via air, satellite, or cable broadcast. The locally delivered content is supplied to the receiving device 108 at the user's home. Here, the content is searched by the user. Of course, the receiving device 108 can take many forms and can be implemented as a set-top box / digital video recorder (DVR), gateway, modem, etc. Further, the receiving device 108 can function as an entry point to the home network system, that is, a gateway. The home network system includes another device configured as either a client device or a peer device.

  The second type of content is called special or additional content. Special or additional content includes content such as movies, video games, and other video elements delivered as premium viewing, pay-per-view, or other content that is not provided to the broadcast affiliate manager. In many cases, the special content is the content requested by the user. Special content is delivered to the content manager 110. The content manager 110 may be a service provider such as an Internet website affiliated with a content provider, a broadcast service, a delivery network service, or the like. The content manager 110 may also incorporate Internet content into the delivery system. The content manager 110 delivers the content to the user receiving device 108 via the delivery network 2 (112), which is another delivery network. The delivery network 2 (112) may include a high-speed broadband Internet type communication system. Content from the broadcast affiliate manager 104 may be delivered using all or part of the delivery network 2 (112), and content from the content manager 110 may be delivered using all or part of the delivery network 1 (106). It is important to note that this may be done. Further, the user may obtain the content directly from the Internet via the delivery network 2 (112), and the content manager 110 does not need to manage the content.

  Multiple applications for using additional content delivered separately are also possible. In one approach, the additional content is provided as an enhancement of the broadcasted content, providing alternative displays, purchasing and merchandising options, enhancement material, and the like. In another embodiment, the additional content may completely replace programming content provided as broadcast content. Finally, the additional content may be completely separate from the broadcast content, or simply alternative media that can be selected and used by the user. For example, the additional content may be a library of movies that are not available as broadcast content.

  The receiving device 108 can receive different types of content from one or both of the delivery network 1 and the delivery network 2. Receiving device 108 processes the content and provides content separation based on user preferences and commands. The receiving device 108 includes a storage device such as a hard disk drive or an optical disk drive that records and reproduces audio and video content. Further details of the operation of the receiving device 108 and functions related to the playback of stored content will be described below with reference to FIG. The processed content is provided to the display device 114. The display device 114 may be a conventional two-dimensional type display or an advanced three-dimensional display.

  The receiving device 108 may interface with a second screen such as the touch screen control device 116. Touch screen controller 116 is configured to provide user control over receiver device 108 and / or display device 114. The touch screen device 116 can also display video content. The video content may be a graphics input, such as a user interface input, or may be part of the video content delivered to the display device 114. The touch screen controller 116 can interface with the receiver 108 using any well-known transmission system such as infrared (IR) or radio frequency (RF) communication, and can be an infrared data association (IRDA) standard, WiFi, Bluetooth, or other standard. Protocols and other proprietary protocols can be included. In some embodiments, touch screen controller 116 may interface directly with delivery networks 1 and 2. The operation of the touch screen controller 116 will be described in more detail below.

  In the example of FIG. 1, the system 100 also includes a backend server 118 and a usage database 120. The backend server 118 includes a personalization engine that analyzes user usage habits and makes recommendations based on the usage habits. The usage database 120 stores user usage weeks. In some cases, usage database 120 may be part of backend server 118a. In this embodiment, the back-end server 118 (and the usage database 120) is connected to the system 100 and accessed via the delivery network 2 (112).

  Referring now to FIG. 2, a block diagram illustrating a receiving device 200 according to one embodiment. Receiving device 200 operates similarly to the receiving device shown in FIG. 1 and may be included as part of a gateway device, modem, set-top box, or other similar communication device. Further, the illustrated apparatus 200 may be incorporated in another system including an audio apparatus and a display apparatus. In any case, for the sake of clarity, some components necessary for complete operation of the system are not shown because they are well known to those skilled in the art.

  In the apparatus 200 shown in FIG. 2, the content is received by the input signal receiver 202. The input signal receiver 202 is a known receiving circuit used for receiving, demodulating and decoding signals supplied via networks such as air, cable, satellite, Ethernet, optical fiber and telephone line. . A desired input signal is selected and read by the input signal receiver 202 based on a user input supplied by the control interface or touch panel interface 222. The touch panel interface 222 is an interface of a touch screen device. In addition, the touch panel interface 222 may be configured to interface with a cellular phone, a tablet, a mouse, a high-end remote control, and the like.

  The decoded output signal is sent to the input stream processor 204. The input stream processor 204 performs final signal selection and processing, including separation of video content from audio content for the content stream. The audio content is sent to an audio processor 206 that converts a received format such as a compressed digital signal into an analog wave signal. The analog waveform signal is sent to the audio interface 208 and further sent to a display device or an audio amplifier. Alternatively, the audio interface 208 provides a digital signal to the audio output device or the display device using another alternative audio interface such as a high-definition multimedia interface (HDMI) cable or a Sony / Philips Digital Interconnect Format (SPDIF). May be. The audio interface may include an amplifier that drives one or more speaker sets. The audio processor 206 also performs conversions necessary for storing audio signals.

  The video output from the input stream processor 204 is sent to the video processor 210. The video signal is one of a plurality of formats. The video processor 210 converts video content as necessary based on the input signal format. The video processor 210 also performs conversions necessary for storing video signals.

  The storage device 212 stores audio content and video content received at the input. The storage device 212 reads and plays back content based on commands (for example, navigation commands such as fast forward (FF) and rewind (Rew)) received from the user interface 216 and / or the touch panel interface 222 under the control of the controller 214. Enable. The storage device 212 is one or a plurality of large-capacity integrated electronic memories such as a hard disk drive, SRAM (static RAM) or DRAM (dynamic RAM), or a CD (compact disk) drive or a DVD (digital video disk) drive. It may be a replaceable optical disk storage system.

  The converted video signal from the video processor 210, whether from the input or from the storage device 212, is sent to the display interface 218. Display interface 218 further sends display signals to display devices of the type described above. The display interface 218 may be an analog signal interface such as RGB (red-green-blue) or a digital interface such as HDMI. Needless to say, the display interface 218 generates various screens representing search results in a three-dimensional grid, as will be described in more detail below.

  Controller 214 is interconnected via a bus to a number of components of device 200 including input stream processor 202, audio processor 206, video processor 210, storage device 212, and user interface 216. The controller 214 manages a conversion process for converting the input stream signal into a signal to be stored or displayed in the storage device. The controller 214 also manages reading and playback of stored content. Further, as will be described below, the controller 214 searches for stored content or is delivered via a delivery network and generates and adjusts a grid display representing the content.

  The controller 214 further stores a control memory 220 (for example, a volatile or nonvolatile memory such as a volatile or non-volatile memory, RAM, SRAM, DRAM, ROM, PROM (programmable ROM), flash memory, EPROM, and the like. (Electronically programmable ROM), EEPROM (electronically erasable programmable ROM, etc.). Control memory 220 stores instructions for controller 214. The control memory may store a database of elements such as graphic elements including content. The database may be stored as a pattern of graphic elements. Alternatively, the memory stores graphic elements in a plurality of identified or grouped memory locations and uses an access or location table to identify memory locations for various information about the graphic elements. Details regarding storage of graphic elements are further described below. Further, the implementation of the control memory 220 may include a single memory device, or two or more memory circuits that are communicatively connected or coupled to form a shared memory, and even larger circuits, where the memory is a bus communication circuit. Some may be included with other circuits.

  The user interface process of the present disclosure utilizes input devices that can be used to represent functions such as fast forward and rewind. To enable this, the touch panel device 300 can be interfaced via the user interface 216 and / or the touch panel interface 222 of the receiving device 200 as shown in FIG. The touch panel device 300 enables the operation of the receiving device or the set top box through the panel based on hand movements, gestures, and operations that are converted into commands for the set top box or other control devices. In one embodiment, touch panel 300 simply functions as a navigation tool for navigating the grid display. In other embodiments, the touch panel 300 further functions as a display device that allows a user to interact more directly with navigation through a grid display of content. The touch panel device may be included as part of a remote control device that includes more conventional control functions such as actuator buttons and activator buttons. The touch panel 300 can also include at least one camera element. In some embodiments, touch panel 300 may also include a microphone.

  Referring now to FIG. 4, many types of user interaction are provided through the use of a gesture detection controller or touch screen as shown. Input from the controller is used to define a gesture, which in turn defines specific contextual commands. The configuration of the sensor can define the movement of the user's finger on the touch screen and can define the movement of the controller itself in one or two dimensions. Any three-dimensional movement, such as a swing, can be defined using a two-dimensional movement, such as diagonal, and a combination of yaw, pitch, and roll. A number of gestures are illustrated in FIG. Gestures are interpreted in context and identified by defined movements made by the user.

  Bumping 420 is defined by a two-stroke drawing that points in one direction of up, down, left, or right. A bumping gesture is associated with a specific command in context. For example, in the time shift mode, the left bump gesture 420 indicates rewind, and the right bump gesture indicates fast forward. In other contexts, the bump gesture 420 is interpreted as incrementing a value in the direction specified by the bump. Checking 440 is defined as writing a check mark. This is similar to the downward bump gesture 420. Checking is specified in the context of specifying reminders, user tags, or selecting items or elements. Circling 440 is defined as drawing a circle in either direction. It is also possible to distinguish both directions. However, to avoid confusion, the circle is specified as one command regardless of direction. Dragging 450 is defined as the angular movement of the controller while pressing a (virtual or physical) button on the tablet 300 (ie, a “trigger drag”). The dragging gesture 450 can be used for navigation, speed, distance, time shift, rewind, and fast forward. The dragging 450 can be used to move the cursor, virtual cursor, and change states such as highlights, outlines, or selections on the display. The dragging 450 may be in any direction, and is generally used for two-dimensional navigation. However, in some interfaces it is preferable to modify the response to the dragging command. For example, in one interface, depending on the position of the virtual cursor and the direction of movement, operations in one dimension or direction are preferred over other dimensions or directions. Nodding 460 is defined by two fast trigger-drag up-and-down vertical movements. The nodding 460 indicates “Yes” or “Accept”. Crossing 470 is defined as writing the letter “X”. The crossing 470 is used as a “delete” command or a “block” command. Wagging 480 is defined by two trigger-drag fast back-and-forth horizontal movements. The wagging gesture 480 is used to indicate “No” or “Cancel”.

  Depending on the complexity of the sensor system, only one-dimensional movement may be allowed. For example, a simple left and right movement in the sensor shown here may indicate a fast forward or rewind function. In addition, a plurality of sensors may be arranged at different locations on the touch screen. For example, a horizontal sensor that detects left and right movements can be arranged at a certain spot and used for volume up / down, and a vertical sensor that detects vertical movement can be arranged at different spots and used for channel up and down. . Thus, a specific gesture mapping may be used.

  Depending on the hardware and content involved, the system and method can be implemented in various ways. Examples of such deployments include DVD, Blu-ray Disc (BD), streaming video or video on demand (VOD), and broadcast (satellite, cable, terrestrial). Each of these deployments has a different architecture, but triggers may be standardized for each of these events (additional content) representing what is queued by the application running on the second screen. For example, event A and event B are triggered by a synchronization mechanism associated with the video source. When “event A” occurs in the tablet, the program executed on the second screen device (for example, tablet) displays “event A”. Similarly, when “event B” occurs, the post-pro ram running on the second screen device executes (do) “event B”.

  FIG. 5 shows a generic system 500 that can implement such a method. Here, the system 500 includes a first screen device 510, a second screen device 520, a playback device 530, a network 540, and a server 550. Each element is described in more detail below.

  The first screen device 510 is a display device such as the display device 114 described above with reference to FIG. 1 that displays content such as television programs, movies, websites, and the like. Examples of such first screen devices include, but are not limited to, televisions, monitors, projectors, and the like. The first screen device 510 is connected to the playback device 530. The playback device 530 can supply main content to be displayed on the first screen device 510. Examples of such communications include, but are not limited to, HDMI, VGA, display port, USB, component, composite, radio frequency (RF), infrared (IR), and the like. In one embodiment, the first screen display device 510 is connected to the network 540 by wire or wireless (WiFi). The network 540 further provides a connection to the second screen device 520 and the server 550. In an embodiment, the first display device 510 may include the function of the playback device 530. In yet another embodiment, the first screen display device 510 may be in non-network communication 560 with the second screen device 520. Examples of such non-network communication 560 include, but are not limited to, RF, IR, Bluetooth, and other audio communication techniques and protocols.

  The second screen device 520 is a device that can display additional content related to the main content displayed on the first screen device 510. The second screen device may be the touch screen control device 116 or the touch screen device 300 as described above. Examples of the second screen device include, but are not limited to, a smartphone, a tablet, a laptop, a personal media player (for example, ipod), and the like. Second screen device 520 communicates with playback device 530 using network 540, non-network communication 560, or both. The second screen device 520 communicates with the server 550 via the network 540 to request and receive additional content related to the main content displayed on the first screen device 510. In some embodiments, the second screen display device 520 may be in network communication with the first screen device 510 or may be in non-network communication 560. Examples of such non-network communication 560 include, but are not limited to, RF, IR, Bluetooth, audio communication techniques and protocols.

  The playback device 530 is a device that can supply main content to be displayed on the first screen device 510. Examples of such playback and display devices include DVD players, Blu-ray Disc (BD) players, game consoles, (cable or satellite) receiver devices, digital video recorders (DVR), streaming devices, personal computers, etc. It is not limited to. The playback device 530 is connected to the first screen device 510 to supply main content to be displayed on the first screen device 510. Examples of such connections include, but are not limited to, HDMI, VGA, display port, USB, component, composite, radio frequency (RF), infrared (IR), and the like. The playback device 530 is connected to the network 540 by wire or wireless (WiFi). The network 540 provides a connection to the second screen device 520 and the server 550. In an embodiment, the function of the playback device 530 may be included in the first screen display device 510. In yet another embodiment, the playback device 530 may be in non-network communication 560 with the second screen device 520. Examples of such non-network communication 560 include, but are not limited to, RF, IR, Bluetooth (BT), and other audio communication techniques and protocols.

  Network 540 may be a wired or wireless communication network implemented using Ethernet, MoCA, wireless protocols, or combinations thereof. Examples of such networks include, but are not limited to, delivery network 1 (106) and delivery network 2 (112) described above.

  Server 550 is a content server configured to supply additional content to second screen device 520. In some embodiments, the server may supply main content for display on the first screen device 510. The service is connected to the network 540 and can communicate with any connected device. Examples of such servers include, but are not limited to, the content source 102, broadcast affiliate manager 104, content manager 110, and backend server described above.

  FIG. 6 is a flow diagram 600 illustrating a method for displaying additional content related to the main content being viewed. The method includes the following steps: displaying main content on the first screen device 510 (step 610). In connection with the display of the main content on the first screen, a synchronization mechanism for synchronizing the additional content is provided (step 620). The additional content related to the main content on the first screen 510 synchronized with the content displayed on the first screen device by the synchronization mechanism is displayed on the second screen device 520 (step 630). In one embodiment, the method is based on receiving a command from the second screen device 520 that controls the display of the main content on the first screen device 510 (step 640), and on the command received from the second screen device 520. The first screen device 510 includes a step of controlling the display of the main content (step 650). Each step is described in more detail below.

  The step of displaying main content such as a movie or a television show (step 610) is executed by the first screen device 510. This also includes supplying main content to the first screen display 510. The main content can be supplied by the playback device 530 or received directly from the content provider at the first screen display device 510. The main content is displayed on the first screen display 510. The display of the main content includes control of the displayed content. This includes conventional playback commands for play, stop, pause, rewind, fast forward as well as on-screen menu navigation for selecting content and other playback options. In some embodiments, the display (step 620) of the first screen display 510 further includes an indicator of an additional content type indicator displayed on the second screen device 520.

  The provided synchronization mechanism (step 620) can be implemented in many ways. In some embodiments, the synchronization mechanism is performed by an application running on the second screen device 520, the playback mechanism 530, the first screen device 510, or any combination thereof. Most basically, the second screen device 520 updates the display of the additional content to the second screen device 520 and instructs the second screen device 520 to match the main content displayed on the first screen device 510. Are configured to detect (via application) types of indicators. Synchronization signals, cues and other types of indicators may be provided as part of the main content, or the playback device of the first screen device 510 depending on the chapter, scene, time code, subject matter and displayed content May be generated by 530 (via an application). Synchronization signals, cues, and other types of indicators are transmitted to the second screen device 520, either wired or wireless (WiFi), using a network, or using non-network communication 560 such as an audio signal. . Some implementation examples are given below. Other possible implementations will be apparent with reference to this disclosure.

  The step of displaying additional content such as supplemental materials, video clips, websites (step 630) is performed on the second screen device 520. The additional content can be stored locally on the second screen device 520 or provided by the server 550, the playback device 530, or the first screen device 510. The display of the additional content is synchronized with the main content displayed on the first screen device 510 by a synchronization mechanism. For example, when the second screen device 520 detects a synchronization signal, a cue or other type of indicator, the second screen device 520 updates the display of the additional content as appropriate. In some embodiments, this further includes contextual request to server 550, playback device 530 or first screen device 510 to request additional content and then download and display the additional content. In some embodiments, the additional content that is displayed is selected, modified, or omitted based on the user using the system.

  In some embodiments, the display on the second screen device 520 (step 630) further includes a display of the main content on the first screen device 510, such as whether the display of the main content on the first screen device 510 has been paused. Including displaying status. In some other embodiments, the display on second screen device 520 (step 630) further displays the status of synchronization between the additional content on second screen device 520 and the main content on first screen device 510. Including doing.

  In some embodiments, the second screen device 520 can transmit as well as receive. Optional steps 640 and 650 address this capability. In step 640, a command is received from the second screen device 520. Ideally, these commands are received at a device that controls playback of the main content on the first screen device 510. In some embodiments, playback device 530 is the device that receives the command. Commands can be sent via network 540 or non-network communication 560. Once received, the command can control the display of the main content (step 650). Examples of such control include, but are not limited to, play, stop, pause, rewind, fast forward as well as chapter, scene, and selection. Also, using these commands, the main content displayed on the first screen device 510 can be synchronized with the additional content displayed on the second screen device 520.

  FIG. 7 is a high-level overview of an example system 700 having a synchronization mechanism implemented using non-network communication 560. In this system 700, the non-network communication synchronization mechanism is an audio watermark 710. In this example, the audio watermark 710 includes inserting a high frequency signal, cue or other indicator into the audio signal of the main content displayed on the first screen device 510. The audio watermark cannot be heard by humans but can be detected by the microphone of the second screen device 520. When the second screen device 520 detects the audio watermark, the displayed additional content is updated to synchronize with the main content displayed on the first screen device 510 based on the detected watermark. The audio watermark may be embedded in the main content at the content source or may be inserted locally by the playback device 520 or the first screen device 510.

  FIG. 8 is a diagram illustrating a high level overview of an example system 800 with a synchronization mechanism implemented using network 540. In this system 800, the synchronization mechanism is wireless communication (WiFi) 810 between the playback device 530 (Blu-ray disc player) and the second screen device 520 (iOS device that executes an application). In the example of FIG. 7, the function and protocol of a BD live enabled device is used. This protocol has two main components: connection and communication. Both will be described below. For simplification, the second screen iOS application is called “iPad” and the BD live compatible device is called “disk”.

When an iOS enabled device 520 starts a second screen application first and attempts to connect to a BD-Live enabled device 530 on the same WiFi network 540, it is connected.
1. 1. Disc is inserted into BD player 2. Disk enters UDP “listening” loop 3. iPad starts second screen application 4. iPad performs UDP broadcast of authentication token 5. The disk receives the authentication token and authenticates 6. Disc reads IP (IPad's IP) from token sender 7. Disk responds to authentication with its IP and PORT 8. iPad confirms IP and PORT IPad closes UDP socket communication10. IPad establishes TCP socket communication directly with disk based on provided IP and PORT Communication is established when a connection is established between the second screen iOS application and the BD live compatible device.
1. 1. iPad and disk know which PORT to communicate with each other's IP. TCP socket communication is maintained during the application life cycle As can be seen in this example, one advantage of such wireless communication is that it is bi-directional and the second screen device can not only receive but also send commands. Thereby, bidirectional synchronization and playback control from the second screen device 520 are possible.

  In some embodiments, the application of the second screen device 520 may be specific to a program or movie in a system (eg, BD). In some other embodiments, the second screen application is generic to the studio and a plug-in is available that configures the application for a specific program or movie. In still other embodiments, the second screen application may be universal for the system (BD, VOD, broadcast), content, or both. Upon reading this disclosure, other possible implementations and configurations will be apparent to those skilled in the art.

  The system can operate in a passive or interactive approach. In the passive approach, the icon displayed on the first screen device 510 displays an additional content event related to the main content displayed on the first screen device 510, so that the second screen device 520 is viewed. To prompt the user. The icon preferably indicates what type of additional content event is available on the second screen device 520 (eg, a shopping cart icon is a shopping event, an “i” icon is an information event, and a stickman icon is a character information event. Show). 9A to 9F show aspects displayed to the user in the passive mode.

  FIGS. 9A to 9F show the screen 900 of the second screen device to the user when using a passive mode application that provides additional content of the second screen device 520 synchronized with the main content of the first screen device 510. A skeletal example of what is displayed in

  FIG. 9A shows a splash screen that is displayed to the user when the application is launched. This includes a product logo and main content display 902. Here, as indicated by arrow 904, the new content screen moves from the right like a conveyor belt.

  FIG. 9B shows a pop-up message 906 that is displayed to the user when the second screen device 520 does not detect the playback device 530.

  The screen 900 of FIG. 9C includes a sync button / icon 908, a chapter timeline 910, an active chapter indicator 912, a chapter event indicator 914, a chapter number indicator 916, an event timeline 918, a chapter background 920, an event card 922, and a timeline. A view icon 924 is shown. The sync button 908 provides a mechanism for synchronizing content between the first and second screen devices 510, 520. The synchronization button 908 may indicate a synchronization state between contents on the first and second screen devices 510 and 520. The chapter timeline 910 shows the chapter of the main content. The movie title reader is in the background of the chapter timeline 910 and shows the main content. As the main content progresses, the chapter moves along the chapter timeline like a conveyor belt, and the active chapter indicator 912 indicates the current chapter in the main content by the highlight and center position of the chapter timeline 912. The chapter event indicator 914 indicates that the event displayed on the event timeline 918 is a part of the active chapter displayed on the chapter timeline 910. The event timeline 918 displays an event card 922 indicating events corresponding to what is happening in the current chapter of the main content. For each chapter, the event card 922 displayed first indicates the chapter in which the next event occurs. As the main content progresses, the event card 922 moves along the event timeline 918 like a conveyor belt, and the current event is at the center of the event timeline 918. Each chapter may be provided with a unique background 920 of events for that chapter. Timeline view icon / button 924 indicates that the viewer is in a timeline view showing chapter timeline 910 and event timeline 918 and provides a mechanism for accessing the timeline view.

  The screen 900 of FIGS. 9D and 9E shows how the event card 922 proceeds through the event timeline 918. Here, the synchronization button / icon 908 indicates that the timeline view of the additional content is synchronized with the main content on the first screen device 510. In FIG. 9D, the currently triggered event card 926 is shown in the center position of the event timeline 918 and represents the first triggered event. To the left of the currently triggered event card 926 in the event timeline 918 is the previous event card 928, where the card represents a chapter. To the right of the currently triggered event card 926 in the event timeline 918 is the next event card 930, where the card indicates the next event. In FIG. 9D, since the currently triggered event card 926 is that of the first triggered event, the chapter indicator 916 indicates chapter 1. The currently triggered event card 926 includes additional content 932 related to the main content. The currently triggered event card 926 also provides an indicator 934 regarding what type of additional content is displayed. In some embodiments, this indicator matches the indicator shown on the first screen display 510. The current event card 926 also includes buttons / icons for synchronizing 936 and sharing 938. The sync button / icon 936 provides a mechanism to synchronize the main content displayed on the first screen device 520 with the current event. Share button / icon 938 provides a mechanism for sharing additional content of an event with a social network. The elements of screen 900 of FIG. 9E are similar to those of FIG. 9D, but the currently triggered event card 926 occurs later in the timeline, as indicated by the chapter indicator 916 indicating that the current chapter is chapter 3 It is different in that it is for an event.

  FIG. 9F shows examples of other possible functions provided as part of the display on the second screen device 920. Here, the chapter timeline 910 is provided with a collapse icon / button 940 that provides a mechanism for toggling the chapter timeline between visible 940a and hidden 940b. Similarly, the sync button / icon 908 can toggle between states indicating whether the state 908a indicates that synchronization is currently active, or state 908b indicates that it is not synchronized and can be resynchronized. In some embodiments, a volume button icon 942 is provided. The volume button / icon 942 provides a mechanism to “off” or “on” the audio of the first screen display. The volume button 942 can also indicate whether the volume is “on” and mute can be used 942a, or “off” and sound can be used 942b. In some embodiments, a play / pause button / icon 944 is provided. The play / pause button 944 provides a mechanism for pausing or resuming playback of content on the first screen display 510. The pause / play button may also indicate whether playback can be paused 944a or resumed 944b.

  In the interactive approach, the user selects an additional content event on the second screen device 520 and what is displayed on the first screen device 510 is synchronized with the selected event. As previously indicated, the additional content event is synchronized with the main content. If the user swipes a movie timeline or event, the event will be out of sync with the movie displayed on the main screen. To resynchronize, touch the resynchronization button on the tablet. The timeline or event is synch backed to what is displayed on the main screen. Similarly, the user can select a trivia event or map event, touch the sync button, and the movie scene associated with the selected trivia event or map event is played on the main screen. Examples of this are shown in FIGS. 10A to 10D.

  FIG. 10A shows how a user interacts with chapter timeline 910 and event timeline 918 on screen 900. Here, the icons 1000 and 1002 represent how the user can touch the screen to scroll the chapters or event timelines 910 and 918 left and right.

  FIG. 10B illustrates one embodiment of a screen 900 when the user interacts with the chapter timeline 910. In this example, sync button / icon 908 indicates that the additional content on second screen display 520 is not synchronized with the main content on first screen display 510. Icon 1000 represents the user scrolling through chapter timeline 910. The current chapter remains highlighted until the transition to the new chapter is complete. When navigating the chapter timeline 910, a chapter position indicator 1004 is provided that indicates which of the available chapters has been selected. The chapter indicator 916 indicates the selected chapter and is updated when the transition to the new chapter is complete. In this example, the user navigates the chapter timeline 910, but the event timeline 918 is dimmed. In some embodiments, the user can jump directly to a chapter by selecting the chapter from the timeline 910.

  FIG. 10C illustrates one embodiment of a screen 900 when the user interacts with the event timeline 918. Icon 1002 represents the user scrolling through event timeline 918. Here, the timeline 918 transitions from the currently triggered event card 926 to the next event card 930. When navigating the event timeline 918, an event position indicator 1004 is provided that indicates which of the available events has been selected.

  FIG. 10D illustrates one embodiment of a screen 900 when the user interacts with the event timeline 918 to transition from one chapter to another. The icon 1002 represents a place where the user has changed the chapter by scrolling the event timeline 910. Here, the timeline 918 transitions and the new event card 922 shows a new set of events associated with the new chapter. When navigating to a new chapter by navigation in the event timeline 918, the event position indicator 1004 remains central until a series of new events begins.

  FIGS. 11A-11C and FIGS. 12A and 12B illustrate other interactive activities that can be accessed via the event card 922. 11A to 11C show a social media sharing function. 12A and 12B illustrate chapter selection and additional media file selection and playback.

  FIGS. 11A-11C show various pop-up fields on the display 900 when the sharing function is activated by the share button / icon 937. FIG. 11A shows a field 1100 that is displayed when a user logs into their social network (in this case Facebook). An area 1102 shows an event to be shared, and an area 1104 shows a comment that the user wants to share regarding the event. Button 1106 provides a mechanism for sending comments to be shared with the event. FIG. 11B shows the field 1100 displayed when the user has not yet signed in to the social network. In this example, a button 1108 is provided for signing in to Facebook, and a button 1110 is provided for signing in to Twitter. You may have the option to sign in to other social networks. FIG. 11C shows an on-screen Qwerty keyboard 1112 used to enter comments in the user comment area 1104. In some embodiments, this is the default keyboard provided by the second screen device 520.

  12A and 12B illustrate the selection of media content and chapters for the user to play back. In the example of FIG. 12A, when the user performs a single tap 1200 on a chapter that is currently being reproduced as shown in the chapter timeline 912, the display on the first screen device 510 is temporarily stopped. When the user double taps 1202 the currently playing chapter shown in the chapter timeline, the playback on the first screen device jumps to the beginning of the chapter, and the event timeline 918 is the first event of that chapter. Is set. In some embodiments, the event card 922 includes media files 1204 such as video clips and audio clips. When the media file is an audio clip, when the audio clip is selected, it is played back on the current screen 900. If the media file is a video clip, selecting the video clip launches a full screen media player as shown in FIG. 12B. In this example, the media player includes an on-screen control 1208. The user may tap the non-video surface 1210 of the media player to return to the previous screen.

  13A-13E illustrate other functions that may be related to additional content. This includes a map view 1300, a family tree 1310, and settings 1320. FIG. 13A shows a menu bar for these options. In this example, each menu bar is provided with a first screen device control 1330 that includes pause / resume and mute / mute release. FIG. 13B shows a map view display 1300. The map view display 1300 includes a map 1302 that includes a marked location 1304 and information 1306 about the location. An icon 1308 for selecting another map is also provided. FIG. 13C shows a family tree view 1310. The family tree view shows a family tree having a field 1312 that shows relationships between family members. In this example, the bottom button / icon 1314 indicates which view is currently displayed (in this case, the Family Tree view is displayed). When field 1312 is selected, a pop-up field 1316 is displayed, as shown in FIG. 13D, providing information about the person in field 1312. FIG. 13E shows the settings view 1320. In view 1320, the user is provided with controls that adjust their preferences for audio and video 1322, events 1324, and social network sharing 1326, respectively.

  14A to 14L are displayed on the screen 900 of the second screen device to the user when using an application that provides additional content of the second screen device 520 synchronized with the main content of the first screen device 510. Here is a skeletal example of things. FIG. 14A is a version with the skin of the splash screen shown and described in FIG. 9A. 14B-14F show skinned versions of the timeline view described with reference to FIGS. 9C-9F and 10A-10D. FIG. 14G shows a screened skinned version that displays to the user all available video clips that are part of the additional content. FIG. 14H shows a version with a screen display skin that displays to the user all available audio clips that are part of the additional content. FIG. 14H is a version with the map view skin shown in FIG. 13B described. FIGS. 14J and 14K show skinned versions of the described Family Tree view shown in FIGS. 13C and 13D, respectively. FIG. 14L is a version including the setting view skin shown in FIG. 13E.

  The event function shown in the drawing is just an example of a possible event. In some embodiments, the user can set or select which events they want to display (eg, do not want shopping events to be displayed). In certain other embodiments, the user can select or bookmark the event for later viewing. In other embodiments, certain events may not be available or locked, depending on the version of the program being viewed (eg, purchase vs. rental, or BD vs. VOD vs. broadcast). In some other embodiments, based on previous viewing habits, the available events can be personalized to the user (ie, in a system such as TIVO that tracks the user's viewing habits, or the personalization engine 118 of FIG. make use of).

  Another possible configuration includes a shopping function. For example, a storefront for purchasing movie-related products may be provided on the second screen so that it can be accessed. In another embodiment, points or awards may be provided to the user for viewing, reviewing or recommending a program or movie. For example, if you watch more movies or share with friends, you will get more points. Points are used for discounts on prizes and related products.

Similarly, an achievement may be awarded. These achievements may be pushed to social networking sites. Examples of achievements include:
Achievement for certain disc viewing in an achievement series for certain scene viewing Achievement for certain disc viewing for a particular studio or actor In yet other implementations, Wiki functionality can be implemented. Running Wiki allows users and other users of the disc to comment on certain scenes. For example, tracking metadata may be generated and pushed to a web-based wiki. Such metadata may include the following:
Chapter information Information about timecode scene thumbnail actors and directors This pushed information can be used to generate a running Wiki where other people can comment on the movie. These comments can be re-embedded in the second screen application as accessible events.

  This description is illustrative of the principles of the present disclosure. Needless to say, those skilled in the art can express the principles of the present disclosure and devise various configurations included in the spirit and scope of the present disclosure, although not explicitly described or illustrated herein.

  All examples and conditional words contained herein are intended to make it easier for the reader to understand the principles and concepts of the present disclosure that the inventor has made for technological development, It should not be limited to the specifically described examples and conditions.

  Further, all descriptions of principles, aspects, embodiments, and examples of this disclosure include both structural and functional equivalents thereof. Such equivalents also include currently known equivalents and equivalents that will be developed in the future, i.e., all elements that are developed that perform the same function regardless of configuration.

  Thus, for example, it goes without saying to those skilled in the art that the block diagrams described herein conceptually illustrate circuits that embody the principles of the present disclosure. Similarly, it goes without saying that flowcharts, flow diagrams, state transition diagrams, pseudocode, etc. represent various processes, even if these methods are substantially represented on a computer-readable medium (shown explicitly). It may be executed by a computer or a processor.

  The functions of the various elements shown in the figure can be provided using dedicated hardware or a combination of hardware capable of executing software and appropriate software. When the processor is provided, the function may be provided by a single dedicated processor, may be provided by a single shared processor, or may be provided by a plurality of individual processors that are partially shared. Good. Further, the explicit use of the terms “processor” or “controller” should not be construed to refer only to hardware capable of executing software, including but not limited to digital signal processor (DSP), software A ROM, a RAM, and a non-volatile storage device may be included implicitly.

  Other hardware may be conventional or custom. Similarly, the switches shown in the drawings are conceptual. The function of the switch may be executed by program logic operation, dedicated logic operation, program control or dedicated logic interaction, or manually, and a specific method can be selected by a practitioner based on context.

  While embodiments incorporating the teachings of the present disclosure have been illustrated and described in detail, those skilled in the art can still readily devise many other variations that incorporate these teachings. While preferred embodiments of methods and systems for providing media recommendations have been described (these are exemplary and not limiting), it should be noted that those skilled in the art can make modifications and variations in view of the above teachings. Should.

  Although the above embodiments focus on electronic devices, it will be appreciated that the invention may be implemented as a computer program product. The computer program product has all the features that enable the implementation of the methods described herein, and can execute these methods when loaded into a computer system. In the case of the present invention, a computer program or application is either directly to a system with information processing capabilities, or a) conversion to another language, code, or notation, or b) playback in a different material format, or After both are included any representation of any language, code, or notation of a set of instructions intended to perform a particular function.

  Further, the above description is merely an example, and is not intended to limit the present invention except as described in the claims.

Claims (23)

A method of displaying additional content related to the main content being watched,
Displaying main content on the first screen device;
Providing a synchronization mechanism for synchronizing additional content in relation to the display of the main content on the first screen;
Displaying on the second screen device additional content relating to the main content on the first screen synchronized with the content displayed on the first screen device by the synchronization mechanism.   The method of claim 1, wherein providing a synchronization mechanism comprises providing an audio watermark as part of a display of main content on the first screen device. The step of displaying additional content is
Detecting the provided audio watermark in the second screen device;
Displaying the additional content on the second screen device based on the detected audio watermark.
The method of claim 2. Receiving from the second screen device a command for controlling the display of the main content on the first screen device;
Controlling the display of the main content on the first screen device based on the command received from the second screen device;
The method of claim 1.   5. The method of claim 4, wherein receiving and controlling is performed by a playback device that supplies main content to the first display device.   The method of claim 4, wherein the command is received via a wireless communication protocol.   The method of claim 1, wherein providing a synchronization mechanism comprises providing a wireless communication signal that synchronizes additional content in connection with display of main content on the first screen. The step of displaying additional content is
Detecting the supplied wireless communication signal in the second screen device;
Displaying the additional content based on the detected wireless communication signal in the second screen device;
The method of claim 7.   8. The method of claim 7, wherein the step of supplying a wireless communication signal is performed by a playback device that supplies main content to the first display device.   The method of claim 1, further comprising displaying an icon on the first screen device indicating an additional content type to be displayed on the second screen device. The second screen device further includes a step of displaying a synchronization state between the content on the first screen device and the additional content on the second screen device.
The method of claim 1. The step of displaying additional content is
Receiving a synchronization command via the synchronization mechanism in the second screen device;
Contacting a server based on the received signal;
Loading additional content from the contacted server based on the received signal;
Displaying the additional content on the second screen device,
The method of claim 1.   The method according to claim 1, further comprising displaying a display state of main content on the first screen device in the second screen device.   The method according to claim 1, further comprising providing a function of sharing content on a social network in the second screen device. The displayed additional content is based on a user of the first and second screen devices,
The method of claim 1.   The method of claim 1, wherein displaying additional content comprises scrolling the additional content across the second screen display in synchronization with main content displayed on the first screen device. The type of additional content displayed can be selected by the user.
The method of claim 1.   The method of claim 1, wherein the type of additional content displayed is based on a type of content displayed on the first screen device. A system for displaying additional content related to the main content being viewed,
A first screen device configured to display main content;
A second screen device configured to display additional content related to main content on the first screen synchronized with the content displayed on the first screen device by the synchronization mechanism.   The system of claim 19, wherein the first screen device includes one of a television, a monitor, and a video projector.   The system of claim 19, wherein the second screen device includes one of a smartphone, a tablet, a personal media player, and a laptop.   And a playback device configured to provide a synchronization mechanism for supplying the main content to the first display device and synchronizing additional content on the second screen with the main content on the first screen device. The system of claim 19.   The system of claim 19, further comprising a server configured to provide additional content for display on the second screen device.

Images