MXPA00005641A - Context life time management of a user interface in a digital tv broadcast - Google Patents

Abstract

In a digital television broadcast that contains executable code segments that are associated with specific video segments, a context duration time is defined for each executable component. The context duration time is determined so as to correspond to the context of the video segment that is associated with the executable component. In a preferred embodiment, the context duration time for the executable component is determined when the corresponding video segment is being prepared for encoding into the transport stream. The context duration time is subsequently included in the encoding of the executable component in the transport stream. When the executable component is decoded fromthe transport stream, the context duration time is used to effect a termination of the executable component after the context duration time has elapsed. In this manner, each launched executable component will be terminated within a predetermined time from the start of execution of the executable component, independent of the subsequent contents of the transport stream.

Description

HANDLING THE LIFETIME OF THE CONTEXT OF A USER INTERFACE IN A DIGITAL TV EMISSION BACKGROUND OF THE INVENTION 1. Field of the Invention This invention relates to the field of communications, and in particular to the field of digital television broadcasts with interactive and included applications. 2. Description of the Related Art The broadcasting of television signals in a digital format provides the transmission of data packets, as well as conventional video and audio packets. The data packets may contain auxiliary information related to the video and audio packets. The data packets may also contain a code that is executable on a computer, such as a box placed at the top, a computer included in the television receiver, or a conventional personal computer. By transmitting executable code, a user can interact with the computer in the context of the video and audio information that is being transmitted. That is, for example, when a product is being announced, the executable code can be configured to allow the user to place an order for that product. Conventionally, digital television signals are transmitted as a series of packets that form "a transport stream." The MPEG-2 standard includes means for identifying each packet of digital information to determine whether it contains video, audio, or interaction information. conventional digital television with interactive capabilities, the packets are demultiplexed from the transport stream in separate video, audio and data streams, video and audio streams are processed by video and audio decoders, respectively, and the data streams are stored in memory by an application processor, and then released for execution.The data flow typically consists of multiple packets that each form an executable component of the code.Typically, the released executable component performs the presentation of one or more buttons for the user's subsequent selection, for example, during the announcement of a For example, the executable component associated with the advertisement may have a button that reads: "Click here for additional information", or a note that reads: "Press 1 on your remote control to buy this product". When the user activates the appropriate button, the executable component proceeds to present additional information. The additional information may be contained within the executable component, or may be contained in a separate data component or produced by another executable component. Multiple executable components can be associated with one or more segments of the video or audio streams. For example, in a shopping channel, the products are presented in a sequenced manner, and a variety of products can be presented on the television screen at the same time. Each executable component that is related to a product in this example would be configured to present a different message, such as "Click here to order the bicycle", "Click here to buy the diamond ring", and so on. Executable components can be independent, or they can be components of a larger application. For example, an application for general purposes can be associated with a particular sales program that contains the code commonly used to secure credit card information and the like, and each executable component related to a product is designed to be part of the application program. general. In order to synchronize the execution of each executable component, queue markers are placed in the video stream. A first queue marker channel is used to notify the application processor when to start a particular executable component, and another queue marker identifies when the application processor must finish the executable component and remove the component from memory. This synchronization process, however, is problematic. The detection of queue markers in the video stream requires a continuous verification of the flow of those markers, which increases the complexity of the video processor, and can have an impact on the operation and total cost of the interactive television system. Most video streams conform to protocols, such as MPEG, which compensates for bandwidth congestion by dynamically adjusting the content of each video frame. In such protocols, the information in the video frames will be discarded once there is insufficient bandwidth to transmit the frames in their entirety. Thus, there is a likelihood that one or more queue markers in the original encoding of the video stream will be discarded or lost through the encoding-transmission-decoding process associated with conventional digital television broadcasting. If a marker is lost from the initial queue, the execution of the component will not be released; If the terminal queue marker is lost, the component will remain active outside the context of the video stream.

A similar difficulty may arise when a user changes the source of the transport flow, for example, by changing channels on the television receiver. If the terminal queue marker is lost because the user changed channels before the corresponding announcement ended, the execution of the component will remain active outside the context of the video stream. The user will be given the option of, for example, purchasing an article without a video reference corresponding to the article. Therefore, there is a need for means to initiate and terminate the execution of each component that are independent of the received video stream.

BRIEF DESCRIPTION OF THE INVENTION An object of this invention is to effect the synchronization of executable components in a digital television broadcast to corresponding segments of a video stream without relying on continuous or complete reception of the original content of the video stream. This object and other objects are achieved by assigning a "lifetime" to each executable component. The lifetime is determined so that it is within the video segment that corresponds to the executable component. In a preferred embodiment, the duration of the context lifetime for the executable component is determined when the corresponding video segment is being prepared to encode it in the transport stream. The duration time of the context is subsequently included in the encoding of the executable component in the transport flow. When the executable component is decoded from the transport stream, the duration time of the context is used to perform the termination of the executable component after the duration time of the context has elapsed. In this way, each released executable component will end within a predetermined time, regardless of the subsequent content of the transport stream.

BRIEF DESCRIPTION OF THE DRAWINGS FIGURE 1 illustrates an exemplary block diagram of a transport stream encoder and transport-stream decoder in accordance with this invention. FIGURE 2 illustrates an exemplary transport flow in accordance with this invention. FIGURE 3 illustrates an exemplary timing diagram of video segments and corresponding context times according to this invention. FIGURE 4 illustrates an exemplary flow chart for the decoding of video segments and corresponding executable components in accordance with this invention.

FIGURE 5 illustrates an exemplary flow chart for processing video segments and corresponding executable components according to this invention.

DETAILED DESCRIPTION OF THE INVENTION FIGURE 1 illustrates an exemplary block diagram of a transport stream encoder 100 and a transport stream decoder 200. The video images 112 and the executable code 114 are encoded in packets 121, 131, respectively to be included in a transport stream 141. For ease of understanding, only images of -video 112 and executable code 114 are illustrated, although transport stream 141 may contain other components, such as audio and other packets. In the example of FIGURE 1, a context editor was used to associate the executable code 114 with a particular segment of video images 112. In a preferred embodiment of this invention, the context editor determines how much of the segment of the images of video 112 will be presented on the video display device 210. During the time in which the -segment is presented, the associated executable code 114 will be "in context" with the video images 112 being presented. Before and after the associated images 112 are presented, the executable code 114 will be "out of context". That is, an executable code 114 that presents a "Click here to buy the bicycle" button will be out of context, and effectively nonsense, some time before and some time after the image of a bicycle appears on the visual representation device. Based on the duration of the display of the associated images 112, the context editor determines an appropriate context duration time parameter that defines the time interval that the executable code 114 is expected to be in context with the images of video 112. In a preferred embodiment, the context duration time parameter is an integer representing the duration time of the context in seconds. Alternatively, the context duration time parameter is an encoding from which the actual context duration time, such as the number of hundreds of video frames, or the number of clock cycles of the decoder can be determined. predefined After the period of time defined by the context duration time parameter, the executable code 114 effectively loses all sense, and may end. The video images 112 are encoded in video packets 121 by a video encoder 120, and the executable code is encoded in data packets 131 by a data encoder 130. As is common in the art, the video image segments 112 are encoded with a presentation start time that identifies when each segment of the video images 112 will be presented. For example, in an encoding that complies with the MPEG, each video frame has one or more time parameters that are referred to a real-time clock PCR. In a preferred embodiment of this invention, a context start time that is based on the presentation start time is included in the encoding of the executable code 114. In this manner, the synchronization of the executable code 114 with the video images 112 may be substantially independent of the sequencing of the video packets 121 and the data packets 131 in the transport stream 141. In an alternative embodiment of this invention, the placement of video packets 131 between the video packets 121 in the stream of transport determines when the executable code that is contained in the data packet 131 is executed. In this alternative mode, the duration time of the context will be defined somewhat longer than the corresponding presentation duration time, to allow the inaccuracy between the time at which the executable code will be executed and the time at which the segments of the packages of video surrounding this executable code in the transport stream will be presented. The start time of the context and the duration time of the context may also depend on the effects of the executable code 114. If the executable code 114 requires a particular set-up time to produce an application image corresponding to the video image segment, the context start time, if provided, and the duration time of the context, are adjusted appropriately to allow this settling time. That is to say- that in a preferred embodiment of this invention, the duration time of the context will be defined equal to the presentation duration time plus an appropriate damping factor that depends on the degree of coincidence between the start of the presentation of the video images and the start of the presentation of the corresponding application images. 'Video packets 121 and data packets 131 are multiplexed by the multiplexer 140 to form the transport stream 141 which is transmitted to a receiver 250 via a transmitter 150 as a communications signal 151. The term packet is used herein to define identifiable and distinguishable amounts of information, and the term multiplexer defines a technique for combining packets so that they can be subsequently demultiplexed to substantially reproduce those original identifiable or distinguishable information entities. Similarly, the transmitter 150 and the communications signal 151 symbolically identify means of communication of the transport stream 141 to a receiver, as would be apparent to one skilled in the art. The decoder of the transport stream 200 receives the transmitted communications signal 151 via a receiver 250 and produces a reproduced transport stream 241. Ideally, the transport stream 241 is identical to the transport stream 141. The reproduced transport stream 241 is demultiplexing to produce from the same video packets 221 and data packets 231 that correspond substantially to the video packets 121 and data packets 131, respectively. The video packets 221 are processed by a video processor 220 to produce video images 212, and the data packets 231 are processed by an application processor 230 to reproduce the executable code 114. It should be noted that, consistent with the techniques of conventional coding, the encoding of video images 112 to video packets 121 are not necessarily loss-free. Due to bandwidth constraints, video packets 121 may contain less resolution and details than video images 112. Similarly, transmitter 150 may bypass the transmission of some packets due to congestion of the media, or the transmission means between the transmitter 150 and the receiver 250 can adversely affect some packets. Although the video images 212 may be an inaccurate reproduction of the video images 112, the techniques required to effect accurate and accurate reproduction of the executable code 114 of the application processor 230 are applied. That is, for example, that each packet of data 131 is assigned a sequential packet number that contains an error detection code. If the received data packets 231 show a gap or space in the sequence number or packet, or an error code detects an error in the packet, the decoder of the transport stream 200 requests the retransmission of a particular packet of the flow encoder. 100. In the example of a radio transmission of the transport stream encoder 100, for which the feedback of the transport stream decoder 200 is not feasible, the transport stream encoder 100 transmits redundant copies of each transport packet. data 131. In this example, application processor 230 ignores redundant packets after an error-free packet is decoded. Additionally, data packets 131 in this example contain error correction codes that will allow correction of detected errors. If none of the redundant packets for a particular packet sequence number is decoded error-free, the executable code 114 is not produced by the application processor 230 for this sequence of data packets. Those and other techniques for reliable data communication are common to one skilled in the art. The application processor 230 stores the executable code 114 in a memory 260 for execution by an execution processor 270. As would be apparent to one skilled in the art, the video processor 220, the application processor 230, and the processor execution 270 can be independent or dependent processes that operate in a single physical entity, although they are presented here as discrete processors for clarity and ease of understanding. The application processor 230 issues the executable code 114 by communicating a launch command to the execution processor 270 via the link 232. The application processor 230 determines when to launch the executable code 114 based on the methods used to encode the time parameters of the application. context. If a specific context start time is encoded in the data packet sequence 131, the application processor 230 issues the launch command to the execution processor (270) to effect the start of execution at the start time of the specified context . If a specific context start time was not encoded, the application processor 230 issues the launch command immediately after loading the executable code 114 in the memory 260. The execution of the executable code 114 typically produces one or more application images 270 to present them on the video display device 210. The application images 271 are presented with the video images 212 corresponding to the segment of the video packets 221 that are associated with the executable code 114. The executable code 114 also allows a user (not shown) enter application responses 281 in response to the presented application images 271. An application response 281 may be, for example, closing a switch on a mouse when the mouse pointer is placed on top of the mouse. a "Click here" button on the visual display device 210. The processor application 230 also determines the context duration time parameter of the received data packets 231. As discussed above, the context duration time parameter defines that both executable code 114 can be expected to be in context with the images 212 that are being presented on the video display device 210. The application processor 230 communicates a termination command to the execution processor 270 which performs a termination of the executable code 114. In a preferred embodiment, the application processor 230 communicates the termination order to the execution processor 270 at the same time as it communicates the execution order. In this embodiment, the termination order contains the duration time parameter of the context, and the execution processor 270 performs a countdown process based on the context duration time parameter, and terminates the executable code 114 at the end of the duration of the context. In an alternative embodiment, the application processor performs the countdown process based on the context duration time parameter, and issues a termination order at the end of the context time duration. In response to this termination order, execution processor 270 terminates execution of executable code 114. In an alternative embodiment, if application responses 281 are received in response to executable code, execution processor 270 delays completion of executable code. 114 until the user completes the transaction. In this way, for example, the user's selection of "Click here to buy" extends the duration of the executable code beyond the context time of the associated video segment, to allow the user enough time to enter the appropriate credit card and send information. In this embodiment, the execution processor 270 notifies the application processor 230 when the executable code 114 terminates. After the executable code 114 terminates, the application processor may place subsequent executable code segments 114 in the area of the memory 260 which were contained in the finished 114 executable code segment. Although presented here as a system to execute a single executable code, the encoder 100 and the decoder 200 of this invention allow the concurrent execution of multiple executable code segments 114, each associated with a specific segment of video images. Each sequence of data packets 131 corresponds to an executable code segment 114 uniquely identified. Each launch order and termination of the application processor contains an identifier that identifies the particular executable code segment 114 to be launched or terminated. In a clear implementation, for example, each active executable code segment 114 is identified by its start address in memory 260, and the launch and termination commands, each of which contain the start address of the executable code segment. . FIGURE 2 illustrates an exemplary transport flow. The packets VI, 1, VI, 2 and VI, 3 represent a sequence of video packets 121 corresponding to a segment of video images 112. The packets Dl, l and DI, 2 represent a sequence of data packets 131 corresponding to to a segment of executable code 114. The packet D2, l represents the start of a sequence of video packets 121 corresponding to a second segment of video images 112. The packet D2, l represents the start of a sequence of packets of data 131 corresponding to a second executable code segment 114. In accordance with this invention, the data packet sequences 131 each include a duration time parameter of the context; as illustrated in FIGURE 2, those context duration time parameters 330a, 330b are contained in the first data packets Dl, l and D2, l of the sequence of data packets corresponding to each executable code segment. Conventionally, the sequences of the video packets 121 each contain a presentation start time 310a, 310b which identifies when each of the sequences are to be presented. As discussed above, the data packet sequences 131 may also contain a context start time 320a, 320b that is based on the presentation start time 310a, 310b. Because the presentation start time 310a, 310b is typically not significantly different from the time in which the video packets are actually received in the decoder 200, the context start time 320a, 320b may be omitted, provided that the execution of the executable code associated with the received video packets start immediately after the received executable code 114. FIGURE 3 illustrates a timing diagram corresponding to the presentation of the exemplary transport stream of FIGURE 2. In this example, each video pack VI, 1, V2, 2, etc., corresponds to a video information box , and the frames are presented in equal frame periods 350. The first frame of each sequence of video packets VI, 1, and V2.2 begins at the specified presentation start times 310a and 310b, respectively. Each subsequent video pack VI, 1, V2.3, etc., occurs in periods of subsequent frames 350. Execution of each executable code segment 114 occurs at context start times 320a and 320b, which can be encoded expressly in the sequence of data packets 131 or be involved by the occurrence of the data packets in the transport stream between the sequence of the video packets of their associated segment of video images, as discussed above. As illustrated in FIGURE 3, the context start time 320a, 320b is correlated with the corresponding display start time 310a, 310b, but may occur before, after, or coincide with the presentation start time 310a, 310b . The duration time of the context 330a, 330b defines that both of each segment of the executable code 114 remains in the context with the presentation of the corresponding segment of video images 112. As with the start time of the context, the duration time of the context 320a, 320b is correlated with the duration 340a, 340b of the presentation of the corresponding segment of the video images 112, but may be shorter, longer, or equal to the presentation duration time 340a, 340b. As illustrated in FIGURE 3, the time period of duration 360, both of the first and second segments of executable code are defined in the context with their corresponding video image segments, even though the video image segments do not overlap. over time. Corresponding to this timing diagram, the application processor 230 loads the first executable code segment in the memory and communicates a "throw executable code segment 1" to the context start time 320a to the execution processor 270, then it loads the second segment of executable code in the memory and communicates a "launch segment of executable code 2" to the start time of the context 320b. At a point in time 335a that is equal to the duration time of the context 330a after the context start time 320a, the application processor 230 communicates a "terminate executable code segment 1" to the execution processor 270. At a point at time 335b which is equal to the duration time of the context 330b after the context start time 320b, the application processor 230 communicates a "terminate executable code segment 2" to the execution processor 270. FIGURE 4 illustrates a diagram Exemplary flow rate for the encoding of executable code segments having associated video image segments according to this invention. In 410, the start and duration of the presentation of the video image segment ends. At 420, the video segment is encoded, as well as the time to start presenting the video segment. At 430, the time parameters of the context, start and duration are determined, based on the start times and duration of the presentation. At 440, the executable code segment, if any, is encoded, as well as the context parameters of time. As noted above, explicit encoding of the start time of the context is optional, provided that the appropriate start time for the executable code can be implicitly determined, for example, by the occurrence of the data packets between the corresponding video segments. in the transport flow. The encoded video segment and the executable code segment are multiplexed for the transport stream, at 450, and the transport stream is transmitted to the receiving system, at 460. This process 410-460 is repeated continuously. FIGURE 5 illustrates an exemplary flow chart of the decoding and execution of executable code segments that have video image segments associated in accordance with this invention. The transport stream is received, at 510, and demultiplexed into a video segment and an associated executable code segment, if any, at 520. The start time of the presentation of the video segment ends, at 530, and the presentation of the video segment begins at this start time, at 540. If an associated executable code segment is present, the parameters of the context time are determined from the received sequence of data segments at 550. The execution of the executable code begins at the start time of the express or implied context, at 560, and the descending count is started to affect a waiting period that corresponds to the duration time parameter of the context, at 570. After a period of time corresponding to the parameter of the duration time of the context, execution of the executable code ends, at 560. As discussed above, in a preferred embodiment, the termination of the executable code will be postponed to 560 if the user is involved in a transaction that depends on the segment of executable code that is currently running. Note that the termination of the executable code in 560 does not depend on the presentation of the corresponding segment of the video images in 540. In this way, if the sequence of video packets corresponding to the segment of video images is interrupted, the executable code which corresponds to the incomplete segment of the video images will end within a finite period of time after the start of the executable code. The foregoing merely illustrates the principles of the invention. In this way, it will be appreciated that those skilled in the art will be able to devise various arrangements which, although not explicitly described or shown here, incorporate the principles of the invention and are thus within their spirit and scope. For example, a maximum duration time parameter of the default maximum context can be defined, and the encoded context time duration parameter can be this maximum time per deletion. In this way, for example, a particular sales program can define a maximum duration time constant for each segment of executable code, because each product will be presented during the same period of time. By using the parameter of time of maximum duration by default, the coding of the presence of data packets can be simplified, because it is known that the context time of each product is constant. It will also be appreciated by one skilled in the art that the transport stream encoder 100 and the transport stream decoder 200 can be implemented without physical computing components., programs and programming systems, or a combination of both. For example, the video encoder 120 may be a dedicated computing physical component device, although the data encoder 130 and the multiplexer 140 may be the code of the fixed instructions that are executed in an embedded microprocessor. Similarly, the distribution of the functions within the encoder 100 and the decoder 200 need not correspond to a repartition of the devices within the encoder 100 and the decoder 200. For example, the multiplexer 140 and the transmitter 150 can be a single device of transmission, and the receiver 250 and the demultiplexer 240 can be a single receiving device. Similarly, the execle code segments 114 may consist of a plurality of individual and independent or dependent execle code segments. Those and other optimization and distribution options will be apparent to one skilled in the art and are within the spirit and scope of this invention.

Claims (18)

CHAPTER CLAIMEDICATORÍO Having described the invention, it is considered as a novelty and, therefore, the content is claimed in the following CLAIMS:

1. A transport flow encoder, characterized in that it comprises: a video encoder that encodes video images in a sequence of one or more video packets, the sequence of one or more video packets has a determinable duration of presentation, a data encoder encoding the executable code in a sequence of one or more data packets, the sequence of one or more data packets contains a context duration time parameter that is based on at least one of: duration time of presentation, and a predetermined maximum duration time. The transport stream encoder according to claim 1, characterized in that: the sequence of one or more video packets also has a determinable presentation start time, and the sequence of one or more data packets also includes a context start time that is based on the presentation start time. 3. The transport stream encoder according to claim 1, characterized in that it further includes a multiplexer, operatively coupled to the video encoder and the data encoder, which combines the sequence of one or more video packets and the sequence of one or more data packets in a transport stream to facilitate the transmission of the transport stream to a receiver. 4. The transport stream encoder according to claim 3, characterized in that the transport stream is a transport stream that complies with the MPEG. The transport stream encoder according to claim 3, characterized in that it further includes a transmitter that transmits the transport stream to facilitate the reception of the transport stream in a transport stream decoder. 6. A transport stream decoder, characterized in that it comprises: a demultiplexer that receives a transport stream and produces therefrom a sequence of one or more video packets and a sequence of one or more data packets, an application processor that processes the sequence of one or more data packets to produce an executable code, a memory, operatively coupled to the application processor, which stores the executable code, and an execution processor, operably coupled to the application processor in the memory, which executes the executable code in response to a launch command from the application processor, and terminates the executable code in response to a termination command from the application processor, and where the sequence of one or more data packets contains a parameter of the duration time of the context, and the application processor communicates the termination order to the execution processor of depending on the parameter of the duration time of the context. The transport stream decoder according to claim 6, characterized in that the execution processor produces application images in response to execution of the executable code, and the decoder of the transport stream includes, in addition: a video processor that processes the sequence of one or more video packets in video images, and a video display device, operably coupled to the video processor and the execution processor, which displays the video images and the application images. The transport stream decoder according to claim 7, characterized in that the sequence of one or more video packets has an associated display duration time that is substantially correlated with the context duration time parameter. 9. The transport flow decoder according to claim 8, characterized in that the sequence of one or more video packets has a determinable display start time, and the application processor communicates the launch command to the execution processor to execute executable code at a launch time that is substantially equal to the presentation start time. The transport stream decoder according to claim 8, characterized in that: the sequence of one or more video packets contains a presentation start time, the sequence of one or more data packets contains a start time of the In this context, the video visualization device presents the video images at the start of presentation time, and the application processor communicates the launch order of the execution processor to execute the execution of the executable code at the start time of the context. The transport stream decoder according to claim 6, characterized in that the transport stream is a transport stream that complies with the MPEG. The transport flow decoder according to claim 6, characterized in that it includes a receiver to facilitate the reception of the transport flow of a transport flow encoder. 13. A method for encoding a transport stream, characterized in that it comprises the steps of: encoding video images in a sequence of one more video packets, determining the duration time of the presentation associated with the sequence of one or more packets of video video, and encode the executable code into a sequence of one or more data packets, the encoding of the executable code includes an encoding of a context duration time parameter that is based on at least one of: the duration time of the display, and a predetermined maximum duration time, and multiplexing the sequence of one or more video packets and the sequence of one or more data packets to form the transport stream. The method according to claim 13, characterized in that it further includes the steps of: encoding a presentation start time in the sequence of one or more video packets, and encoding a start time of the context that is based at the start time of presentation in the sequence of one or more data frames. The method according to claim 13, characterized in that it further includes the step of transmitting the transport stream to facilitate the reception of the transport stream in a transport stream decoder. 16. A method for decoding a transport stream, characterized in that it comprises the steps of: demultiplexing the transport stream and producing therefrom a sequence of one or more video packets and a sequence of one or more data packets, processing the sequence of one or more data packets and producing from it an executable code and a time parameter of duration of the context, executing the executable code for a duration time that is based on the duration time parameter of the context. The method according to claim 16, characterized in that the step of executing the executable code produces application images, and the method further includes the steps of: processing the sequence of one or more video packets to produce images of video, and present the video images and ~ application images on the visual representation device. 18. The method according to claim 16, characterized in that it also includes the step of receiving the transport flow to facilitate the reception of the transport flow of a transport flow encoder.