KR20020041828A - Method and system for active modification of video content responsively to processes and data embedded in a video stream - Google Patents

Abstract

Video or other media data is synchronized with procedural data streams that change the video stream. When the video is played or rebroadcast, the video is changed by the procedures defined in the procedural data stream. In an unconditional embodiment, the procedural stream contains commands that are executed immediately upon receipt by the interpreter. In a particular embodiment, the procedural data stream is directly included in the media data stream and output separately by a demultiplexer. In another particular embodiment, the combined medium / procedure data stream may be divided into portions while transmitting the appropriate code to the portion that has been erased in its entirety.

Description

Method and system for active modification of video content responsively to processes and data embedded in a video stream

Any parts of the video content are dissatisfied, and the rest of the video content may be considered to be a favorite. For this reason, it has been proposed to include markers in video data to mark parts of the audience that do not seem appropriate. The application running at the transmission location recognizes the markers in the video stream and selectively mutes or deletes passes in response. For example, PCT application WO98 / 21891, published May 22, 1998 under the heading "INHIBITION OF A TV PROGRAMME DISPLAY ACCORDING TO THE CONTENTS," is a part of a video stream that is marked as containing an unfavorable subject. Blanking out for deleting them is described.

Application No. GB 2 284 914, filed December 18, 1993 under the heading "DISCRETIONARY VIEWING CONTROL," is incorporated into a television program based on time, program identification, program rating, etc., in accordance with the conditions defined at the point of transmission. It describes a system with constraints. That is, the video signal is deleted or prohibited when certain conditions are given.

Application WO 83/02208, published June 12, 1983 under the heading "METHOD AND APPARATUS FOR EDITING THE OUTPUT OF A TELEVISION SET", filters content based on markers in a video and executes it in a television set-top box. Describes a system for filtering applications. Markers inserted in the signal are classified according to their contents. UK application GB 2315175, filed July 10, 1997 under the heading "RESTRICTING ACCESS TO VIDEO MATERIAL", describes a system in which video content is permitted to be transmitted or reproduced in accordance with its characteristics. Application filtering is predetermined with the nexus between transmission and recording, or viewing.

US Patent No. 5,751,335, filed February 23, 1996, entitled "VIEWING RESTRICTING METHOD AND VIEWING RESTRICTING APPARATUS," describes a system for muting a TV program if it exceeds the permitted rating.

US Patent No. 5,778,135 to " REAL-TIME EDIT CONTROL FOR VIDEO PROGRAM MATERIAL " describes a system for segmenting video and transmitting the segmented segments. The application outputs segments that are sent higher than the selection level. This is the same technique used in the PCT application WO 96/41438 of "ENCODER APPARATUS AND DECODER APPARATUS FOR A TELEVISION SIGNAL".

Deletion of video content responsive to content indicators of a video stream is already known. In conventional systems, some of the video is stopped by simply prohibiting the video display at a predetermined interval. In addition, applications that implement such blocking are words that are used in connection with a procedure or process, although markers in the video stream (ie, "specific codes"-the specification and the word "code" in the prior art are related to markers or indicators. And "code" must be identified based on the content). In addition, there are other kinds of software and computer games that selectively play video sequences according to the execution path resulting from user interaction with the software. For example, there may be several replacement video sequences that may be played during game execution depending on the choices made by the user.

What is needed is a mechanism to allow a creator of video content to control features related to the selective display of video content. Current techniques limit this control to the degree of sophistication provided at the intermediate control or transmission point.

The present invention relates to video systems in which video content is actively changed in accordance with a condition at a transmission site, a user's preference, or other condition, and specifically includes procedures for controlling such a change, or It relates to synchronous systems.

1 is a diagram describing a user environment in which the present invention is used.

FIG. 2 illustrates an embodiment of the invention in which multiplexed video data from a source extracts and decodes data defining a procedure, the procedure being executed to change the video in response to a profile.

FIG. 3 illustrates an embodiment of the present invention wherein demultiplexing video data from a source extracts data defining a procedure, wherein the compressed file modified by the procedure is executed in response to the profile and the modified compressed file is decoded. The figure which shows.

4 illustrates an embodiment of the invention in which the data demultiplexing from the source and data defining the procedure are obtained from an independent source and the video file is executed to change the video in response to the profile.

5 is a diagrammatic representation of a video file to describe features relating to any embodiments of the present invention.

Various techniques can be used to allow real-time change of the video stream. According to the invention, the way in which these changes are made increases the possible range of changes. It also increases the control that the creator of the video stream can take regarding the features provided to the receiving end. These advantages are provided by associating each video stream or file with one or more software procedures executed by a display or other generator, such as a copy station (e.g., storage station, transmission station, recording station, broadcast station, etc.). do. The generator is implemented to receive both the original video stream and the code defining one or more procedures to modify the video stream in any way. For example, in one embodiment, the generator is a television. According to this embodiment, the television receives a video signal with an embedded procedural code. The television has an internal controller that separates the software data from the original video data, executes the software data, and then changes the video data. For example, the software data may be included in the video blanking interval (“VBI”) of the analog video signal or may simply be included in a header file attached to the video file. In this embodiment, the internal controller may be programmed using an application program interface (“API”) that provides a set of functions that the procedure accesses to produce various effects. This is java System or improved Java Can be. Software data defines a procedure that is executed to change the video data. The procedure is keyed with visual or segment markers on the video data so that the procedure can identify portions of the video data that are to be changed.

This API may provide a rich feature set or a lean feature set. It may also be written at a high or low level. For example, the API may provide the ability to draw an object, such as a flat rectangle or a smooth ellipse of a particular color, over a particular area displayed only at certain time intervals of the video. These functions can obtain arguments in the procedure specifying the object, size and shape, color, and coordinates of the start and stop time segments. Another example is to apply a specific filter to a portion of the screen. The filter mask can be supplied as an argument.

By providing a set of multiple functions, or one of the basic functions that can be accessed and implemented in a variety of different ways, according to the present invention the video content generator provides many features and options regarding the distribution and use of video content. do. One result is that the available features are not limited to some set that is predefined in a transmission device or an output device (eg, a television) as in the prior art. Multiple sets of functions or basic functions of more built-in features may all provide the same degree of flexibility. Both may be provided.

At an abstract level, the present invention provides the video itself with an association of executable procedures for modifying the video data. The association may be provided by the supply of procedures for processing the video to substantially synchronize the presentation of the video on display processing equipment that ultimately converts the multiplexed or compressed or coded signal into a video data stream. Associations may be provided by mounting procedural code in the same or related file. Other embodiments may generate associations by supplying interleaved codes in the interleaved form to the video data stream, whether analog or digital.

As mentioned above, the terms "procedure" and "code" and other terms relate to definitions of processes or potential processes that broadly include both review definitions and procedural definitions. Therefore, by this meaning it is not limited by expressing the present invention regarding algorithms as above. The present invention includes languages of mapping driven types, object oriented languages, and the like.

In the procedures defined for each video, the range of possible changes, conditions, rules, criteria and possible options are too large to provide a comprehensive list. This is a significant advantage. In addition, these possibilities need not be known when the display (recording, broadcasting, etc.) device is deployed. Thus, video content can be delivered with its feature upgrades. Improvements in content are possible without requiring software changes in the display (or other) device. The number of changes and their types are infinite. The following are some examples to illustrate.

The video may be shipped into a plurality of language tracks selected according to the user profile accessed by the procedure.

The procedure decrypts the video using the password and profile data entered by the user.

The procedure applies a blur filter to portions of the frame during movie showing to hide the front naked body.

The procedure provides a control console that allows the user to increase the display speed of the video in accordance with the user's preferences entered into the console generated by the procedure.

The procedure provides a low resolution image, and authorizes data display payment authorization in that full resolution video can be viewed.

The procedure recognizes video signal parts based on pattern recognition and parts containing censored material and omits them by skipping frames to make the playback speed very fast.

The procedure, for example, expresses profanity and omits the sound track segments indicated by the markers of the video signal.

Techniques not previously known are transmitted in the video, for example in the procedure of responding to the user profile in any particular way or giving the user some choices.

Procedure for video or flash Provides an animation text overlay at the top of the video.

The procedure retrieves the commercial from the website and displays the commercial at regular intervals during the video broadcast.

The procedure also reduces the number and duration of the commercial by providing the user with the medium providing the commercial by approving the payment for viewing the same video as the shareware that displays the banner commercial until recorded.

The procedure controls playback rights so that various licenses that can be exercised by the user can be controlled by the machine's profile file.

A common feature of all the above examples is to enjoy the benefit of such a feature rather than requiring the program associated with the file to be displayed or available on a display or playback device. According to the present invention, the creator or distributor of a video can control the video display or other usage with considerable flexibility.

In an embodiment, the procedure consists of commands for operating the separable portions of the video stream. There is no condition that an execution environment can always be copied using any applicable number of portions of these portions. Thus, if no information persists over the segmentable portions of the media stream when the appended procedure is not feasible ("persistent" is a new expression meaning "persisted"), and the information is not found Generates the stream itself. In order to prevent these separable parts from splitting themselves, the environment used to copy a part of the program must take into account the possibility of splitting that part. One way to not interfere with the possibility of splitting is to encode each separable portion block by block so that the block is not read every bit. One example of a segmentable portion may be a video segment that filters a portion of the video image in any way, for example, to mask portions of the naked scene. The procedure can be performed to apply a mask and to turn on many frames, and to turn off the generation of that mask. The video portion between the turn on and turn off commands should not be at least split into turn on commands so that they are not operated before the sensitive subject.

Another way to implement the present invention is to ensure that every frame of video includes own state-generating procedure code. This environment is also unconditional. Next, any number of frames to be copied will contain the appropriate code to apply the correct attributes to that frame. In the above example, when the filter is applied to the portion of each frame of the sequence, the filter applying command, the definition of the filter, processes each frame. In this embodiment, the environment is unconditional between frames. This embodiment can be used for a broadcast model. In addition to the procedures, information about the video may be encoded into the procedure data. For example, a title, author, description, and the like may be included so that any copied video sequence may include overall information about the video file from which the segment occurred. Such data cannot be stored for each frame, but may be distributed in multiple frames.

The invention has been described in connection with any preferred embodiment with reference to the following technical drawings in order to be more clearly understood. Referring to the drawings, the depicted features are provided as an example for technically discussing preferred embodiments of the present invention, and are provided in order to make the description of the principles and concepts of the present invention most useful. In this regard, no attempt is made to show the structural details of the invention in more detail than is necessary to a fundamental understanding of the invention, and the techniques employed by using the drawings illustrate, to those skilled in the art, how some aspects of the invention actually Clarify if it can be embodied.

Referring to FIG. 1, one example of a physical internal configuration capable of supporting the functional features of the present invention includes a computer 140 and a view system 100 with various types of input and / or storage devices. The latter is a keyboard 112, a remote control 110, a removable medium 120 such as a floppy disk, an optical disk, a memory card, Philips Fronto , Programmable controller, voice recognition / active controller, mouse, gesture-recognition controller, and the like. Data may be stored locally on the fixed disk drive 135. Output devices may include a monitor or TV 130, speakers 131, and / or other output devices. Computer 140 (ie, operable as described above) receives data 160 and / or video 170 from an external source, which may include broadcast transmission, data storage, the Internet, network, satellite transmission. , Switched circuit transmission, or any other source of data or other signal. With respect to the term "computer", it is understood to mean a set top box, an embedded system, a storage device with a controller, or any digital device capable of carrying out functional requirements. The computer 140 is embedded in files stored in the data storage device 135 or inserted into the data 160 and / or video 170 received from an external source, or transmitted to the computer in the form of a data file. Carry out possible procedures. The procedures change the video into either compressed or decompressed form. After the change, the video may be stored in the VTR 133 or transmitted to the wireless transmitter 137 as a broadcast, or displayed on the TV or monitor 130. The input and output devices shown are only examples.

In another embodiment of the present invention, data 160 and video 170 are transmitted by two different transmitters. Also in another embodiment, data 160 is distributed by a plurality of transmitters, while corresponding video 170 is transmitted by a single transmitter. In this embodiment, the video is distributed throughout, for example, while the data is distributed locally. This makes it possible to provide different procedures for video in different areas. In this embodiment, the range of transmitters that transmit video 170 is greater than the range of transmitters that are part of multiple transmitters that transmit data 160.

In the previous two embodiments, protection must be taken in synchronization with data 160 and video 170 so that the procedures defined by the data are applied correctly to the corresponding portions of the video stream.

Referring to FIG. 2, in an exemplary embodiment, the computer 140 may include a cable, microwave, satellite, or other broadcast transmission 180, a computer such as a laptop 185, a network 190 such as the Internet, and data storage. Receives a video file from device 195, or from any other source that may be any other source of analog and / or digital data. In addition, they may include smart mobile phones, PDAs, and the like. In the present embodiment, the received data is a video stream. The video stream is received by demultiplexer 205 that separates the video stream into an active video procedure data stream and an original video stream. The former is applied to the active stream engine 225 and the latter is applied to the decoder 210 (if it is necessary to decode the compressed video format). The output of the decoder 210 is applied to the processing device 215 for checking the profile stored in the computer 140. Profiles store data that characterize the audience. If the profile matches the current video, the procedure is applied to the processing device 225 in response to the profile data to generate a modified video stream. If the profile does not justify the change of the video stream as such, the original decompressed video is output. The output stream is applied to the output device, which can be any of a variety of different sinks. For example, the output device may be a broadcast transmission 180, a computer 185, a TV or monitor 131, or a data storage device 195. The output devices may also include a VTR as shown in FIG. 1, with the example shown in FIG. 2 being merely an example.

The demultiplexer receives an analog or digital signal. An example of an analog signal is an NTSC signal from television broadcast. In this case, the common place to place the data is in the VBI, where the demultiplexer can extract the data in the VBI from the original video stream and apply it to the active stream engine 225. The active stream engine simply executes the procedure provided to it. The active video procedure consists of more code that can be packaged in a single VBI, in which case the active stream engine 225 is programmed to obtain the entire procedure, so that the data preceding the delimiter is a file marker indicating that the procedure has been executed. Or in the normal form as at the end of other delimiters. Any suitable protocol is defined to accumulate procedures in the computer's memory before the video segment to be applied is reached. If the video data file is digital, the procedural data may be packaged as a header or interleaved in a data file or any other suitable method. If it is stream data, the procedure can be sent in a header file or in small packages so that the video is buffered so that playback can start immediately without waiting for the entire procedure or set of procedures to be loaded, and the procedure (s) May accumulate outside the prescribed time. The latter accumulation configuration assumes that the video to be applied was not loaded before the procedure (s) were loaded. Alternatively, the procedure may be distributed through a video file and executed by an interpreter running on computer 140. (An interpreter is a program that immediately executes instructions, such as the command line of an operating system cell based on text such as MSDOS or the command mode of a database program such as dDase III, when received without prior editing.)

Once the procedure has accumulated in memory, it can be executed in accordance with the indicators and profile data of the video file. Referring to FIG. 5, indicators of a video file or data stream may take various forms. Some different examples are described in FIG. 5 showing file or stream media data 501 with time to advance in the indicated direction. The audio sequence Au ₁ can be provided as a marker, in which case the sound classifier can be operated on the audio track until any feature is detected. Likewise, images Img ₁ , Img ₂ or other signal fractions can be recognized to identify a portion of the video stream. Subimages SI of frame image 510 may also be classified to trigger an event.

Markers such as M ₁ , M ₂ , M ₃ can be recorded in a file. In analog files, such as NTSC, markers can be placed in the VBI. The time since the start of the data start point can be tracked and used to indicate portions of the video stream, such as time separators T ₁ , T ₂ . In other words, the procedure 500 may be inserted into the video stream before the appearance of the portion of the stream to which it applies. For example, procedure 500 can be applied to a sequence distinguished by T ₁ , T ₂ , but not to what is represented by M ₃ (note that time is rolling up the page).

Indicators are not needed if the instructions are executable immediately upon receipt. One form of indicator is a simple place marker. In an analog stream, the marker may take the form of a watermark, or in the form of an icon that can be recognized as a portion of a video image or data multiplexed with VBI. In the digital stream, the marker may be any appropriate symbol and temporary position indication. Of course, in digital embodiments, the marker need not necessarily occupy a position in the stream consistent with the application of the procedure, but may not.

Decoder 210 may be a processing device for decompressing, decrypting, unpacking, unbundling, or executing any other prescribed processing that requires access to video data. The details of this processing apparatus are not critical to the embodiment of the present invention.

The profile may simply include information about the identification of the user, preferences of the user or group of users (eg, family), or any variety of data. For example, the profile may indicate that the user group is a family with very young children. The active procedure may query the user before displaying a very violent or sexual topic, and in the case of non-response, may hide or delete a topic that is potentially disagreeable. The profile database contains subject preferences that use the procedure to filter the set of selectable attributes. For example, suppose a video file contains many different video files that are all aggregated so that a particular person can be seen. The profile filters them, displaying only one or a plurality of things to be selected during viewing.

As mentioned above, in an embodiment, the procedure (s) consists of commands that operate the separable portions of the video stream. The execution environment may be copied using any limited number of such portions at all times using applicable procedure (s). No information persists between the segmentable parts of the execution environment if the attached procedure is unable to process the information or if the procedure can generate the information itself. In this frameless unconditional embodiment described above, the demultiplexer generates consecutively received commands. The commands are executed as soon as they are demultiplexed or keyed into markers or unique indicators of the video stream.

Referring to FIG. 3, in another embodiment, an active procedure is applied to the compressed video stream. Decoder 310 is done only after the active procedure is applied to the original video stream. In this example, the video is described as being compressed, but can be encrypted, bundled, or otherwise encoded. In other words, although not shown, the profile data can be supplied to the active stream engine to perform a procedure in response to the profile data. In FIG. 4, the active procedure may be sent or supplied in parallel to the active stream engine 425. For example, in a broadcast environment, parallel transmission may occur and video may change accordingly. In other words, synchronization can be ensured by keying execution on markers or indicators in the video stream. In this example, such keying is advantageous for the short life of the procedure code. The video is always updated according to the procedures most recently supplied by the source of the active procedure.

In an environment where unconditional demands are not critical, the active procedure data may be placed at locations independent of the portions of video being applied. However, one requirement is that the streaming source, such as a TV broadcast or Internet streaming file, must be loaded before the procedure is required. The procedure can be broken down, but all of that must accumulate in memory before the procedure is required. Next, the procedure code can be dumped. Code and events that trigger a dump may preferably be encoded within the procedure itself.

A code defining procedure is not necessarily high level and includes elements that define complex predetermined procedures or, in the low level, elements that define small incremental procedures that must be assembled to execute useful actions. do. The following are illustrative examples of the various types of commands that may be executed by an API suitable for changing the media data stream.

Block b0 to b1 playback: Play a series of video blocks of blocks b0 to b1.

Line (x1, y1, x2, y2, W, C) Draw: Draws a line overlapping from the indication coordinates, together with the indication weight and color.

Lines (x1, y1, x2, y2, W, C, F) Draw: Draws an overlapping rectangle from the indication coordinates, together with the indication border weights, color and fill.

Filter [(c11, c12, c13, c21, c22, c23, c31, c32, c33), x1, y1. x2, y2] Apply: Apply a filter to the zone formed by the specified matrix on the designated area.

Include video segment at path: //filename.vid: Stops video to form a replacement stream and insert that replacement stream.

It is apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, but may be embodied in other specific forms without departing from the spirit or essential attributes of the present invention. The present embodiments are to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes that occur within the meaning and scope of the claims. Are intended to be included within the scope of the claims.

Claims (23)

A method of changing a media information stream, Including in the media information stream (500) a definition of a procedure (500) to be applied to modify a portion of the media information stream; Implementing the procedure at the time of reproduction, recording, or retransmission (140, 133, 130, 137, 135) of the media information stream. The method of claim 1, wherein the procedure comprises a plurality of steps, each step being included in the media information stream adjacent to a portion of the media information stream to which each step is applicable. The method of claim 1, wherein the implementing comprises changing a medium information portion of the media information stream. The method of claim 1, wherein the implementing comprises providing a command to an interpreter of a computer at the time of the playback, recording, or retransmission. 2. The method of claim 1, wherein the implementing step includes inputting a command to an interpreter, the interpreter being programmed to execute the command as soon as the command is entered, and the interpreter being at the time of the playback, recording, or retransmission. In the method, which runs continuously on the computer. 6. The method of claim 5, wherein the interpreter's execution environment is unconditional in that no variables persist from one command to the next. 6. The method of claim 5, wherein the execution environment of the interpreter is adapted to change and is conditionless in that no variable persists from one set of commands before the video segment to the next set of commands. A method of changing a media information stream, Instructing a procedure and synchronizing at least one command residing within the data stream with respect to the media information stream; Invoking the procedure in response to the command at the time of reproduction of the media information stream such that the procedure changes the media information stream; And wherein said synchronizing step is effective to cause said procedure to operate a designated portion of said media information stream. Streaming video data having tokens 500 disposed at various points of streaming video data, the tokens being interpreted by the interpreter in response to the command when the media information stream is applied to an interpreter. A media information stream that is enabled to change data. 10. The media information stream of claim 9, wherein the interpreter is not subject to each successive set of commands being executed at the start of the same virtual machine state as all other successive sets of commands. 10. The media information stream of claim 9, wherein modifying the streaming video data comprises applying a filter to at least one frame of the streaming video data. 10. The media information stream of claim 9, wherein the streaming video data also includes markers, and the commands are executable in response to the markers. 10. The media information stream of claim 9, wherein the commands are responsive to features of the streaming video data. In a broadcast system, A first transmitter 137 that is efficient for outputting a media information stream; A second transmitter 137 that is efficient for outputting a procedural data stream that is synchronous with the media information stream; The procedural data stream includes data indicative of at least one procedure that, when executed, is effective to modify the media information stream, And synchronizing the media information stream with the procedural data stream such that the procedures defined in the procedural data stream are applied to designated portions of the media information stream. 15. The broadcasting system of claim 14, wherein the second transmitter comprises a matrix of transmitters, each of the transmitters of the matrix having a range less than the range of the first transmitter. 16. The broadcast system of claim 15 wherein each of the transmitters in the matrix transmit different data. 15. The broadcasting system of claim 14, wherein an initial state of a state machine that defines a procedure corresponding to any sequence of commands in the procedural data stream and that is applied to consecutive portions of the media information stream is the same state. An apparatus for receiving and modifying a media information stream, A demultiplexer (205, 305, 405) having inputs for receiving the combined data stream and for outputting the media information stream and the procedural data stream; Controllers 225, 325, 425 having inputs and outputs, The controller is programmed to receive the media information stream and the procedural data stream and to modify the media information stream in response to the procedural data stream. 19. The method of claim 18, wherein the procedural data stream includes a plurality of commands, each command being included in the combined data stream adjacent to a portion indicating a portion of the media information stream to which each step is applicable. The procedural data stream is synchronized with the media information stream. 19. The method of claim 18, wherein the controller is programmed to generate an interpreter process that executes the commands of the procedural data stream to realize a state machine whose cycles are synchronized in response to the structure of the combined data stream. 19. The method of claim 18, wherein the controller is programmed to realize interpreter processing that is executed continuously so that the commands of the procedural data stream are executed immediately upon receipt by the controller. The method of claim 21, wherein the interpreter's execution environment is unconditional in that no variables persist from one command to the next. 22. The conditional apparatus of claim 21 wherein the interpreter's execution environment is unconditional in that it does not last from one set of commands before one video segment to the next set of commands applied to change any variables. Way.