JP2004513589A - How to switch from scanning content to playing content - Google Patents

Abstract

The data stream representing the content information is processed for playback. The processing of the stream is switched from processing at the first speed to processing at the second speed. The first speed is different from the second speed. Processing at the first speed is stopped, and processing at the second speed is started at a position in the stream that was passed during processing at the first speed. The position can be readjusted.

Description

[0001]
[Field of the Invention]
The invention relates to a method and a device for processing a data stream. The method of the invention can be used, for example, when reproducing content information from a data stream.
[0002]
[Background Art]
U.S. Pat. No. 4,649,442 discloses an automatic program selector for a video tape recorder. In the video tape recorder, each time a recording start signal is detected, the corresponding content is automatically reproduced. The recording start signal is a control signal indicating the start of a recorded program. By automatically playing back the content simultaneously with the detection of the signal, the user can check whether or not the played back program is the desired one. Thus, when a signal is detected in the fast forward (FF) or rewind (REW) mode, the magnetic tape is stopped to play back the beginning of the detected program over a predetermined time. If the user does not perform a key operation during the reproduction of the detected program, the FF or REW mode is automatically restarted.
[0003]
[Summary of the Invention]
Known selectors automatically play portions of the program without user intervention based on the recording start signal. The invention provides, inter alia, a method for controllably reproducing pre-recorded content information, which allows the user to determine which part of the content or program is to be played. The present invention provides, for example, a user-friendly method for retrieving a desired sequence or scene of a pre-recorded program. The invention further provides the user with the option of playing the content starting at a location as close as possible to the location selected by the user within the content.
[0004]
The present invention provides a method for processing a data stream. When the data stream is being processed at the first speed, for example, a user is provided with a command to switch from processing at the first speed to processing at the second speed. Processing at the first speed is stopped. Processing at the second speed starts from a position in the data stream that has already passed when the stream was previously processed at the first speed. The position at which to start processing at the second speed can be readjusted to take into account the example situation described below.
[0005]
The expression "data stream" shall in particular denote the concept of a linear continuation of data. The data stream may be processed in either the forward and reverse (or rewind) directions. Data streams can be processed at different rates.
[0006]
The present invention allows for switching from processing at the first speed to processing at the second speed, taking into account a desired starting point for starting processing at the second speed. The processing at the second speed need not necessarily be started from the point at which processing at the first speed in the stream was interrupted. The inventor of the present invention determines when the process at the first speed is actually stopped, and when the user gives a command for switching to the process at the second speed, and the user actually executes the process at the first speed. I realized that there was a delay between when I wanted it. The starting point in the stream for processing at the second speed is generally passed during the transition from the first speed to the processing at the second speed. The starting point in the stream is located in a direction opposite to the starting point of the processing at the first speed. The present invention seeks to compensate for this delay.
[0007]
The interval between the stopping point and the starting point may depend, among other things, on the user's profile (or user history), the user's habits, the user's reaction time, the magnitude of the first speed, or the data stream with respect to the interaction with the processing device. It may depend on the technical characteristics of the processing equipment. The spacing between the stopping point and the starting point may be adaptive, dynamically or in real time.
[0008]
The invention is used for the reproduction of content information obtained from the processing of a data stream in a recording device, for example a video or audio tape recorder, or a TiVo or Replay HDD-based video recorder. For example, the user scans content information that is played forward at high speed (forward fast forward). The user specifies a particular portion during fast forward and decides to resume playback at normal speed starting from that particular portion. The user sends a command to the device to stop the fast forward processing. The user intends to stop the fast forward process at the beginning of a particular part. However, if no measures are taken to compensate for the above-mentioned waiting time, the above-mentioned waiting time will overshoot this desired point. In the present invention, as described above, the start point is determined to be a point that is more advanced than the point at which the forward fast-forward processing was actually stopped. The information content can then be played forward at normal speed starting from the data preceding the fast forward stop. As a result, the information content is played starting near the point where the user originally wanted to switch from the fast forward mode to the normal playback mode.
[0009]
In an embodiment of the present invention, one or more locations in the data stream are previously marked by respective bookmark pointers. Instructing the system to rewind or fast forward and then start the process with normal playback begins at the point indicated by the particular bookmark pointer. This particular bookmark pointer precedes the position where processing at the first speed was stopped.
[0010]
As used herein, the term "bookmark pointer" is any information that indicates the beginning, and possibly the end, of a particular scene in a data stream ("bookmark pointer" is also abbreviated as "bookmark"). A bookmark is a label or a data item in a stream that can be embedded in a stream that can be recognized as a bookmark. Bookmarks may also be provided separately from the data stream or in combination with the stream. For example, a bookmark indicates the beginning of a scene or sequence in the content information, or a bookmark indicates a portion of interest or indicated by the user.
[0011]
The present invention is also useful for video-on-demand or audio-on-demand systems in which a remote server supplies a video or audio stream to a client over a data network. In addition to the above problems, there is a problem of network latency between the client and the server, and the present invention aims to solve this problem. The term "latency" shall refer to a significant propagation delay between cause and effect. In real-time interaction, network latency is a difficult problem to solve. In today's Internet environment, there is considerable latency, typically on the order of seconds. High speed analog modems introduce delays on the order of 30 ms, or 60 ms end-to-end. The effect of latency tends to be more severe as the size of the network increases and the network load increases. It is not possible to eliminate all network latency. The waiting time can be determined in various ways. One way is to send messages back and forth between devices communicating over a network and divide the round trip time by two. For example, on a Unix (R) machine connected to the Internet, "ping-s <Internet name of destination device>" is input, and a large number of round-trip packages are transmitted to the destination device to obtain round-trip time. This ping operation may be performed once, for example, when initializing the communication, or at regular intervals to adapt to dynamic changes in network congestion. The network latency determined in this way determines the point of jumping back to the stream when switching from processing at the first speed to processing at the second speed when the user issues an instruction. Is taken into account by the system.
[Detailed description of embodiments]
Hereinafter, the present invention will be described in detail by way of example with reference to the accompanying drawings. In the figures, same reference numbers indicate similar or corresponding features. FIG. 1 is a diagram showing switching of the processing of the data stream 100 from forward fast-forward to normal reproduction. Stream 100 is shown as a linear array in which data is processed sequentially from left to right for forward processing (forward fast forward and normal playback). Stream 100 is initially processed at a first speed 102 in a fast forward mode. If the user gives a command to stop forward fast-forward and start normal playback, the waiting time causes the forward fast-forward process to overshoot the data in which normal playback is to be resumed. The processing at the first speed 102 actually stops at the stop point 108. If the transition is made to the normal reproduction mode at the stop point 108, the user fails to reproduce the overrun period 110. Here, the processing of the stream 100 at the second speed 104, which is the speed corresponding to the normal playback mode, starts at the start point 106. In the present invention, the start point 106 is a point that goes back beyond the stop point 108. The distance between points 106 and 180 takes into account the overshoot period 110. As a result, a backward jump is required in switching the stream 100 processing from the first speed 102 to the second speed 104. The starting point 106 should be where the user wants the process to start at the second speed 104. The present invention involves determining a starting point 106. The backward jump is based on the overshoot period 110 and takes into account, among other things, the user's reaction time and system latency.
[0012]
FIG. 2 is a diagram showing switching of the processing of the data stream 100 from rewinding to normal reproduction. Stream 100 is first processed at a first speed 102 in a reverse fast forward or rewind mode. When the user initiates a request to switch processing of the stream 100 from rewinding at speed 102 to normal playback at a second speed 104, the processing in rewind mode depends on the waiting time. , And stops at a stop point 108. Accordingly, processing of the stream 100 at the second speed 104 should start at the starting point 106, which is a point beyond point 108 in the rewind mode. This requires a forward jump in normal playback mode. The jump is based on the overshoot period 110.
[0013]
FIG. 3 is a diagram illustrating a data stream 100 having bookmark pointers 142, 144, 146, and 148. Pointers 142, 144, 146 and 148 can be created in various ways, examples of which are described below. A bookmark pointer has been added to stream 100 to provide system control signals. The control signal allows the system to find the correct starting point to start playback when switching from fast forward or rewind to normal playback mode.
[0014]
The first example is based on detecting in a stream 100 the start of a new scene (or new data). When data stream 100 includes a video stream having consecutive encoded frames, bookmark pointers 412, 144, 146, and 148 may point to respective key frames. In this application, the expression "key frame" generally indicates a frame in the stream 100 that represents a portion of the content. See, for example, published international patent application WO 98/55942, which is incorporated herein by reference. A simple way to select keyframes is to detect video scene changes in the content information. A key frame representing a scene is, for example, the first frame of the scene. The scene can be further analyzed to select keyframes that are a meaningful representation of the scene. Each scene is further examined to find a key frame that is a better representation of the scene than the first frame.
[0015]
A second example is based on detecting a stream 100 with metadata attached. It is assumed that stream 100 includes, for example, a video stream with embedded closed caption subtitle metadata. Further assume that the user is interested in golf as indicated by the user's profile or history. When the word "golf" is found in the closed caption caption data, a bookmark pointer is created that points to the start of the corresponding scene.
[0016]
A third example is based on detecting a certain object or motion in the content information of stream 100. For example, data stream 100 includes episodes of popular television programs. At some point, a guest star enters the scene. A bookmark may be provided at that particular point in the video stream 100.
[0017]
A fourth example is based on the detection of certain sounds or words. For example, assume that stream 100 represents a movie with little conversation. At the start of the conversation, a bookmark may be provided by detecting audio in the audio data stream of the movie.
[0018]
A fifth example is based on bookmarks that indicate content information for a portion of the data stream 100 that a user is or will be interested in. In this context, see U.S. patent application Ser. No. 09 / 585,825, filed 6/100 by Eugene Shteyn, entitled "Content with booksmarks obtained from an audition's application," which is incorporated herein by reference. This document relates to a method of providing bookmarks to indicate elements or portions of content information that a person may be interested in. The creation of a bookmark is based, for example, on ratings that are registered while monitoring the test audience.
[0019]
A sixth example is based on providing a bookmark for the first frame of a GOP (Group Of Pictures). The data stream 100 includes a video stream, and possibly a series of GOPs that are MPEG2 compressed. The beginning of a GOP is generally I-frame coded and is indicated by reference numeral 150. For example, the data stream 100 is, for example, a video stream compressed by MPEG2. In a compressed video stream, the beginning of a new scene is generally encoded as an I-frame to increase efficiency. An I frame is the first frame of a GOP.
[0020]
Bookmarks may be created by, among other things, the creator of the data stream, the provider or broadcaster of the data stream, the user, or an automated method. The user creates a bookmark, for example, when he points to a desired element in the data stream. The user creates a bookmark for the position in the data stream corresponding to the desired element, for example, by a remote control. Again, latency and reaction times are taken into account by the system when adding bookmarks to the stream.
[0021]
Start point 106 may be a bookmark that is near stop point 108. The data stream 100 is desirably stored using a random access memory (RAM) medium, for example, an optical or magnetic disk or a semiconductor memory. The RAM minimizes the time it takes to switch from processing the stream at a first rate to processing at a second rate. The data stream 100 and the storage medium may be, for example, an MPEG2-stream of a compressed movie in the form of a DVD video disc, a compressed television broadcast stored on a hard disk drive (eg, TiVo personal TV), a flash memory (eg, Philips Rush). This is an MP3 compressed audio file in the MP3 player.
[0022]
The data stream may be stored on a non-random access memory medium, such as a magnetic tape, but may switch between a first processing speed and a second processing speed unless some random access buffer memory is available. May not be smooth. The bookmark may be part of the data stream structure (embedded in private data in the case of an MPEG stream) or another data structure (eg a content content file in the case of a DVD movie). In either case, the bookmark is provided before the transition from processing at the first speed to processing at the second speed. Such a bookmark is generally the newest found in the data stream being processed at the first rate.
[0023]
FIG. 4 is a functional block diagram showing the end-to-end content distribution system 400. System 400 includes a service / content provider 402, STB 406, display and speakers 408, a link 422 between provider 402 and STB 406 over network 404, and a link 424 between STB 406 and display and speakers 408.
[0024]
Provider 402 may provide content 412 to network 404 through its output on link 422. The provider 402 is, for example, a broadband cable operator, a DirectToHome (DTH) satellite operator, or an ISP. The content 412 is, for example, a media clip of a pay-per-view movie, a television broadcast, or a stream of a real network. The link 422 is, for example, a TV cable, a POTS telephone / xDSL line, a wireless link between a DTH satellite and a parabolic antenna. The network 404 is, for example, a data network such as a cable head end distribution system or the Internet.
[0025]
The STB 406 includes a microprocessor 416, a control interface 418 for the user, a local storage 420, an input for the link 422, and an output to the link 424. The STB 406 is, for example, a consumer digital cable receiver having a personal television function. The microprocessor 416 is, for example, a Philips Trimedia processor. Such a processor can perform such tasks as demultiplexing of the transport stream and MPEG2 encoding or transcoding. The local storage unit 420 is, for example, an HDD, a DVD + RW, and a flash memory. User control interface 418 allows a user to control through the STB. The storage 420 may include data with the properties described above for the data stream 100. The processing speed of the microprocessor 416 is controlled when the data supplied via the link 422 includes the data stream 100.
[0026]
The display and the speaker 408 are, for example, a television receiver and an audio amplifier set.
[0027]
STB 406 and display and speaker 408 are typically located at the end user's premises.
[0028]
The STB 406 and the display and speaker 408 may be physically coupled or their components may be distributed, for example, for cost or convenience reasons.
[0029]
For example, the provider 402 broadcasts a movie “Never miss it again” on the cable network 404. Here, the user referred to as "Never Home" is expected to be away from home during the broadcast, so the STB408 called "Philips super TV" was programmed to record "Never miss it again". Microprocessor 416 uses the data from the EPG and the built-in timer to know when "Never miss it again" will be broadcast. The microprocessor 416 controls the storage unit 420, which is a 100 GB HDD here, and controls recording of “Never miss it again” during live broadcasting. In this example, the data stream 100, which is "Never miss it again", is physically in the storage unit 420 after the broadcast. After returning to home, “Never Home” can search for the scene that he / she likes best among “Never miss it again” via the control interface 418.
[0030]
User control 418 provides trick play control for previously stored data stream 100 on storage 420. Stream 100 may be, for example, a favorite television program. For example, user controls 418 can receive and process remote control (RC) commands. RC commands may include fast forward (FF) playback, fast forward (FB) playback, and normal (1x speed) playback of a television program. Microprocessor 416 interprets the processed RC command and processes the stored stream 100, and thus the television program from storage. The result is a baseband AV signal output over a link 42, for example, an S-video connection to a television set.
[0031]
The inventor also assumes a user command such as "to the next bookmark" (or to the previous bookmark). With this command, the user can reproduce a program recorded in advance from the position after the next bookmark. This command is very useful for busy users who want to quickly scan previously recorded programs. For example, "Never Home" is only interested in the home run of the baseball finals last week. Thus, "Never Home" instructed "Philips super TV" to bookmark all home runs using voice-controlled input. “Philips super TV” uses the closed caption data of the final match to bookmark the home run. "Never Home" uses "go to next bookmark" to quickly procure information about the missing finals. "Never Home" can be used by users such as "Quick-skip" (jump forward for about a minute in pre-recorded programs) as implemented in playback as provided on the STB of personal TV. Will recognize it as an alternative to the command.
[0032]
The features of system 400 described above may reside in a microprocessor-executable program in STB 406. The program may be entered into the STB 406 at the same time as manufacturing or at a later stage. If the program is entered at a later stage, for example, by downloading over a network such as the Internet, the STB 406 preferably has a means to upgrade the program.
[0033]
FIG. 5 is a functional block diagram showing the end-to-end content distribution system 500. System 500 includes service / content provider 502, STB 506, display and speakers 408, link 422 between provider 502 and STB 506 over network 404, interactive link 520 between provider 502 and STB 506 over network 404, and , STB 506 and a link 424 between the display and speaker 408. Links 422 and 520 may be physically the same.
[0034]
Provider 502 may provide content 512 to network 404 through its output on link 422. The example given above for provider 402 may be the same for provider 502. An example of the content 412 is the same as the content 512. Microprocessor 510 allows provider 502 to play the content at different speeds. Interactive link 520 may be used to transmit the required speed to microprocessor 510 to play content 512. Data stream 100 is part of content 512.
[0035]
STB 506 is similar to STB 416, but with some significant differences. The STB 506 can control the playback speed of the data stream 100 even if the data stream 100 is not physically located in the STB 506. Instead, the microprocessor 516 remotely controls the playback speed of the data stream 100. Microprocessor 516 communicates using link 520 which connects to microprocessor 510.
[0036]
For example, provider 502 may be an Internet service provider or a video on demand (VOD) provider that allows the playback speed of its content to be controlled remotely. When the provider 502 receives the request for switching the playback speed, the network latency in the networks 520 and 422 is taken into account. This is done by microprocessors 510, 516 or a combination of 510 and 516.
[0037]
The STB 506 may be a Thin Client STB with space, power and / or cost constraints. A good example of this STB 506 can be a web-pad device STB, which may have a built-in display and speaker 408. The user can carry the device to a desired location and play the desired content. In this case, the connections to the provider 502, ie, the links 422 and 520, may be wireless, for example, based on Home-RF, IEEE 802.11 or Bluetooth. Provider 502 may be a home server at the user's home, which may then be connected to an external content provider (not shown in FIG. 5). The provider 502 may be a remote server using a wireless public network such as GSM or UTMS. When using these types of networks, the network delay is known. STB 506 may perform a “ping” operation on provider 502, for example, to determine network delay. Another method is to time-stamp a command to request the switching of the playback speed of the data stream 100 when the user requests it.
[0038]
The above-described features of system 500 may reside in a microprocessor-executable program of STB 506 and / or provider 502. The program may be entered into the STB 506 at the point of manufacture or at a later stage. If the program is entered at a later stage by downloading over a network, such as the Internet, the STB 506 preferably has a means to upgrade the program.
[Brief description of the drawings]
FIG.
It is a figure which shows switching of the process of the data stream from forward fast forward to normal reproduction | regeneration.
FIG. 2
It is a figure which shows switching of the process of the data stream from reverse fast forward (or rewind) to normal reproduction.
FIG. 3
FIG. 3 shows a data stream with a bookmark pointer.
FIG. 4
FIG. 2 is a functional block diagram illustrating an end-to-end content distribution system.
FIG. 5
1 is a functional block diagram illustrating an interactive end-to-end content distribution system.

Claims (16)

A method for switching from processing a data stream at a first rate to processing the data stream at a second rate, the method comprising:
The data stream represents content information,
The first speed is different from the second speed,
The above method
Stopping the processing at the first speed;
Starting the processing at the second speed at a starting point at a position in the stream that has passed during the processing at the first speed;
The method wherein the position of the starting point is readjustable. Determining the location of the starting point, the determining step taking into account at least one of a user's history, device latency, network connection latency, and information content of the data stream. Item 7. The method according to Item 1. The method of claim 1, wherein the starting point is indicated by a bookmark. The method of claim 3, wherein the bookmark is user programmable. 4. The method of claim 3, wherein the bookmark has been added by a third party. The content information includes a video,
The method of claim 3, wherein the bookmark indicates a scene change in the video. The data stream includes metadata associated with the content information;
4. The method of claim 3, wherein the location of the bookmark is dependent on the metadata. The content information includes a video,
The method of claim 3, wherein the bookmark represents an I-frame that is the first frame of a GOP encoded stream. The content information includes a video,
3. The method of claim 2, wherein the starting point is dependent on a user profile and an I-frame, which is the last frame of a GOP closest to where the processing at the processing speed stopped in the video. The method of claim 1, wherein the data stream is provided over a data network. An apparatus for enabling a user to switch from processing a data stream at a first rate to processing the data stream at a second rate,
The data stream represents content information,
The first speed is different from the second speed,
The above device
Stopping the processing at the first speed,
Responding to a particular user input by initiating processing at the second speed at a starting point at a position in the stream that has passed during processing at the first speed;
A device wherein the position of the starting point is readjustable. The data stream contains metadata,
The apparatus of claim 11, wherein the apparatus begins processing the data stream at the second rate at a location in the stream depending on the metadata. Provided to the user as a data stream with metadata to enable the user to switch from processing the data stream at a first rate to processing the data stream at a second rate different from the first rate. Content information, wherein the processing in the second stream is passed during processing at the first speed and starts at a position in the stream determined by the metadata. A method for enabling a user to receive a data stream over a data network, the method comprising:
Enabling the user to control switching from processing the data stream at a first rate to processing the data stream at a second rate;
The data stream represents content information,
The first speed is different from the second speed,
The above method
When a user command is given, the processing at the first speed is stopped,
Starting processing at the second speed at a starting point at a position in the stream that has passed during processing at the first speed;
The method wherein the position of the starting point is readjustable. The method of claim 14, comprising determining the starting point based on latency in the data network. Executable under the control of a microprocessor and provided for performing at least one step of a method of switching from processing a data stream at a first speed to processing the data stream at a second speed. A set of instructions,
The data stream represents content information,
The first speed is different from the second speed,
The above method
Stopping the processing at the first speed,
Starting processing at the second speed at a starting point at a position in the stream that has passed during processing at the first speed;
The starting point position is a set of instructions that can be readjusted.

Images