KR100633711B1 - Video display method and device based on key frame using video cursor frame on multi key frame screen - Google Patents

Abstract

A display based on keyframes in the video representation allows the user to select keyframes and displays a substantially continuous video stream associated with the representation based on the user selection. In particular, various keyframes are displayed in parallel in a reduced static video format, and the display is controlled by initiating at a particular active keyframe that substantially acts as a dynamic cursor frame within the video format. Cursors can be dynamic video with dynamic audio, dynamic video only, or static video over a period of time including dynamic audio.

Description

Method and device for video display based on keyframe using video cursor frame on multikey frame screen

The present invention relates to a method of displaying a video representation based on said keyframe, which enables a user to select among keyframes and, through such selection, displays a substantially continuous video stream based on the video representation. As such, the method displays the various said keyframes in parallel in a reduced static video format.

In the prior art, it has been proposed to use keyframes as representative portions of a recorded video representation for subsequent selective playback. A continuous video stream means that the video continues to be "on", which can include animations, series of stills, or interactive sequences of images. Characters can vary, such as movies, news or, for example, shopping lists. The situation with this technology is described in the S.W. Smoliar and H.J. It is published in Zhang's article Content-Based Video Indexing and Retrieval.

Keyframes may be derived from the video material upon receipt of the video material on the premise of the user through some derivation algorithm, or the keyframes may be, for example, a video provider in that each new video shot starts with a keyframe. Can be labeled. The present invention recognizes that such keyframes can be utilized to provide users with a dynamic aspect of the video presentation, in combination with useful facilities that allow video material to be edited in a discernable manner.

1 is a block diagram of a TV recorder combination.

2 is an exemplary configuration diagram of video recording.

3 is an illustration of a scrolling mosaic user interface.

4 is a diagram of a scrolling list user interface.

FIG. 5 is a view showing the realized state of FIG. 3; FIG.

FIG. 6 is a view showing the realized state of FIG. 4. FIG.

7 is a state diagram of system operation.

Accordingly, it is an object of the present invention, in particular, to provide the user with an immediate feel of the dynamic aspect as well as the static content of the video presentation, which is combined in a single video screen. One characteristic feature of the invention is characterized in that the display is controlled to start from a particular active keyframe which acts as a dynamic video cursor frame in the video format according to user selection. In this embodiment, the cursor frame is moved in real time, allowing the user to see between keyframes that are similar scene configurations but may have different dynamic states.

Advantageously, the method may render a video representation involving audio and the execution begins from audio associated with a particular active keyframe that acts as a separate video cursor frame in the video format, depending on user selection. It is characterized in that it is controlled to. For certain indications, for proper editing the audio is more distinct than the video, so the audio can represent the stream in progress, and the video keyframe is a kind of "header" that falls within the interval between the corresponding two consecutive keyframes. Will act as "." Video keyframes can remain static and active keyframes that act as cursor frames that individually or individually stepping can be highlighted in an appropriate manner.

Preferably, the dynamic video cursor frame is frozen when it matches the content of the next displayed keyframe. This paused display content may belong to the beginning or end of the immediately displayed section. The next keyframe may then be a dynamic video cursor frame. Such takeover may be controllable and / or programmable. This allows the user to observe what editing of the display will be performed. Also, when proceeding to the next section, the display mode may be changed from normal speed to high speed or low speed display. The present invention can be used as fully digital video, or various video frames can be used for video with digital labels for easy addressing.

Preferably, the keyframes may be set in a hierarchical structure, and various key frames may be allowed to be deleted from the display, and the sections of the display may be combined before and after the deleted keyframe. In this way, fast changes can be applied to fine-grained and coarse-grained editing. Again, the hierarchical structure may be executed by the provider or by some cluster algorithms. For example, if the keyframes are known through some cross-correlation, then a small mismatch may indicate a low level keyframe, whereas a high mismatch may indicate a high level keyframe. The former occurs essentially between successive shots in the same movie scene, and the latter belongs to a transition between scenes, which may dramatically change the overall conditions such as brightness level. Another situation is when various other forms of video material such as news, movies, or service shop lists are connected to each other. All of this facilitates the operation of the end user.

The present invention is also an apparatus for displaying a video representation based on keyframes, the apparatus having a selection means for allowing a user to select among the keyframes, and which is substantially continuous with respect to the display based on the user's selection. The display device arranged for executing the sensory stream and displaying the various keyframes in parallel in a reduced static video format is then executed from a particular active keyframe that acts as a video cursor frame within the video format. It is characterized by having a control means for controlling to start. In the context of a user terminal, such as an improved video recorder and TV set combination, the provision of the device broadly extends the direct execution by that user.

These and other features of the invention are described in more detail below with reference to the accompanying drawings and appropriate embodiments.

Although not limiting, many of the beneficial features included in regular regular customer and home use include the following:

The keyframes should be displayed in a manner that can typically be distinguished from each other by the user located at a typical TV viewing distance with respect to the corresponding video size. In a family set, this distance is a few meters, and in a desktop set, it is approximately 0.5 meters.

The number of keyframes displayed simultaneously on the screen should be sufficient to provide the user with an overview of the significant portion of the content of the digital video material.

Keyframes should generally be displayed in an undistorted manner to maximize discernment. As an example, the aspect ratio of a particular keyframe should preferably be maintained.

The TV set's own remote control device acts as a user control input device.

The feedback should be such that the feedback related information can be perceived from the general viewing distance of a typical television set as described above.

Computer concepts such as "drag and drop" are generally not necessary.

Temporary use of the equipment should be provided.

Equipment should be provided so that it can only be used temporarily, rather than continuously.

The concept of the user interface should reflect the linear model of the video representation that is familiar to the user.

In connection with the implementation of the present invention, Fig. 1 shows a TV recorder combination. Reference numeral 20 denotes a TV set display and associated instant control and power supply. Reference numeral 22 denotes a connection with an antenna or other type of signal distribution entry. Such signs include derivation of digital video information or digital signal parts from the received signal, as appropriate. Reference numeral 32 denotes a route of a video stream and related information between various subsystems of FIG. This route is controlled by control box 34 via control signals on line 35. The control line is shown as a single bidirectional interconnect, but is actually installed with a number of unidirectional or bidirectional lines. The control box receives detection signals from the display and from other substreams 38 and 40 and also controls the substrip. Block 38 is a linear recorder with a large storage capacity of multiple gigabytes. Block 40 is a magnetic disk recorder with a large storage capacity but only a fraction of the storage capacity of recorder 38. However, the access of recorder 40 is much faster through cross track jumping. Blocks 28 and 40 are somewhat similar to a computer memory cache system and form a two-level storage configuration that stores at least one item of all of the video representations.

2 illustrates an example configuration of a video presentation. To achieve video material functionality, bar 60 includes the video itself as a compressed video material string, such as coded MPEG, or in the form of a frame. The actual storage needs do not have to be uniform in time, but the information is stored along the bar as a video time process. Scattered keyframes are represented by dark vertical strips, such as 68. The keyframe is used to represent or represent the entire video in the interval up to the next keyframe. One implementation is for the video provider to select the first frame of each new shot as its keyframe through adding some label or inclusion to the "Content Table" (TOC). Another implementation detects that the receiver instantly changes every frame video content through an algorithm. The present invention takes the relevant algorithm allowed. As shown, these dispersions are not uniform. An alternative mechanism is for consecutive keyframes to be contiguous with each other in preset intervals, such as tens of seconds. According to the present embodiment, the keyframes are only indicated by the display 62. In addition, the keyframes are organized in some hierarchical manner, and the indication 64 has only a limited set of related keyframes at high positions. This layer is repeated, and the indication 66 relates only to a single keyframe for all video indications 60. Keyframes of various levels may be determined differently from the above-described configuration, and may exist in parallel. The storage mapping on FIG. 1 can be achieved such that the main body of the video representation is stored in the tape recorder 38 and the keyframes are generated at least in the disc recorder 40 in possible combination with a small video section immediately following the associated keyframe. have. The length of such interval may correspond to the time latency of the linear tape recorder 38 so that real time access can be achieved. As such, the video display may be essentially linear, such as film. Alternative uses may include animation, still or other images in which any memory interval will be used by the current consumer. The possible effect of keyframes is to suppress them. This effectively combines the time interval before the keyframe with the time interval after it. The reset feature can separate the segments again. In addition, various kinds of keyframes such as types separated from each other by fixed time intervals may be suppressed. Various other kinds of keyframes may be available for one indication, such as introduced by the provider for what was generated by the logic algorithm upon receipt.

3 illustrates an example design of a scrolling mosaic user interface. All screens show 20 keyframes starting from the normal left corner, from left to right and from the bottom right corner (each keyframe is the number of the total rank of the keyframe shown). In practice, keyframe 144 is highlighted by a rectangular control cursor. The user operates the remote control to move the control cursor freely on the displayed keyframes, as well as on the buttons displayed on the top and basebar via navigation control on the cursor device. If the user moves the control cursor to the left of the upper left corner, the display jumps back by 20 keyframes. Moving to the right of the lower right corner causes a 20 frame forward jump. In addition, access to the top bar of the full screen achieves shortcut control for access to other parts of the display, with the latter split into five equally long parts (dark horizontal bar is the 20 keys displayed here of the full display). The total time covered by the frame).

Another function can be started by first selecting a particular keyframe and then selecting one of the base buttons. The "view program" controls the start from the cursor access keyframe. The "view segment" may be the same, but may be limited to playing only a single segment ending at the next keyframe. "View x-y" controls the start at the earliest time of the two cursor access keyframes. Various other control modes or functions may be implemented in combination with the keyframe selection functionality described above. As an example, there is a fast forward or slow forward that allows a user to check a particular interval for a given occurrence or vice versa in order to get a definite video effect. During display, when passing the moment belonging to a particular keyframe, the latter is active until the moment associated with the next keyframe is reached, effectively displaying the video stream. In that case the latter becomes the active frame. This feature allows the user to program the video recorder directly for a segment display sequence, such as omitting certain segments, such as advertisements, or to notice certain details by slow forward. During display, audio can be activated or suppressed via control buttons, not shown. Alternatively, the control may advance the audio, but the video cursor is separated and proceeds only every segment with proper emphasis.

4 is an exemplary diagram of a scrolling list user interface. Here, five keyframes are displayed at the base of the entire screen. Keyframe 145 is highlighted by a rectangular control cursor displayed along the edge of the keyframe. Keyframe 145 is also displayed at greater magnification in the background. The control interface is the same as in FIG. 3 although the control button positions are different. Alternatively, the extended keyframes are suppressed in multiple keyframe bars.

FIG. 5 implements FIG. 3 with an actual image representing a keyframe and some modifications. The control interface is consistent with the above, although some items have not been processed aesthetically. In this embodiment, the time stretch covered by 20 keyframes is approximately 2 seconds, which approximates a useful subrange. The various keyframes can actually be completely distinguished from each other. Also, if the user does not feel the need for expensive editing, the change may include various buttons omitted from the control buttons.

The principle of a device for having a dynamic representation of a video cursor progressing in time in a real-world keyframe field is that if a static representation of keyframes alone is such that the dynamic representation is absolutely necessary to understand the evolution of an event, the meaning of the video representation as a whole is insufficient. Is to deliver. Therefore, the meaning is improved as follows. After the system has been idle for a certain amount of time, keyframes with the cursor 'enclose' become 'com-live' as they begin to play small digital video material containing some associated audio and other effects. If the next keyframe is reached during playback, the cursor automatically jumps to the next keyframe displayed in the user interface until the user initiates (resumes) interaction with the system.

In this regard, FIG. 7 shows a state diagram of system operation. In state 100, the system waits for input from the user and displays a number of keyframes. Such input may mean selecting a keyframe for displaying jumping among the displayed keyframes, jumping to another set of keyframes, and the associated period. Such input constitutes an arrow 104 and initiates a new period. If there is no such input for n seconds (eg 20 seconds), an arrow 108 is made and state 102 is reached. Here the system proceeds to a dynamic video cursor frame. As long as user input is not reached, arrow 110 is made and the system continues to display displayable video material. However, if user input is received, an arrow 106 is achieved and the system is stopped at the beginning of the actual content or actual period of the dynamic video cursor.

FIG. 6 realizes FIG. 4. This displays a screen capture of the actual run. Here, the lines diverging from the top bar of the screen are not executed for aesthetic reasons. In addition, the cursor position remains fixed at the center of the screen, thus acting as the focal point of large actual film snapshots. This small portion is changed separately and the user interface acts as described above.

Claims (14)

A method of displaying a video presentation based on a keyframe, the method comprising: Displaying the key frames of the video representation in parallel in a reduced static video format; Allowing a user to select from the keyframes; Displaying a substantially continuous video stream based on the indication in accordance with the selected keyframes, And the substantially continuous video stream is displayed in the reduced format instead of the selected static keyframe. The method of claim 1, wherein the video stream carries audio and the method enables suppression of audio associated with the video stream. 3. A method according to claim 1 or 2, wherein the substantially continuous video stream is stopped when it matches the content of the next displayed keyframe. 4. The method of claim 3, wherein upon matching, the next substantially continuous video stream is displayed instead of the next keyframe. 5. The method of claim 4, wherein the display of the next substantially continuous video stream is controllable and / or programmable. 5. The method of claim 4, wherein the display of the next substantially continuous video stream may comprise a programmable mode change. 3. A method according to claim 1 or 2, wherein the displayed keyframes accept substantially all of the actual video format. 3. A method according to claim 1 or 2, wherein the displayed keyframes accept a portion of an actual video format and the substantially continuous video stream is displayed in an enlarged format relative to other keyframes. Display method of video display. 3. A method according to claim 1 or 2, further displaying a preset set of soft control keys. 3. A method according to claim 1 or 2, wherein the video representation is at least digitally labeled. The key according to claim 1 or 2, wherein the keyframes are set in a hierarchical structure, and / or various keyframes can be deleted from the display to combine the sections of the indication before and after the deleted keyframe. Display method of displaying a video based on a frame. A device for displaying a video representation based on a keyframe, comprising: Display means for displaying keyframes of the video representation in parallel in a reduced static video format; Selecting means for allowing a user to select among the keyframes; And Rendering means for executing a substantially continuous video stream in accordance with the selected keyframe, And the execution means is capable of executing the substantially continuous video stream in the reduced format instead of the selected static keyframe. The method of claim 12, And the execution means stops the substantially continuous video stream when it matches the content of the next displayed keyframe. 14. A display apparatus according to claim 13, wherein in said matching, said execution means executes a next substantially continuous video stream instead of said next keyframe.