JP3595871B2 - Apparatus for displaying / editing continuous media information, method for displaying / editing continuous media information, and recording medium recording the processing procedure thereof - Google Patents

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention provides a technique for displaying continuous media information such as video moving image frames and audio waveform data on a screen of a display device such as a CRT (Cathode Ray Tube) or an LCD (Liquid Crystal Display) by a computer. Related to a technology for editing continuous media information displayed on a screen based on an operation instruction from an operator input from an input device such as a keyboard or a mouse, and in particular, the information content is easily understood and fine editing is performed. The present invention relates to a continuous media information display / editing device suitable for facilitating work, a continuous media information display / editing method, and a recording medium on which a processing procedure is recorded.
[0002]
[Prior art]
With the widespread use of AV equipment such as VCRs and camcorders at lower prices, anyone can easily shoot and acquire images. Further, by improving the processing speed by hardware or software, it is possible to digitize a captured video and reproduce or edit the video on a computer.
[0003]
As a conventional technique for editing digitized video using such a computer, for example, AVID / 1 media composer (MEDIA COMPOSER) manufactured by Avid Technology or JP-A-7-46462 (""). Method and Apparatus for Editing Video Program "), and a premium (Premiere) manufactured by Adobe.
[0004]
The feature of these technologies is that when displaying a video to be edited, frame images constituting the video are displayed in a line like a movie film.
Hereinafter, a frame unit editing technique using this frame as an editing unit will be described with reference to FIG.
[0005]
FIG. 16 is an explanatory diagram showing an example of a screen display by the conventional frame unit editing technique.
The window 300 on the screen of the display device of the computer system is provided with a row 310 for displaying an image of a person to be edited and a row 330 for displaying an image of a car.
Here, the video to be edited is configured as a sequence of a series of frame images. For example, the image of a person in the row 310 is composed of frame images 111 to 113, and the image of a car is composed of frame images 121 to 124.
[0006]
A feature of such an editing technique in units of frames is that video editing is performed using a video expression in which a sequence of frame images is linearly arranged like a movie film. However, in this technique, since the frame image sequence in the video is arranged in a straight line, all the frame images cannot be displayed in the display frame of one window at a time. To check the video content outside the display frame, the operator scrolls the frame image sequence using the scroll bar 301.
[0007]
Further, in this example, a screen at the time of editing processing for joining two videos “video 1” and “video 2” into one is displayed. Here, when switching from the image of the person in "Image 1" to the image of the car in "Image 2", a special image effect called "wipe" is added, in which the image of the car gradually appears from the end of the image of the person are doing. More specifically, an image obtained by synthesizing the video end frame images 112 and 113 of the person and the head frame images 121 and 122 of the car video is used as the image of the video editing result.
[0008]
In a typical frame-based editing technique such as the above-described media composer or the like, a special effect display line 320 is specially provided in order to present the special effect-attached editing operation to the user in an easily understandable manner. Further, a special effect icon 321 such as “wipe” is displayed at a place where the two images are actually combined.
Such a video editing technology using a video expression in which a frame image sequence to be edited is arranged in a straight line like a movie film is a frame-based video editing technology.
[0009]
This frame-based video editing technology can edit video such as VHS or 8-mm format video tape, which has been difficult to confirm the content, by an intuitive operation such as cutting a movie film.
However, if the frame image sequence is arranged in a straight line, it becomes difficult to find out what kind of video scene exists where.
[0010]
For example, a video usually has a unit representing a series of logical video sections called a scene or a cut, such as an opening scene, a scene in which a person appears, and an ending scene. No special video section is considered, and only the frame image sequence, which is a physical unit in the video, is arranged. For this reason, it is difficult to understand from where to what a series of scenes.
Furthermore, when it is desired to delete only a scene in which a certain person appears, for example, the operator needs to check at which frame position the scene is switched by shifting one frame at a time.
[0011]
As a conventional technique for solving such a problem of the frame unit editing technique, for example, Japanese Patent Application Laid-Open No. 7-79404 (“Method and Apparatus for Creating a List of Representative Images of Video”) and Japanese Patent Application Laid-Open No. 5-268517 ( There is a video editing software “Mediachef / CUT” manufactured by Hitachi, Ltd. described in “Display Method of Moving Image Editing System”). This technique uses a scene-based editing technique that uses a scene, which is a logical unit of video, as a basic unit of editing.
Hereinafter, a scene unit editing technique using this scene as a basic unit of editing will be described with reference to FIG.
[0012]
FIG. 17 is an explanatory diagram showing an example of a screen display by the conventional scene-based editing technique.
In a window 400 on the screen of the display device of the computer system, four video scenes to be edited are displayed in a scene block icon format. In the present editing technique, the images 111 to 113 representing the scenes of the persons in FIG. 16 are displayed as scene block icons 410, and the image scenes 121 to 124 of the car are displayed as scene block icons 420. A video scene is displayed as a scene block icon 430, and a video scene of a house is displayed as a scene block icon 440.
[0013]
In each of the scene block icons 410 to 440 of this example, an image of a characteristic frame in the scene is displayed on the surface as an icon image, the name of the scene is displayed below the icon image, and the time length of the scene is further reduced. Is displayed as the depth of each of the rectangular parallelepiped scene block icons 410 to 440.
As described above, in the scene-based editing technique, only scenes having a logical structure are displayed by the scene block icons 410 to 440, so that it is possible to see at a glance what scenes exist in the video.
[0014]
In this scene-by-scene editing, a scene can be easily edited by selecting a certain scene block icon and deleting or moving it. For example, in FIG. 17, to move the house scene before the control scene, first select the house scene scene block icon 440 using the mouse cursor 170 and move the mouse cursor 170 to the control scene scene block icon 430. Move before. As a result, the scenes are rearranged, and the scene block icon 440 moves before the scene block icon 430.
[0015]
As described above, with the scene-based editing technique, it is possible to easily move or delete a logical scene, which is difficult with the frame-based editing technique.
It should be noted that some scene block icons can be grouped depending on the scene unit editing technology, and some of the grouped scene block icons and the scene block icons in the group can be represented by a tree structure. According to this technology, it is possible to perform editing in accordance with the logical structure of the actual video.
[0016]
However, in this scene-based editing technique, since the scene is represented as a scene block icon, it is difficult to catch a change in a fine frame image in the scene. For example, in the case of the above-described car scene, in the frame unit editing technique of FIG. 16, since it is represented by the frame images 121 to 124, it can be easily grasped that the car is moving to the left. However, in the scene-based editing technique of FIG. 17, since only a frame image representing a certain scene is displayed on the scene block icon, it is determined whether the car is stopped or the car is moving. It is difficult to grasp whether it is moving at the speed of.
[0017]
Also, in this scene-based editing technique, since a frame image sequence is not displayed, the frame-based editing can be easily performed as described above (adding a special effect between nine frames by “wiping”, and furthermore, Editing work on a frame basis, such as inserting subtitles in a certain frame section, has become difficult.
[0018]
The above problems at the time of video editing also exist in the editing of continuous media other than video, for example, audio.
First, a typical audio editing technique used in a media composer or the like will be described using the video / audio editing screen in the window 300 of FIG.
[0019]
In this audio editing technique, an audio waveform is represented by being arranged in a straight line. In FIG. 16, the audio accompanying the “video 1” in the row 310 is represented as an audio track 340. Similarly, the audio accompanying the “video 2” in the row 330 is expressed as an audio track 350. These audio tracks 340 and 350 can select an audio section or process the selected audio by using a mouse, a scroll bar, or the like as in the case of video editing.
[0020]
Such an audio editing technique also has a problem that it is difficult to grasp the logical structure of the entire audio, as in the above-described frame-based video editing. In other words, the audio has a logical structure like the opening song and the voice of the interviewer, just like the video, but since the audio is expressed in such a straight line, the interviewer is It's hard to tell if it's a voice.
[0021]
As for audio editing, it is also possible to devise a scene-based editing technique as in the case of video. However, as in the case of scene-by-scene editing of video, there is information that cannot be grasped only by looking at the scene block icon, such as where in the audio the volume changes.
[0022]
[Problems to be solved by the invention]
The problem to be solved is that, in the prior art, in an editing process by a computer, an operator easily understands the logical structure of continuous media such as video and audio while simultaneously understanding its constituent units / elements (for video). The point is that the image data and the waveform data cannot be easily understood.
[0023]
SUMMARY OF THE INVENTION An object of the present invention is to provide a continuous media information display / editing apparatus and a continuous media information display / editing apparatus which can solve the problems of the conventional techniques and can efficiently edit a continuous media such as video and audio using a computer. An object of the present invention is to provide a recording medium recording a display / editing method and a processing procedure thereof.
[0024]
[Means for Solving the Problems]
In order to achieve the above object, the present invention provides a continuous media information display / editing apparatus and a continuous media information display / editing method, for example, in which each image frame of a moving image is continuously displayed in a horizontal direction on a screen. The line breaks at the point of change, and each scene is displayed on a line-by-line basis.
[0025]
Then, a line feed mark is clearly indicated at each line feed position to make it easier to understand the scene unit. When the display frame is exceeded, a scroll operation or the like may be performed. However, the display may be forcibly wrapped around the display frame and displayed on the next line. In this case, the line feed mark is not specified.
[0026]
Also, a scale line representing a predetermined elapsed time is displayed from the head of each line on the screen and in the direction of the subsequent line from the head line, so that the time position of each frame and each scene can be easily understood. .
In addition, scene block icon information created by the related art corresponding to all frames displayed in this row is also displayed in the area before the head of the row.
[0027]
Further, when the operator designates a frame to be edited, the target frame portion is highlighted, so that the quality of the designation can be easily determined, and the editing operation such as moving the frame is facilitated. In addition, for example, when a plurality of frames are combined into one line, each combined portion is displayed larger than the other portions, so that the result of the combined editing can be easily determined. Further, the display state of each frame line before the combined editing is displayed in a window from the line following the combined edited line.
[0028]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram showing an embodiment of a configuration according to the present invention of a continuous media information display / editing apparatus of the present invention.
In FIG. 1, reference numeral 1 denotes a display device including a CRT or an LCD, etc., 2 denotes an input device including a keyboard, a mouse, a trackball, etc., 3 denotes a storage program including a CPU (Central Processing Unit), a RAM (Random Access Memory), and the like. A display / editing device for performing display and editing of continuous media information according to the present invention by performing computer processing according to the system; 4, an auxiliary storage device such as an HDD (Hard Disk Drive); A continuous media information input device for inputting audio information, an optical disk 6 on which a program comprising a display / edit processing procedure according to the present invention to be processed by the display / edit device 3 is recorded, and a drive device 7 for driving the optical disk 6 to execute the program Enter display / editing device 3 An optical disk drive for.
[0029]
The display / editing device 3 includes a display processing unit 8 and an editing display processing unit 9 for performing display and editing in units of frames by a program read from the optical disk 6. A data converter 10 is provided for converting an analog signal such as input video or audio information into a digital signal that can be processed by a computer.
[0030]
The display processing unit 8 includes a frame display unit 8a, a break detection unit 8b, a line feed unit 8c, a line feed mark designation unit 8d, a display frame setting unit 8e, a turnback unit 8f, a horizontal time scale display unit 8g, and a vertical time scale display unit. 8h, a scene block icon display section 8i, and the edit display processing section 9 has a combined display processing section 9a and an outline edit display processing section 9b.
[0031]
With such a configuration, the display / editing device 3 converts the video and audio input from the continuous media information input device 5 into image frames and waveform data and displays them on the window screen of the display device 1. The editing process is performed on the displayed image frame and waveform data based on the operation instruction from the operator.
[0032]
In this display processing, the display / editing device 3 sequentially displays the image frames and the waveform data in the horizontal direction by the frame display unit 8a. At this time, the break detection unit 8b detects a logical break of the image frame or waveform data, that is, a scene change point, and the line feed unit 8c detects the image frame or waveform data of each detected scene change point. Break the display.
[0033]
As described above, since the image frames and waveform data are displayed by changing the lines in scene units, the operator performing the editing process can easily understand the logical structure of each image frame and waveform data, that is, each scene structure. In addition, the contents of the constituent units / components, that is, the image frames and the waveform data, can be easily understood, and the editing work of them can be performed efficiently.
[0034]
A line break mark is highlighted at the line break position of each scene by the line break mark specifying section 8d. This makes it easier for the operator to understand the scene units.
A display area in the horizontal direction of the screen is set by the display frame setting section 8e, and each image frame and waveform data of a scene exceeding this display area are turned back from the area by the turn-back section 8f and displayed on the next line. . Note that the line feed mark highlighting unit 8d does not highlight the line feed mark at this return position.
[0035]
Also, the horizontal time scale display unit 8g and the vertical time scale display unit 8h display a scale line representing a predetermined elapsed time from the head of each line on the screen and in the direction of the succeeding line from the head line. . Thereby, the operator can easily understand the time position of each frame and each scene.
Further, the scene block icon display section 8i also displays scene block icon information corresponding to all the frames displayed in this row in the area before the head of the row.
[0036]
Then, in the editing process, when the operator specifies a frame to be edited on the screen of the display unit 1 via the input device 2, the display / editing device 3 performs, for example, moving editing and the like by the edit display processing unit 9. , The target frame portion is highlighted. Thus, the operator can easily determine whether the designated operation is good or not, and can easily confirm the progress of the editing operation such as the movement and the result.
[0037]
In this editing operation, if the instruction is to combine a plurality of frames into one line, the combining display processing unit 9a combines the frames as instructed and combines each of the combined parts with another part. Display larger. This allows the operator to easily determine the result of the combined editing.
Further, the outline edit display processing unit 9b displays the display states of the scenes of the respective lines before the combined editing from the line following the combined edited line in a smaller window.
[0038]
A video expression process in which the above series of frame image sequences (rather than simply being arranged in a straight line as in the related art) is performed by dividing the frame image sequence by a logical scene break of the video and displaying a line feed. Is a display / editing device 3 in which a metaphor of a word display (word processor) sentence display technique or editing technique is used. Hereinafter, the text display technology and the editing technology of such a word processor will be described.
[0039]
First, the relationship between the word display / editing technology of a word processor and the video display / editing technology will be described, then the display / editing device 3 will be described, and finally, the processing operation of the display / editing device 3 for audio. Will be described.
[0040]
Among various media such as video, audio, still images, and character strings, character strings have been used for a long time, and various techniques have been developed for display and editing techniques. A character string is simply a sequence of characters, if it is the original. However, a character string actually has a meaning (usually called a sentence), and has a logical structure such as a word or a sentence, a chapter or a section. Such a logical structure is expressed in a plain manner on a plane called a page by using punctuation marks, line feeds, and chapter and section headings in paper and word processing sentences.
[0041]
A video is simply a sequence of frame images, and there is also a logical structure called a scene. However, there has been a problem that there is no technique for easily expressing the logical structure of the frame image sequence.
Therefore, if concepts such as line breaks, punctuation marks, and pages can be applied to a video as well as a character string, the video can be expressed as easily as understanding a sentence composed of a character string.
[0042]
Therefore, as a video expression technique, a concept of a line feed expressing a logical break in the video is added. Specifically, to display a series of videos,
(A) displaying a video segment from the start position of the video to be displayed to the logical switching point in the video as a sequence of frame images having a length proportional to the time length of the video segment;
(B) moving the next display position to the position next to the previous frame image string display position displayed in (a),
(C) The remaining video section obtained by removing the video partial section already displayed in (a) from the display target video is set as a new display target video.
Then, this process is performed at least twice.
[0043]
Furthermore, this video display / editing technology is applied to audio and the like.
That is, the following technology is used in order to make the expression technology for video applicable to continuous media such as audio, animation, slide show, video, and media in which these are multiplexed.
[0044]
First, (d) a continuous media segment from the start position of the continuous media to be displayed to a logical switching point in the continuous media is represented by a bar-like representation having a length proportional to the segment length of the continuous media segment. Display,
(E) Move the next display position to the position next to the previous bar-shaped display area displayed in (d),
(F) A continuous media section remaining after removing the continuous media partial section already displayed in (d) from the continuous media to be displayed is set as a new continuous media to be displayed.
Then, this process is performed at least twice.
[0045]
Hereinafter, first, a basic expression operation example and an editing operation example of the continuous media information display / editing device 3 in FIG. 1 will be described with reference to FIGS. , And FIG. 3, and an example of a data structure and a processing flow will be described with reference to FIGS. 6 to 11. Finally, the present example will be described with reference to FIGS. 12 to 15. A technique for further improving usability will be described.
[0046]
FIG. 2 is an explanatory diagram showing a first processing operation example of the display / editing device in FIG. 1 according to the present invention.
In this example, the processing result of the display / editing device 3 on the screen of the display device 1 in FIG. 1 is shown. In the window 100, a frame image display area 151 is displayed. Then, as the video to be edited, a video in which the frame images 111 to 113, and subsequently, the frame images 121 to 124, the frame images 131 to 132, and the frame images 141 to 143 are displayed as a continuous frame image sequence is displayed. ing.
[0047]
The greatest feature of the display technology of this example is that instead of displaying such a series of videos in a straight line as in the conventional frame-based editing technology, a frame image sequence is displayed for each logical unit of video, that is, A line of a frame image sequence is displayed for each scene. In this case, the frame images 111 to 113 representing “scene 1” are displayed as the top row, and similarly, the frame images 121 to 124 representing “scene 2” are the second row, and the frame images representing the scene of “control”. 131 to 132 are displayed in the third row, and frame images 141 to 143 representing the scene of “house” are displayed in the fourth row.
[0048]
The following effects can be obtained by displaying the line of frames in a line feed.
(1) The operator can easily grasp the logical structure of the video, that is, what kind of scene exists in the video by looking at the arrangement of the rows.
(2) Also, by looking at each line,
(2-1) From the line length (the number of frame images in FIG. 2), the time length of each scene can be easily grasped.
(2-2) From the display contents of the row, the contents of the scene such as the transition state of the frame image in each scene can be easily grasped.
[0049]
In the example of FIG. 2, some further contrivances are made to further improve the understanding of such display contents.
First, the scene information display area 150 is displayed next to the frame image display area 151 in order to present the information of the scene displayed in each row to the user in a more understandable manner.
[0050]
Then, in the scene information display area 150, information on the scene represented by each row is stored in the scene information display area 150 by a scene block icon created by the related art, corresponding to the position of each row in the frame image display area 151. Is displayed.
That is, as scene information, a frame image that best represents a scene from a sequence of frame images in the scene, a scene name that represents the contents of the scene, and a three-dimensional depth expression of an icon that represents the scene length Is displayed as one icon.
[0051]
For example, scene information corresponding to the scenes of the frame images 111 to 113 on the first line is displayed as the scene block icon 110 at the same position in the scene information display area 150. In this scene icon 110, a frame image 112 is displayed as a frame image representing the scene, a character string "Scene 1" is displayed as a scene name, and a three-dimensional depth is expressed as the length of the scene as the icon depth. Is displayed for the length proportional to.
[0052]
The second idea is to display a time scale line indicating a certain time position, such as 30 seconds or 1 minute, in the frame image display area 151.
In the example of FIG. 2, a time scale line 153 representing the 30th second from the beginning of the scene and a time scale line 154 similarly representing a position of 1 minute are displayed. By displaying such a kind of auxiliary line in the frame image display area 151, the operator can more specifically grasp the time length of each scene.
[0053]
In the third method, a line feed mark 156 is displayed in order to more closely resemble a word processing style editing technique.
The fourth idea is to make it possible to select a section in the video, which will be described in detail in the following examples. In FIG. 2, the user selects a section of the video using the mouse cursor 170, and the selected section is displayed in the highlight area 160.
[0054]
Next, as a supplement to the description of FIG. 2, a difference between another similar video display technology and the technology of the present embodiment will be described.
As a first video display technique, a material video management window provided in the media composer described in the related art will be described.
[0055]
That is, in the material video management window of the related art, one video file among a plurality of video files is displayed as one row as a frame image column in one video file, as in the technology of the present embodiment, and a plurality of video files are displayed. By expressing the file as a plurality of lines similarly to the technique of the present example, the configuration of the plurality of video files is displayed in an easy-to-understand manner. However, unlike this technology, the video to be displayed by the technology of this example is not a plurality of independent scene groups, but a line of continuous video as a line.
[0056]
As a second of the other video display technologies, in the display technology of the scene block icon group such as the conventional scene unit edit shown in FIG. 17, the arrangement of a plurality of scene block icons is similar to this example. A technology that breaks lines in units is also conceivable. However, with such a display technique, a plurality of scenes can be displayed side by side, but the length of the display line does not represent the time length of the scene as in this example, but the length of the logical scene. It merely represents a number and cannot intuitively grasp the scene time length.
[0057]
Although not a video display technology, a technical difference between the conventional text display technology and the present example will be described.
The data targeted by the conventional sentence display technology is only character strings, and is not continuous media information targeted by the technology of the present example. The continuous media information has time information for reproducing it, and is different from a simple character string. In addition, continuous media information can be referred to only a part of the information at a certain time, which makes it difficult to grasp the entire content and its logical structure in the related art. In contrast, since the character string has no time information, the entire character string can be simultaneously expressed on two-dimensional paper.
[0058]
FIG. 3 is an explanatory diagram showing a second processing operation example of the display / editing device in FIG. 1 according to the present invention.
In this example, a screen for editing an audio track attached to a video is displayed. When the processing target is audio, an audio waveform display area 251 is displayed instead of the frame image display area 151 in FIG.
[0059]
In the case of this example, the sound accompanying the scene 2 (displayed by the scene block icon 120) of the video in FIG. 2 and the control scene (displayed by the scene block icon 130) is displayed. In the case of such an audio editing, instead of displaying the frame images 121 to 124 of the scene 2, in this example, the waveform of the audio track accompanying the scene 2 is displayed using the bar-shaped area 211 in which a line graph is drawn. I have. The length of the bar-shaped region 211 indicates the time length of the audio scene according to the scale of the time scale on the screen.
In addition, as the track expression technology of the audio, an expression in which the magnitude of the volume of the waveform is graphed may be used other than the line graph of the actual waveform.
[0060]
In this example, the voice section selected using the mouse cursor 170 is displayed as the highlight display area 260 as in the case of FIG.
As described above, the display / editing device 3 in FIG. 1 can also edit audio in addition to displaying and editing video. As can be easily analogized from this, the display / editing apparatus 3 can be applied to other continuous media having time information.
[0061]
For example, as other continuous media, a slide show in which several slides are displayed every few seconds, an HTML document including a program such as a WWW page that moves on the Internet instead of a slide, and the like are also conceivable. . Further, the present invention can be applied to a composite continuous medium in which these basic continuous media are continuously arranged in a line, or a multi-track continuous medium having a plurality of such continuous media in parallel.
[0062]
Hereinafter, technical examples of display / editing operations when the continuous medium is a video will be mainly described, but all of these techniques can be applied to continuous media such as audio.
FIG. 4 is an explanatory diagram showing a third processing operation example of the display / editing device in FIG. 1 according to the present invention.
This example illustrates an example of an editing operation of the display / editing device in FIG. 1, and particularly illustrates an example of an operation of selecting an editing section.
[0063]
FIG. 4A shows an example in which the position of a certain frame in the video is specified by clicking one point on the screen specified by the mouse cursor 170.
FIG. 4B shows an example in which a video section is specified by clicking two points on the screen specified by the mouse cursor 170. FIG. 4C shows an operation example of selecting the entirety of one certain scene.
[0064]
Although not adopted in this example, when the mouse is clicked, a frame image corresponding to the position of the image specified by the mouse cursor 170 is displayed in a separate window, or from the position of the specified image. It is also easy to reproduce the starting video in another window.
[0065]
In FIG. 4A, in order to specify the position of the frame image 112 in the video, the mouse cursor 170 is moved over the frame image 112 and the mouse is clicked. In this way, the position of the frame image 112 in the video is specified, and the frame image 112 at the corresponding specified position is highlighted by the highlight area 560.
[0066]
Further, in FIG. 4B, in order to select and specify a video section including the frame images 112 to 113 and 121 to 122, first, the mouse cursor 170 is moved over the frame image 112 which is the head position of the selection range. Move, click the left mouse button (display of FIG. 4A), and then move the mouse cursor 170 over the frame image 122 that is the end position of the selection range, and click the right mouse button. I do. By performing this operation, a video section spanning a plurality of frame images is selected, and the frame images 112 to 113 and 121 to 122 in the corresponding selection range are highlighted by highlight areas 561 and 562.
[0067]
Further, in FIG. 4C, in order to select a certain scene, in this case, the entire “scene 1”, that is, the frame images 111 to 113, among the lines representing the scenes to be selected, Click the mouse anywhere other than. In the example of FIG. 4C, the scene block icon 110 is clicked. By performing this operation, the entire frame image sequence of “Scene 1” is selected, and the selected section is highlighted in the highlight display area 564. Further, in the example of FIG. 4C, when the entire scene is selected, the corresponding scene block icon is also highlighted using another highlight display area 563.
[0068]
FIG. 5 is an explanatory diagram showing a fourth processing operation example of the display / editing device in FIG. 1 according to the present invention.
This example shows an example of an editing process operation for a selected section of the display / editing apparatus in FIG. 1, and in particular, for an image section specified and selected by the operation of FIG. , And a frame image filter editing operation for blurring an image or the like for each frame image sequence in the selected video section.
[0069]
In the editing operation on the selected section, as shown in FIG. 5A, first, the operator selects a video section to be edited from now on by the operation shown in FIG. Then, as shown in FIG. 5B, the editing menu 181 is designated, and the section editing process desired by the operator is selected from the editing pull-down menu 183. The editing process is performed on the video section selected in this manner.
[0070]
In this example, as a selectable section editing process, cutout of the selected video section and insertion (pasting) of the cutout video section, as shown by an edit menu 181, and menu image 184 as a frame image filter edit, as shown by an edit menu 181 As described above, a process for adjusting the brightness of the frame image sequence, a process for similarly blurring the image, and a process for displaying subtitles in the frame image are prepared.
[0071]
FIG. 5C-1 shows an editing result display when “Cut” is selected as the section video editing process. By the editing process, the video section (frame images 121 to 122) previously selected in FIG. 5A is deleted from the edited video, and accordingly, the corresponding video section is not displayed on the screen.
[0072]
FIG. 5C-2 shows an editing result display when “subtitle insertion” is selected as the section video editing process. By the editing processing, image processing for displaying subtitles is performed on all the frame images 121 to 122 in the video section previously selected in FIG. 5A, and the frame images 121a to 122a resulting from the image processing are obtained. It is created, and the display of the frame image of the corresponding video section on the screen is updated accordingly.
[0073]
The above is the basic display processing, editing operation, and processing of the display / editing device 3 in FIG.
Next, an example of a data structure and an example of a processing operation handled by the display / editing device 3 of FIG. 1 will be described with reference to FIGS.
FIG. 6 is an explanatory diagram showing an example of a data structure processed by the display / editing device in FIG. 1. FIGS. 7 to 11 are first to fifth processes of the display / editing device in FIG. It is a flowchart which shows a procedure example.
[0074]
As shown in FIG. 6, information on the entire video to be edited is stored in the media information 701. Further, a list of rows displayed in the frame image display area 151 in FIG. 2, that is, scene information 711 and 721 indicating a sequence of scenes is stored as a list structure.
The media information 701 includes, for a video file to be actually edited, the file name 702, the time length of the video (media length 703), and a pointer to a scene information list (scene column 704) used during editing processing. Is stored.
[0075]
Scene information 711 indicating one display line or one scene (“scene 1”) includes an icon image 712 used for a scene icon displayed in each line, and a comment 713 (character) of a scene displayed below the scene icon. Column), a start position 714 and an end position 715 of this scene in the video file of the scene, and a pointer 716 to scene information 721 (“scene 2”) of a scene to be displayed next in the sequence of scenes. It is composed of
[0076]
The processing shown in FIG. 7 shows a processing flow that is first executed by the display / editing apparatus 3 in FIG. 1 as a processing operation according to the present invention.
When starting to edit a video, a video file to be edited is specified. The process called in this case is the process flow for opening a file (step 800). Hereinafter, the contents will be described.
[0077]
First, in step 801, a data structure as shown in FIG. 6 is created from a specified video file. If the user has stored the data structure corresponding to this video file in the file, or if the name of this data structure file is specified together with the video file when opening the file, the processing in step 801 is performed. Not required.
[0078]
Next, in step 802, based on the data structure created in step 801 processing, a video is displayed in an expression form as shown in FIG.
Next, in step 803, information on the selected section used for editing is initialized. In this example, at the time of the initial display, the first frame position of the video file is selected.
Finally, in the loop process of step 804, the editing process corresponding to the user operation event is performed until the user instructs the end of the editing process.
[0079]
In the loop of step 804, first, an operation event such as a mouse click, menu selection, or cursor movement of the user is obtained by the processing of step 811. Next, a process corresponding to each user operation event is selected by a branch corresponding to the condition in step 812. For example, when the screen is clicked, screen click processing 821 is performed, and when the edit menu is selected, edit menu processing 822 is performed. Then, if the editing end menu (supposed to be in the file menu) is selected, the loop processing of step 804 ends.
[0080]
FIG. 8 is a flowchart showing details of the processing in step 801 in FIG.
This example shows a processing example of creating the media information of FIG. 6 which is a data structure necessary for the editing process from a specified video file.
First, in step 901, a video file name and a time length of the video file are set in a memory area (701 in FIG. 6) for storing media information.
[0081]
Next, in step 902, a new memory area for scene information (711 in FIG. 6) is generated, and the scene information is set at the head position (704 in FIG. 6) of the scene sequence of the previous media information.
Further, in step 903, the position “0” is set as the scene start position and the media length (time length) set in the media information is set as the end position in the generated scene information as initial values.
[0082]
Then, a loop process of step 904 is performed.
In the loop process of step 904, a process of counting a control variable i from 1 and generating a plurality of pieces of scene information is performed.
That is, first, in step 910, a frame image at the start position of the scene is set in the icon image of the current scene information, and a comment area representing the name of the scene is named "scene i" (i is Do).
[0083]
Next, in order to divide the current scene into logical delimiters, in step 911, the video is reproduced frame by frame from the start position of the scene, and a frame position where a scene change point where a scene changes is found is searched. . As a technique for obtaining a scene change point, an image difference between two adjacent frame images in a frame image sequence is obtained, and when the image difference becomes larger than a certain threshold, the frame image is determined. The position may be set as a scene change point position.
[0084]
Then, it is determined whether the loop of step 904 is completed. In the process of step 911, the position at the end of the video is also set as a scene change point. For this reason, if the position of the scene change point is the video end position under the conditions in the process of step 912, it is determined that the video reading has been completed.
If the video has not yet reached the end position, the process of step 913 is performed.
[0085]
In the process of step 913, a process of dividing the current scene information into two pieces of preceding and succeeding scene information at the position of the scene change point is performed.
First, a new scene information memory area is generated, and then the scene start position of the new scene information is set to the previous scene change point position, and the scene end position is set to the end position of the current scene information. Thereafter, the end position of the current scene information is updated to the value of "scene change point -1", and new scene information is set to the next pointer of the current scene information.
[0086]
At the end of the loop processing in step 904, in step 914, the previous new scene is set as current scene information used in the next loop.
If reading to the end of the video has been completed under the condition of step 912 in the loop of step 904, the process proceeds to step 921. Then, in step 921, the value “none” indicating the end of the list structure is set in the next pointer of the current scene information, and then the loop of step 904 is ended in step 922.
[0087]
FIG. 9 is a flowchart showing details of the processing in step 802 in FIG.
This example shows a processing example of displaying video data. First, in step 1001, a display position on the screen is set to the upper left of the screen. Next, in step 1002, the scene information at the head of the scene information list is obtained from the continuous media information as the display start scene. As a more preferable example, a scroll display function may be provided by changing the display start scene specified in step 1002. In this case, the display start scene may be changed in accordance with an operation such as a scroll bar or cursor movement.
[0088]
Next, in a loop process of step 1003, a process of displaying video data in a screen is performed. First, in step 1011, an icon image of a scene and a comment character string of the scene are displayed from the current scene information. Next, in step 1012, the contents from the start position to the end position of the scene are displayed on the screen. At this time, the shape to be displayed on the screen is a horizontally long bar-shaped area, and the display length is set to “(time length of scene) × display coefficient K”.
[0089]
For example, in the case of displaying video data in FIG. 2, a display is performed in which frame images from the start position to the end position of a scene are arranged side by side like a film. In the case of displaying audio data as shown in FIG. 3, the audio waveform in the scene may be displayed in a graph. Here, the display coefficient K is set in advance at the time of starting the program. Further, the display coefficient K can be used for zoom display of the screen. For example, by reducing the display coefficient K, even a scene having a long time can be displayed without protruding from the screen.
[0090]
Next, in step 1013, the display position is moved down by one line. The movement width at this time may be, for example, the height of the frame image × 2.
At the end of the loop processing of step 1003, the next pointer of the scene information is followed to obtain the next scene information. Then, the loop is performed while the display position is within the range of the screen, and when the display position is out of the screen, the loop processing of step 1003 is ended.
[0091]
In the next loop processing of step 1004, a time axis is displayed. That is, in step 1021, a vertical line is drawn for each display time width designated in advance, such as 30 seconds or 1 minute. The X coordinate position of the vertical line is “display time width × i × display coefficient K” when loop counts i = 0 to n.
[0092]
FIG. 10 is a flowchart showing details of the process of step 821 in FIG.
This example shows the processing operation when the screen is clicked. When the screen is clicked, first, in step 1101, a process for checking which scene on the screen is designated from the click position is performed. Of the clicked line position R is determined from the Y coordinate. In order to obtain the row position R, for example, the Y coordinate value may be divided by the display row width used in the process of step 1013 in FIG. Further, in step 1101, a process of obtaining the R-th scene information from the head of the media scene sequence is performed.
[0093]
In step 1102, a branch process according to the condition is performed. That is, it is determined whether the click position is the scene information display area 150 or the frame image display area 151 based on the clicked X coordinate. If the clicked position is in the scene information display area 150, in step 1111 a process of selecting the entire corresponding scene is performed. Here, the start position and the end position of the scene information obtained in the process of step 1101 are set as the selection start position and the selection end position.
[0094]
If the click position is in the frame image display area 151, the process of setting the selection position in frame units is performed after step 1113. First, in the processing of step 1113, the time position in the scene is calculated from the click position on the screen. Here, the time position C in the scene is obtained by dividing the X coordinate of the click position by the display coefficient K used for displaying the video. This time position is a time position where the start position of the corresponding scene is 0.
[0095]
In the conditional branch processing in step 1114, if the clicked time position C is longer than the length of the selected scene (= scene.end position−scene.start position), in step 1121, the time position C is set to the scene length. Perform correction processing.
When the time position C in the scene is determined in this way, in step 1115, a selection process is performed according to left click or right click. In the case of a left click, in step 1123, the selection start position is set to scene. Start position + (in the scene) time position C.
[0096]
In the case of right-click, in step 1125, the selection end position is set to scene. Start position + (in the scene) time position C.
At the end of the screen click processing, as the processing of step 1103, the area from the updated selection start position to the selected end position is displayed as a selected state (highlighted as indicated by reference numeral 160 in FIG. 2).
[0097]
FIG. 11 is a flowchart showing details of the process of step 822 in FIG.
This example shows a processing operation example when the edit menu is selected.
When the edit menu is selected, first, in step 1201, scene information corresponding to the currently selected section is obtained. The obtained information includes scene information where the selection start position is present (start scene information), scene information where the selection end position is present (end scene information), and the next scene information after the start scene information to one before the end scene information. Of intermediate scene information up to the scene information (intermediate scene sequence).
[0098]
To find start scene information, go to Media. It is sufficient to sequentially refer to the scene sequence from the top and search for scene information that satisfies the following: scene start position ≦ selection start position ≦ scene end position. Similarly, in order to search for end scene information, scene information that satisfies the condition of “head position of scene ≦ selection end position ≦ end position of scene” may be searched. Then, the intermediate scene sequence can be easily obtained from the first scene information and the last scene information.
[0099]
Next, in the conditional branch processing in step 1202, the selected edit menu is determined.
When the selected edit menu is “cut”, a process of cutting the selected section and copying the cut selected section to the clipboard is performed. That is, first, in step 1211, the selected section is stored for the clipboard. Specifically, first, a copy of the start scene, the intermediate scene sequence, and the end scene is created, and then the start position of the start scene copy is updated to the selection start position, and the end position of the end scene copy is updated to the selection end position. Finally, a process of storing a copy on the clipboard is performed.
[0100]
In the next step 1212, a process of actually deleting the selected section is performed. Specifically, first, an intermediate scene sequence is set as a media. It is deleted from the list of scene strings, and then the end position of the start scene is updated to the selection start position-1 and the start position of the end scene is updated to the selection end position + 1.
[0101]
If the selected edit menu is "insert" in step 1202, in step 1215, a process of inserting a scene sequence stored in the clipboard immediately after the start scene is performed.
If the selected edit menu is “filter” in step 1202, in step 1217, subtitle processing or blur processing is performed on each frame image from the selection start position to the selection end position in the video file. And other image processing.
[0102]
For example, in the case of a blur filter, a blur process is performed on a frame image extracted from a video file using an image processing technique called a moving average filter, and then the frame image is written back to a video file.
In the case of subtitle processing, a character string specified by the user is drawn on a frame image at a specified font and display position, and the drawn frame image is written back to a video file.
[0103]
In this example, for simplification of processing, image processing is directly performed when a filter menu is designated. However, desirably, instead of directly performing image processing, it is better to separately record information of repetitive editing processing and operate the image processing in a batch at the time pointed by the user, which is convenient for the user and Good processing efficiency.
At the end of such editing menu processing, in the processing of step 1203, the edited video is displayed again.
[0104]
Next, a description will be given of an expression technique and an editing operation technique by the display / editing device 3 in FIG.
FIG. 12 is an explanatory diagram showing a fifth processing operation example according to the present invention of the display / edit device in FIG.
This example shows a screen configuration when the concept of a page is introduced into the display processing operation of the display / editing device in FIG.
[0105]
Just as the word editing in a word processor has a concept called a page, the concept of a page is introduced into the display / editing technology of continuous media information in this example. In this example, time is used as a unit representing the size of a page.
FIG. 12 shows a screen example of the page unit display mode when the page width is set to 1 minute and the page height is set to 5 minutes.
[0106]
The feature of this example is that, out of “scene 2” (frame images 1301 to 1306), a portion (frame image 1305, 1306) exceeding the display time width of the page is displayed in a folded manner, and page vertical time axis 1311 Is displayed. In other words, “scene 2”, which is about 1 minute and 30 seconds in length here, is displayed as being separated into two frame images 1301 to 1304 for one minute from the beginning and the remaining frame images 1305 to 1306. ing. A time display area 1312 such as "5:00" is provided on the page vertical time axis.
[0107]
As described above, by introducing the display mode in which the page width and the height are considered, the operator can more easily understand the time length of the video. This mode is particularly suitable for checking a CM insertion position in a TV program.
In order to perform such display, the image display processing in step 1012 in FIG. 9 may be changed so that the scene is displayed for each time width. In the time axis display processing in step 1021, a change may be made so as to display not only the time axis of the vertical line but also the time axis of the horizontal line.
[0108]
FIG. 13 is an explanatory diagram showing a sixth processing operation example of the display / editing device in FIG. 1 according to the present invention.
This example shows a screen configuration example at the time of a combined display operation by the display / edit device in FIG.
If a video contains a large number of short scenes of a few seconds, such as a car chase scene sequence, the number of display lines on the screen becomes very large, making it difficult to grasp the logical structure of the scene. It is more convenient to handle such a scene sequence collectively as one scene on the screen.
[0109]
For such a reason, a display technique for expressing a plurality of scenes as one scene on the same line is prepared.
In the example of FIG. 13, the scene rows of “scene 1” of the scene block icon 110 of FIG. 2, “scene 2” of the scene block icon 120, and “control” of the scene block icon 130 are combined, and “new scene” (Scene block icon 1410).
[0110]
As the frame image sequence of the “new scene” of the scene block icon 1410, frame images 1411-1413, 1421-1423, and 1431 of the combined scenes are displayed. In particular, in this example, in order to emphasize the combination, the top frame images 1411, 1421, and 1431 of the scenes being combined are highlighted with a large frame image. Of course, highlighting is not necessary.
[0111]
In order to provide such a combined expression, in the data structure of FIG. 6, a new pointer storage item called “combined scene sequence” is provided in the scene information 711, and the scene sequence combined with the pointer is separately added. You should keep it. When displaying a scene in step 1012 of the video display processing procedure in FIG. 9, scene information and a frame image may be appropriately extracted from the scene information sequence in the “combined scene sequence” and displayed on the screen.
[0112]
FIG. 14 is an explanatory diagram showing a seventh processing operation example of the display / editing device in FIG. 1 according to the present invention.
This example is a screen configuration example at the time of a scene outline display operation by the display / editing device in FIG.
When the combined display of scenes as shown in FIG. 13 is used, each scene has a hierarchical relationship such as further combining the combined scenes.
[0113]
In this example, such a hierarchical relationship is directly handled. Under the “new scene” of the scene block icon 1410 described with reference to FIG. 13, “scene 1” (scene block icon 110), which has been combined and edited, "Scene 2" (scene block icon 120) and "control" (scene block icon 130) are displayed hanging.
[0114]
Also, a frame image sequence corresponding to the combined scene sequence is displayed in the frame image display region 1551 in an indent display in which the display start position is slightly shifted to the right (region 1501).
In this manner, by displaying the hierarchical structure of the scene in an indent or hanging, it is possible to easily grasp the logical structure of the video as if outline editing of a chapter or section in a word processor was used.
[0115]
In order to perform such display, a recursive function call such as “display of video (parameter is a scene. Combined scene sequence)” is performed between the processes of steps 1012 and 1013 in the video display process of FIG. What is necessary is just to insert processing.
The video editing operation or the like is performed after determining in which connected hierarchical area (the area 1551 or the area 1501) the editing processing has occurred based on the click position, and then performing the conventional processing on the video scene sequence in the corresponding area. Just do it.
[0116]
FIG. 15 is an explanatory diagram showing an example of comparison between the editing function of the display / editing device in FIG. 1 and the text editing function of a word processor.
In the table of this example, the functions 1601-1608 of each word processor editing (“character string”, “1 line”, “font change”, “line spacing adjustment”, “search”, “header / footer”, “merge print” , "Sentence check"), a function ("video editing of the present invention") of the continuous media information display / edit technique of the present example.
[0117]
In this example, “character image” is “frame image sequence (= video)”, “1 line” is “1 scene”, “font change” is “special effect (blur)”, “line spacing adjustment” is “2 scenes” "Wipe in between", "Search""Similar video search", "Header / Footer""Opening / Ending", "Inset printing""Video editing template", "Sentence check""Illegal video inspection" Are associated with each other.
[0118]
As described above, the continuous media information display / editing apparatus 3 and display / editing method of the present example can easily edit a frame image sequence, that is, a video, like a character string edited by word processing editing. That is, in this example, not only the frame image sequence is arranged in a single line (1601), but also a line feed is displayed for each meaningful video scene as in line feed editing (1602) in word processing editing.
[0119]
Considering this further, various editing techniques effective for text editing by a word processor can be easily applied to the video editing technique using the display / editing apparatus and method of the present embodiment. For example, as an example in which the operation (1603) of changing the font of the character string selected in the word processor is applied to the video editing of this example, the editing operation technique for the selected video section described with reference to FIG.
[0120]
Based on the idea of applying the editing function of such a word processor to the editing function of the display / editing apparatus of the present example, it is possible to consider a video editing technique that is easier to use as shown in the table of FIG.
[0121]
For example, consider a case in which an operation method of line spacing adjustment (1604) for adjusting the interval between lines in a word processor is applied to video editing.
In a word processor, the length between lines can be adjusted by selecting two or more lines and selecting a line spacing adjustment menu. In the case of video editing, the line spacing adjustment in text editing can be regarded as a special effect such as a wipe between two scenes. From this, a new method of specifying an inter-scene special effect, in which two or more scenes are selected and then an inter-scene special effect setting menu is selected as a video editing operation, can be easily considered.
[0122]
From the same idea, the character string search function (1605) referred to in a word processor is a video editing technology that searches for a video similar to the specified video and sets the found video section to a selected state in the video editing of this example. Applicable. As a technique for finding such a similar video, a technique for finding a video section that satisfies conditions based on a keyword assigned to the video, a technique for finding a video section in which the image difference amount of a frame image is smaller than a certain reference value, and the like can be considered. .
[0123]
The editing function (1606) of a page header or footer in a word processor can be applied as an opening / ending video editing function in the video editing of this example. Also, the insert printing function (1607) referred to in a word processor can be applied as a video editing template function of preparing a template of video editing contents and inserting various videos into the template in the video editing of this example.
[0124]
Furthermore, word processing functions such as sentence check and spell check (1608) provide an illegal video inspection function, such as automatically pointing out subliminal video scenes (scenes shorter than one second) that are said to affect human subconsciousness. Can be considered. As a technique for automatically finding the subliminal video scene, for example, a "method scene display method and apparatus" described in JP-A-09-074575 is exemplified.
[0125]
As described above with reference to FIGS. 1 to 15, in the continuous media information display / editing apparatus and the continuous media information display / editing method of the present embodiment, for example, each image frame of a moving image is continuously displayed on a screen. In the case of displaying in the horizontal direction, a line feed is performed with a change point of the scene as a delimiter, and each scene is displayed line by line. Then, a line feed mark is clearly indicated at each line feed position to make it easier to understand the scene unit. If it exceeds the display frame of the screen, it is wrapped around the display frame and displayed on the next line. In this case, the line feed mark is not specified.
[0126]
Also, a scale line representing a predetermined elapsed time is displayed from the head of each line on the screen and in the direction of the subsequent line from the head line, so that the time position of each frame and each scene can be easily understood. .
In addition, scene block icon information created by the related art corresponding to all frames displayed in this row is also displayed in the area before the head of the row.
[0127]
Further, when the operator designates a frame to be edited, the target frame portion is highlighted, so that the quality of the designation can be easily determined, and the editing operation such as moving the frame is facilitated. In addition, for example, when a plurality of frames are combined into one line, each combined portion is displayed larger than the other portions, so that the result of the combined editing can be easily determined. Further, the display state of each frame line before the combined editing is displayed in a window from the line following the combined edited line.
[0128]
As described above, according to the present embodiment, the frame image sequence of the continuous video data is not lined up, but is displayed as a line feed in each meaningful section, so that both the logical structure of the video and the frame configuration of the video are displayed. It becomes easy to grasp. Further, since the present invention can be similarly applied to continuous media other than video such as audio information, it is easy to grasp both the logical structure of the continuous media and the components of the continuous media.
In addition, such an easy-to-understand visual understanding also facilitates continuous media editing operations.
[0129]
The present invention is not limited to the embodiment described with reference to FIGS. 1 to 15 and can be variously modified without departing from the gist thereof. For example, in this example, when the display frame exceeds the display frame of the screen, the display is folded back to the display frame and displayed on the next line. However, a scroll operation or the like may be performed without displaying the return frame. .
Further, in this example, an example in which a mouse is used in the operation of selecting a video section is described. However, similarly to a normal word processing operation, a video section can be selected using a cursor key or the like of a keyboard.
[0130]
In this example, the storage medium of the continuous media is a storage medium such as a video tape recorder. However, the present invention is not limited to this. For example, a continuous media in a streaming format such as a TV or a radio may be used. In the case of such streaming-type continuous media, the end position of the continuous media may be treated as virtually infinite.
[0131]
【The invention's effect】
According to the present invention, in an editing process by a computer, an operator can easily understand the logical structure of continuous media such as video and audio, and at the same time, its constituent units / elements (image frames for video and waveform data for audio). ) Can be easily understood, and the editing operation of continuous media such as video and audio using a computer can be efficiently performed.
[Brief description of the drawings]
FIG. 1 is a block diagram showing one embodiment of a configuration according to the present invention of a continuous media information display / editing apparatus of the present invention.
FIG. 2 is an explanatory diagram showing a first processing operation example of the display / editing device in FIG. 1 according to the present invention;
FIG. 3 is an explanatory diagram showing a second processing operation example of the display / editing device in FIG. 1 according to the present invention;
FIG. 4 is an explanatory diagram showing a third processing operation example of the display / editing device in FIG. 1 according to the present invention;
FIG. 5 is an explanatory diagram showing a fourth processing operation example according to the present invention of the display / edit device in FIG. 1;
FIG. 6 is an explanatory diagram showing an example of a data structure processed by the display / edit device in FIG. 1;
FIG. 7 is a flowchart showing a first processing procedure example according to the present invention of the display / editing apparatus in FIG. 1;
FIG. 8 is a flowchart showing a second processing procedure example of the display / editing apparatus in FIG. 1 according to the present invention;
FIG. 9 is a flowchart illustrating a third processing procedure example according to the present invention of the display / editing apparatus in FIG. 1;
FIG. 10 is a flowchart illustrating a fourth processing procedure example according to the present invention of the display / editing apparatus in FIG. 1;
FIG. 11 is a flowchart showing a fifth processing procedure example according to the present invention of the display / editing apparatus in FIG. 1;
FIG. 12 is an explanatory diagram showing a fifth processing operation example according to the present invention of the display / editing device in FIG. 1;
FIG. 13 is an explanatory diagram showing a sixth processing operation example according to the present invention of the display / editing apparatus in FIG. 1;
FIG. 14 is an explanatory diagram showing a seventh processing operation example of the display / editing device in FIG. 1 according to the present invention;
FIG. 15 is an explanatory diagram showing an example of comparison and correspondence between an editing function by the display / editing device in FIG. 1 and a text editing function in a word processor;
FIG. 16 is an explanatory diagram showing a screen display example in the conventional frame-based editing technology.
FIG. 17 is an explanatory diagram showing a screen display example in a conventional scene-based editing technique.
[Explanation of symbols]
1: display device, 2: input device, 3: display / edit device, 4: auxiliary storage device, 5: continuous media information input device, 6: optical disk, 7: optical disk drive device, 8: display processing unit, 8a: frame Display section, 8b: break detection section, 8c: line feed section, 8d: line feed mark display section, 8e: display frame setting section, 8f: return section, 8g: horizontal time scale display section, 8h: vertical time scale display section , 8i: scene block icon display section, 9: edit display processing section, 9a: combined display processing section, 9b: outline edit display processing section, 10: data conversion section, 100: window, 110, 120, 130, 140: scene Block icons, 111 to 113: frame images, 121 to 124: frame images, 121a to 122a: frame images, 131 to 132: frame images, 141 to 141 43: frame image, 150: scene information display area, 151: frame image display area, 153, 154: time scale line, 156: line feed mark, 160: highlight area, 170: mouse cursor, 181: edit menu, 183: Edit pull-down menu, 184: menu, 211, 221: bar area, 251: audio waveform display area, 260, 560-564: highlight display area, 300: window, 301: scroll bar, 310, 320, 330: line, 321: icon, 340, 350: audio track, 400: window, 410 to 440: scene block icon, 701: media information, 711, 721: scene information, 702: file name, 703: media length, 704: scene sequence, 712: Icon image, 713: Rice G, 714: start position, 715: end position, 716: pointer, 1301 to 1306: frame image, 1311: page vertical time axis, 1312: time display area, 1410: scene block icon, 1411 to 1413, 1421 to 1423 1431: frame image, 1501: area, 1551: frame image display area, 1601 to 1608: function (“character string”, “1 line”, “font change”, “line spacing adjustment”, “search”, “header / footer” , "Insert print", "sentence check").

Claims (15)

Displayed on the screen of the display device of the image frame of video as continuous media information, continuous media that the display and editing of the video based on the the operation instruction input from the input device performed by the image frame that is displayed on the screen An information display / editing device,
Means for sequentially displaying the image frames in one direction of the screen,
Means for detecting logical breaks in the image frames displayed in order ,
Means for line-feeding the display of the image frame at each break detected by the means. 2. The continuous media information display / editing device according to claim 1, further comprising means for specifying a line feed mark at the detected break position. In the continuous media information display / editing device according to any one of claims 1 and 2, means for restricting a display frame of a screen for displaying the image frame, and a display frame restricted by the means. Means for displaying the surpassed image frame on the next line by folding back the image frame . 4. The continuous media information display / editing device according to claim 1, further comprising means for displaying a scale line indicating a predetermined elapsed time from a line head of the image frame. Continuous media information display and editing device. 5. The continuous media information display / editing device according to claim 1, wherein a scale line representing a predetermined elapsed time is displayed from a first line on which the image frame is displayed to a subsequent line direction. A display / editing device for continuous media information, characterized by comprising means. In the display and editing unit of the continuous media information as claimed in any one of claims 5, before the area of the line head displaying the image frame, in correspondence with the image frame displayed on the row A continuous media information display / editing device, further comprising means for displaying separately created scene block icon information in which a time length is displayed as a rectangular parallelepiped depth . 7. The display / editing apparatus for continuous media information according to claim 1, further comprising means for displaying a portion of the image frame specified through the input device in a specific expression different from other portions. An apparatus for displaying and editing continuous media information, characterized in that it is provided. 8. The continuous media information display / editing device according to claim 1, wherein a portion of the image frame specified via the input device is edited in accordance with an editing operation instruction input from the input device. A display / editing device for continuous media information, characterized by comprising means for performing the operation. 9. The continuous media information display / editing device according to claim 1, wherein a plurality of lines of image frames are combined and displayed on one line in accordance with an editing operation instruction input from the input device. Means for displaying continuous media information, wherein said means for displaying each connected portion of said plurality of rows of image frames is expressed differently from other portions. 10. The continuous media information display / editing apparatus according to claim 9, further comprising means for displaying a display state before the combined editing in a window from a line following the line in which the plurality of lines of image frames are combined and displayed. A continuous media information display / editing device. The continuous media information display / editing device according to any one of claims 1 to 10, wherein a time position of a corresponding image frame is determined from a two-dimensional coordinate position on the screen specified via the input device. A display / editing device for continuous media information, comprising a calculating means. 12. The continuous media information display / editing device according to claim 1, wherein each of the continuous media information display / editing devices is configured to perform processing on the other continuous media information including at least audio waveform data instead of the video image frame. An apparatus for displaying and editing continuous media information, which performs display and editing processing. A continuous media of a processing device that displays an image frame of a video as continuous media information on a screen of a display device, and edits the video in units of the image frame displayed on the screen based on an operation instruction input from an input device. A method of displaying and editing information,
13. A method for displaying and editing continuous media information, comprising displaying and editing said image frame by processing of each means in the apparatus for displaying and editing continuous media information according to any one of claims 1 to 12. . 14. The display / editing method for continuous media information according to claim 13, wherein the display / editing process of each unit is performed on other continuous media information including at least audio waveform data instead of the image frame of the video. A method for displaying / editing continuous media information, characterized in that it is performed. 13. A recording medium for recording data and a program read and processed by a computer, wherein the computer executes a processing procedure by each means in the continuous media information display / editing apparatus according to claim 1. A recording medium characterized by recording a program for causing the program to execute.

Images