JP2002281433A - Device for retrieving and reading editing moving image and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a retrieval and reading editing device capable of efficiently and accurately realizing editing processing of a frame unit and a recording medium on which a processing program is recorded. SOLUTION: Editing points (editing start point/end point) are designated with frame accuracy from retrieval results of cut point retrieval, telop retrieval, audio classification and frame unit retrieval being subjected to thumbnail reading in a retrieval result reading displaying part 76. A frame unit editing part 100 decodes only a portion of data before and after the editing points into a baseband to perform editing and recompressing and performs processing with compressed data as they are with respect to data other than the portion of data before and after the editing points. A splice editing part 101 performs confirmation by splice editing (partial editing of joint part) and splice reproduction (partial reproduction of joint part) to check apparent consistency at the time when a moving image at each editing point (joint part) is reproduced.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and a recording medium for retrieving / viewing / editing digital moving image contents such as MPEG, and more particularly to a moving picture capable of efficiently and accurately realizing frame-by-frame editing processing using thumbnail search / browsing results. The present invention relates to an image retrieval / browsing / editing apparatus and a recording medium.

[0002]

2. Description of the Related Art In the early stage of searching, browsing and editing of digital moving image contents, digital moving image data stored on a hard disk is edited by designating an editing point by fast-forwarding or rewinding using a random access function of the hard disk. However, it takes a lot of time to edit a long content content.

[0003] Thereafter, the apparatus invented by the present applicant, for example, described in "Moving image search and editing apparatus, Japanese Patent Laid-Open No. 10-66008", employs a technique that combines macro search with cut point information and micro search in frame units. Has greatly reduced the time required to specify an edit point.

[0004]

However, accurate detection of a telop appearance frame that characterizes a semantic change of a scene in a news video or the like, and detection of a point at which a human voice switches to BGM in the same scene cannot be performed. Depending on the content of the video, there were many cases where time and human work were required to determine a fine editing point.

[0005] When editing digital video contents in MPEG format on compressed data, a GOP (Group of Picture
s) Editing was possible in units, but when editing with frame accuracy, it was necessary to decode all frames once and then edit and recompress them. Since a storage disk is required, image quality may be degraded due to decoding and re-encoding of MPEG data, and the editing process may take a long time.

Further, in order to check the delicate connection between the image and the sound at the seam portion at each editing point, which is an important point when inspecting the video data after the editing process, all the editing processes must be performed once. It is necessary to repeat the process of making fine adjustments to the edit points, performing the edit processing again, and checking if there is a problem.

[0007] Further, when it is desired to perform the editing work once completed once again from that point on or to perform mutual editing work with another editing system, the user must again specify the moving image content and create each edit point list again. Had to be done by work.

An object of the present invention is to solve the above-mentioned problems of the prior art, and to perform cut point search, telop search, audio classification, or frame unit search from a video file stored in a storage device in a complex manner. It is an object of the present invention to provide a search / browsing / editing apparatus capable of efficiently and accurately realizing editing processing in units of frames using a thumbnail search / browsing result, and a recording medium storing a processing program.

Another object of the present invention is to perform frame-accurate editing on MPEG-compressed data in the editing process, thereby preventing deterioration in image quality, saving temporary storage disk by not performing full decompression, and realizing high-speed processing. It is another object of the present invention to provide a recording medium on which a simple editing processing program is recorded.

Another object of the present invention is to provide a recording medium in which a splice editing / reproducing processing program for realizing a visual consistency check of a joint portion at each editing point before executing the editing processing is provided.

Still another object is to edit a list of edit sections at the time of editing a moving image by using SMPTE (Society of Motion Picture & Tel).
EDL (Edit Decision Li) based on evision Engineers
By providing input / output as st), a search / browsing / editing apparatus capable of restarting the previously completed editing work from that point or sharing the editing work with an external system and a recording medium recording a processing program are provided. It is in.

[0012]

SUMMARY OF THE INVENTION In order to achieve the above object, the present invention provides a moving image search / browsing / editing apparatus for editing digital moving image contents such as MPEG, which includes a cut point search, a telop search, and an audio classification. The first characteristic lies in that there is provided a means for designating an edit start point / edit end point by using a thumbnail search and browsing result using frame unit search in combination.

According to this feature, the edit start point / end point is efficiently and accurately designated by using the thumbnail search and browsing results using the cut point search, the telop search, the audio classification, and the frame unit search, and It becomes possible to perform frame unit editing.

Further, according to the present invention, when performing frame unit editing processing of MPEG content, only part of data before and after the editing point is decoded into baseband and edited / recompressed.
A second feature is that a means is provided for processing the compressed data as it is except for some data before and after the edit point.

According to this feature, it is possible to prevent deterioration of the image quality due to not completely decompressing the MPEG content, to save a temporary storage disk, and to realize high-speed processing.

Also, the present invention provides a process of decoding only a part of data before and after an edit point designated using a thumbnail search and browsing result to a baseband when performing frame unit edit processing of MPEG content. A third feature is to provide a computer-readable recording medium in which a program for performing a process of editing at a point and a process of recompressing the decoded data is recorded.

According to this feature, it is possible to provide a computer-readable recording medium that stores a program capable of preventing deterioration of image quality due to not completely decompressing MPEG content, saving a temporary storage disk, and realizing high-speed processing. it can.

[0018]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below in detail with reference to the drawings. First, an embodiment of a video search / viewing apparatus suitable for use with the present invention will be described below.

FIG. 1 is a block diagram showing a schematic configuration of a first embodiment of the video search and browse device.

In the first embodiment, a thumbnail is extracted from a moving image in real time using cut point search information or a time code string in a frame unit / second unit, thereby providing a high-precision thumbnail in a frame unit / second unit. Searching and browsing can be performed, and by specifying the time as a reproduction start / end point, high-precision section reproduction can be performed on a local host without preparing an additional storage device for storing all thumbnail information. Is made possible.

In FIG. 1, the moving image storage device 12 stores moving image contents in advance. Cut point search information indicating a scene break is detected in advance from the stored moving images and stored in the search information storage device 11. Here, the browsing thumbnail such as the JPEG file corresponding to the time code included in the cut point search information does not need to be extracted or stored unlike the conventional apparatus.

Regarding the retrieval method of the cut point retrieval information,
For example, according to the present applicant, "moving image cut point detection device
The method described in JP-A-11-252509 can be used. Regarding the description of the cut point search information, as shown in FIG. 2, a cut point time code string (C1, C2, C3,...) 20 in which cut points and detected time codes are arranged in chronological order. Is used.

When a cut point search start command is input by designating a video file name from the search / playback command input unit 18 in FIG. 1, the control unit 15 retrieves a cut point search start time code and a cut point browsing thumbnail. The number is output to the search unit 13. The search unit 13 obtains a cut point browsing time code string corresponding to a thumbnail to be browsed from the cut point search information stored in the search information storage device 11 based on the input information, and sends it to the control unit 15. I do. Control unit
15 passes the obtained cut point browsing time code sequence to the thumbnail extracting unit 14, extracts the corresponding thumbnail sequence from the thumbnail extracting unit 14, and outputs the cut point search result to the search result browsing display unit 17 via the control unit 15. To browse.

When searching and browsing more detailed contents in the time direction from the cut point position or an arbitrary position, the time code of the detailed search browsing start position, the display accuracy in the time direction such as a frame unit and a second unit, and Search / playback command input unit 1 for the number of browse thumbnails retrieved
8 to the control unit 15. The control unit 15 generates a browsing time code sequence at regular time intervals from the obtained information, passes the browsing time code sequence to the thumbnail extracting unit 14, extracts the corresponding thumbnail sequence, and searches through the control unit 15 The result browsing display unit 17 browses and displays the search results at regular time intervals.

With respect to the section reproduction of the video, a time code at which the reproduction of the video is to be started and a time code at which the reproduction of the video is to be ended are indicated on the browse display of the retrieval result browsing display unit 17. Then, the information of the reproduction start command including the reproduction start / end time code of the frame precision and the information of the video file name is input from the search / reproduction command input unit 18 to the control unit 15. These pieces of information are passed from the control unit 15 to the video reproduction unit 16, and the video reproduction unit 16 extracts data of the specified section of the specified video content from the moving image content storage device 12, and extracts the data from the video display unit. At 19, section playback from the playback start point to the end point is performed. This enables local section reproduction with frame accuracy.

Here, the operation of the thumbnail extracting unit 14 will be described in detail with reference to the conceptual diagrams of FIGS. The thumbnail extracting unit 14 includes a moving image partial decoding unit 22, a cache 23, and a thumbnail generating unit 24. When the request time code and the retrieved number of thumbnails are input to the control unit 15, the control unit 15 checks whether the corresponding thumbnail exists in the cache 23, and if there is, the control unit 15 outputs the corresponding thumbnail from the cache 23 via the switch unit 26. Output. If it does not exist, the data is read out from the moving image file 20 in the video storage device 12 via the switch unit 25, and only the necessary frames are partially decoded by the moving image partial decoding unit 22.
Generate thumbnails etc. The generated thumbnail is output through the switch unit 26, and is simultaneously cached.
Stored temporarily in 23.

When the compression format of a moving image is MPEG, M
As shown in FIG. 4, the frame structure of PEG is composed of an I-picture for intra-frame encoding and P and B pictures for inter-frame encoding, so that the requested time code indicates an inter-frame encoded part. In this case, it is necessary to decode one or more preceding frames. Therefore, decoding processes of the same frame may overlap with each other, and the processing efficiency is reduced. In order to avoid this, the minimum unit of partial decoding of the moving image is set to GOP (Group Of Pictures), all frames in the GOP including the request time code are decoded, and thumbnails of the request time code are output and stored in the cache 23 at the same time. I do. As a result, when the next request time code is temporally close to the previous request time code, the probability of secondary use of the cached thumbnail increases, and the processing speed can be improved.

The mapping of the requested time code, the frame time code in the moving image, and the time code of the extracted thumbnail in the thumbnail extracting unit 14 will be described in detail with reference to FIG. Request time code strings (R1, R2, R3, R4, ...) at fixed time intervals such as cut points or frame units of thumbnails to be viewed
..) 27 are request time codes arranged in chronological order, and appropriate thumbnail strings (Th (R1), Th (R2), Th (R3), Th (R4),... ) To extract.

The method of determining the thumbnail Th (Rm) corresponding to the m-th request time code Rm [second] is as follows. Is performed. In other words, in the case of extracting the cut point thumbnail, the output thumbnail Th (Rm) using the m-th cut point time code Cm [second] as the request time code Rm [second].
Includes a thumbnail Th generated from a moving image frame having a time code of tn [seconds] as shown in the following equation (2).
(tn) is used.

Here, tn is a time code sequence (t1, t2, t3,...) Of a frame included in the moving image sequence 28.
.., tk) Indicates the n-th frame time code in 29. For example, in the case of an NTSC video signal, tn is given by the following (3)
It looks like an expression. tn ≦ Rm <tn + 1 [sec] (1) Th (Rm) = Th (tn) (2) tn = (1 / 29.97) × n [sec] (n = 0,1, 2, ...) ... (3)

When extracting thumbnails from moving image content compressed in the MPEG format, due to the nature of the compression method,
GOP (Group Of P) including a frame corresponding to time tn
It is necessary to extract the corresponding thumbnail by decoding from the first frame of the ictures).

Next, the search result browsing display unit 17 (FIG. 1)
) Will be described. FIG. 6 shows a browsing display example of cut point search browsing. Cut point time code string (Cn, Cn + 1, Cn + 2,
Cn + 3, Cn + 4) 30 is displayed, and at the same time the cut point thumbnail sequence (Th (Cn), Th (Cn + 1), Th (Cn + 2), Th (Cn + 3), Th (Cn +
4)) 31 is displayed in synchronization with it. Left skip button
If you press 32, the time code string (Cn-5, Cn-4, Cn-3,
Cn-2, Cn-1) and cut point thumbnail sequence (Th (Cn-5),
Th (Cn-4), Th (Cn-3), Th (Cn-2), Th (Cn-1)) are displayed. Similarly, when the right skip button 33 is pressed, the time code string one span after the currently displayed time code string (Cn + 5, Cn + 6, Cn + 7, Cn + 8, Cn + 9) And the cut point thumbnail sequence (Th (Cn + 5), Th (Cn + 6), Th (Cn + 7), Th (Cn
n + 8), Th (Cn + 9)) are displayed.

FIG. 7 shows an example of a browsing display for searching and browsing in frame units. Time code string in frame units (F
n, Fn + 1, Fn + 2, Fn + 3, Fn + 4) 34 are displayed at the same time as the frame unit thumbnail row (Th (Fn), Th (Fn + 1), Th (Fn +
2), Th (Fn + 3), Th (Fn + 4)) 35 are displayed in synchronization with it. When the left skip button 36 is pressed, the time code sequence (Fn-5, Fn-4, Fn-3, Fn-2, Fn-1) and the cut one time before the currently displayed time code sequence Point thumbnail sequence (Th (Fn-5), Th (Fn-4), Th (Fn-3), Th (Fn-2),
Th (Fn-1)) is displayed. Similarly, when the right skip button 37 is pressed, the time code sequence (Fn + 5, Fn + 6, Fn + 7, Fn +
8, Fn + 9) and cut point thumbnail row (Th (Fn + 5), Th
(Fn + 6), Th (Fn + 7), Th (Fn + 8), Th (Fn + 9)) are displayed. Browsing display by time code in frame unit can be changed by changing the time code in frame unit to GOP unit, 1 second unit, 5 seconds unit, etc.
Browsing display at equal intervals with various time precisions is possible.

Also, as shown in “Relationship between cut point time code and frame unit time code” in FIG. 8, browsing display in cut point search and browsing and browsing display at equal intervals such as frame units are combined. An application is also possible in which a rough time position is specified by the cut point browsing display, and the frame position browsing display is finely displayed from that position.

FIG. 9 shows an example of a search / browsing / playback screen using the cut point browser and the frame unit browser individually or in combination.

The cut point browser 38 operates in the same manner as described with reference to FIG. The cut point browser 38 is suitable for roughly grasping the contents, and is capable of designating a reproduction start / end position. By pressing the playback start time point button 42 while the thumbnail at the time position on the cut point browser 38 is selected, the time code is displayed in the playback start point display area 41 and the playback start time point button 42 Display the thumbnail corresponding to. Similarly, by pressing the reproduction end time point button 44, the time code is displayed in the reproduction end point display area 43, and at the same time, the corresponding thumbnail is displayed on the reproduction end time point button 44. Next, by pressing the playback start button 45, the section playback from the designated start point to the end point by specifying the cut point is started on the playback display screen 46. When the reproduction start button 45 is pressed without specifying the reproduction start / end positions, reproduction is performed by specifying a cut point in the entire section from the beginning to the end of the moving image content.

The frame unit browser 39 operates in the same manner as described with reference to FIG. Browsing interval setting menu 40
When you open, the frame unit, GOP (normally about 0.5 seconds) unit, 1
A display time interval such as a unit of second or a unit of 5 seconds is displayed. By selecting this display, the display time interval can be changed. Hereinafter, the frame-based browser 39 is referred to as an equally-spaced browser in the sense of including the frame-based browsing function.

When a certain thumbnail is selected in the cut point browser 38, the equally-spaced browser 39 displays an equally-spaced browsing display from that time to grasp the detailed contents in the time direction and specify the reproduction start / end position. It has become. By pressing the playback start time point button 42 with the thumbnail at a certain time position selected on the equally spaced browser 39, the time code is displayed in the playback start point display area 41 and at the same time, the playback start time point button 42 Display the thumbnail corresponding to. Similarly, by pressing the reproduction end time point button 44, the time code is displayed in the reproduction end point display area 43, and at the same time, the corresponding thumbnail is displayed on the reproduction end time point button 44. Next, by pressing the playback start button 45, the section playback from the start point to the end point specified with the frame accuracy is started on the playback display screen 46. When the reproduction start button 45 is pressed without specifying the reproduction start / end positions, the entire section from the beginning to the end of the moving image content is reproduced.

As described above, the first of the present video search and browse devices
According to the embodiment of the present invention, rough thumbnail search and browsing can be performed by the cut point browser 38, and high-precision thumbnail searching and browsing can be performed on a frame-by-second basis on the equally-spaced browser 39. By designating the time as a reproduction start / end point on 39, high-precision section reproduction can be performed.

Next, a description will be given of a second embodiment of the video search / browsing apparatus. In this embodiment, thumbnail search and browsing can be performed using telop search information instead of the cut point search information described in the first embodiment, and the time is designated as a reproduction start / end point. The advanced section reproduction can be performed on the local host.

In FIG. 10, components other than the search information storage device 47 are the same as those in FIG. 1 and operate the same.

The telop search information describing the start time of a telop scene such as news is detected in advance from the moving images stored in the moving image storage device 12 in advance, and stored in the search information storage device 47. Here, the browsing thumbnail such as the JPEG file corresponding to each telop start point time code described in the telop search information need not be extracted or stored. Regarding a telop search method, for example, a method described in a patent application “Application for detecting telop area in moving image: Japanese Patent Application No. 2000-248794” by the present applicant can be used.

The description of the telop search information is shown in FIG.
1, a telop start point time code sequence (T1, T2) in which time codes indicating the beginning of a scene in which a telop appears are arranged in chronological order in the same manner as the cut point search information description (see FIG. 2). , T3, ...) 48 is used.

When a telop search start command is input by specifying a video file name from the search / playback command input unit 18, the control unit 15 searches the telop search start time code and the number of retrieved telop start point browsing thumbnails. Output to the unit 13. The search unit 13 acquires a telop start point browsing time code string corresponding to the thumbnail to be browsed from the telop search information stored in the search information storage device 47 based on the input information. The control unit 15 converts the obtained telop start point browsing time code string into the thumbnail extraction unit 14
To retrieve the corresponding thumbnail row, and browse and display it as a telop start point search result on the search result browsing display unit 17 via the control unit 15.

The method of searching for and browsing more detailed contents in the time direction from the telop starting point position or an arbitrary position and the method of reproducing a video section are the same as those in the first embodiment.

The mapping of the requested time code, the frame time code in the moving image, and the time code of the extracted thumbnail in the thumbnail extracting section 14 is the same as in the first embodiment, as shown in FIG. The request time code string (R1, R2, R3, R4,...) 27 is shown in FIG.
Telop start point time code string (T1, T2, T3,…) 48
Is performed by substituting

FIG. 12 shows an example of a browsing display for telop search and browsing. Telop start point time code string (Tn, Tn
+1, Tn + 2, Tn + 3, Tn + 4) 49 is displayed and at the same time the telop start point thumbnail sequence (Th (Tn), Th (Tn + 1), Th (Tn + 2),
Th (Tn + 3), Th (Tn + 4)) 50 is displayed in synchronization with it.
When the left skip button 51 is pressed, the time code string (Tn
-5, Tn-4, Tn-3, Tn-2, Tn-1) and the telop starting point thumbnail sequence (Th (Tn-5), Th (Tn-4), Th (Tn-3), Th (Tn -2),
Th (Tn-1)) is displayed. Similarly, right skip button 52
By pressing, the time code sequence (Tn + 5, Tn + 6, Tn + 7, Tn +
8, Tn + 9) and telop start point thumbnail row (Th (Tn +
5), Th (Tn + 6), Th (Tn + 7), Th (Tn + 8), Th (Tn + 9)) are displayed.

As shown in "Relationship between telop start point time code and frame unit time code" in FIG. 13, browsing display in telop search and browsing and display of frame unit and the like described in the first embodiment are performed. An application in which a rough time position (for example, the head of each news item) is specified in the telop starting point browsing display by combining with the equal-interval browsing display (see FIG. 7), and the frame-unit browsing display is used to display finely from that position. Is also possible.

FIG. 14 shows an example of a search / browsing / reproducing screen using the telop search / browsing browser and the frame unit browser individually or in combination.

The operation of the telop search / browser browser 53 is the same as the operation described with reference to FIG. Telop Search Browser 53
In a news video or the like in which a telop is likely to appear at the beginning of a scene at which a scene is switched, the head of each item can be set and the reproduction start / end position can be specified. With the thumbnail at a certain time position on the telop search browser 53 selected, the playback start time point button 42
By pressing, the time code is displayed in the reproduction start point display area 41 and at the same time, the reproduction start time point button 42
Display the corresponding thumbnail above. Similarly, by pressing the playback end time point button 44, the playback end point display area is displayed.
At the same time, the time code is displayed on 43 and the corresponding thumbnail is displayed on the reproduction end time point button 44. Next, by pressing the playback start button 45, the section playback from the designated start point to the end point by specifying the telop start point is started on the playback display screen 46. When only the reproduction start position is specified and the reproduction start button 45 is pressed, the section from the specified start point to the end of the content is reproduced. When the reproduction start button 45 is pressed without specifying the reproduction start / end positions, the entire section from the beginning to the end of the moving image content is reproduced.

The operation of the equally-spaced browser 39 for displaying the frame unit is the same as that of the first embodiment.

Next, a description will be given of a third embodiment of the video search / browsing apparatus. In this embodiment, thumbnail search and browsing can be performed by using audio classification information instead of the cut point search information described in the first embodiment, and the time is designated as a reproduction start / end point. Thus, more advanced section reproduction can be performed on the local host.

In FIG. 15, the search information storage device 54,
The operation other than the search unit 55, the control unit 56, and the search result browsing display unit 57 is the same as that of the first embodiment shown in FIG. Video storage device
Audio classification information indicating whether the audio belongs to voice, music, cheering noise, or silence is detected in advance from the moving image stored in advance in the moving image 12, and the search information storage device 54
To save. Here, there is no need to extract or save the browsing thumbnail such as a JPEG file corresponding to the classification point time code described in the audio classification information. Regarding the audio classification method, for example, a method described in a patent application “Audio information classification device JP-A-2000-66691” by the present applicant can be used.

Regarding the audio classification information, as shown in FIG.
An audio classification information sequence (A1: AC1, A2: AC2, A3: AC3,...) 58 classified into “cheering noise” and “silence” is used. Here, An represents the n-th audio classification point time code, and ACn represents the audio classification type at the n-th audio classification point.

When an audio classification start command is input by specifying a video file name from the search / playback command input section 18, the audio classification start time code and the number of audio classification point browsing thumbnails retrieved from the control section 56 are retrieved. The search unit 55 outputs the audio classification point browsing time code sequence (An, An + 1) corresponding to the thumbnail to be browsed from the audio classification information stored in the search information storage device 54 based on the input information. , An + 3, ・ ・ ・)
And the audio classification type sequence (ACn + 1, ACn + 2, ACn + 3,...) Corresponding to each time code sequence. Control unit 56
Then, the obtained audio classification point browsing time code sequence (An, An + 1, An + 3,...) Is passed to the thumbnail extracting unit 14, and the corresponding thumbnail sequence (Th (An), Th (An + 1 ), Th (An +
3),...), And in the control unit 56 together with the audio classification type code string (ACn + 1, ACn + 2, ACn + 3,...) As the audio classification point search result in the search result browsing display unit 57. Browse display. The method of searching and browsing more detailed contents in the time direction from the audio classification point position or the arbitrary position and the method of reproducing the video section are the same as those in the first embodiment.

The mapping of the requested time code, the frame time code in the moving image, and the time code of the extracted thumbnail in the thumbnail extracting unit 14 is the same as in the first embodiment, as shown in FIG. The request time code sequence (R1, R2, R3, R4,...) 27 includes the time code sequence (A1, A2, A3,
…)) By substituting 58.

FIG. 17 shows an example of browsing display for browsing audio categories. Audio classification point time code string (A
n, An + 1, An + 2, An + 3, An + 4) 59 are displayed at the same time as the audio classification point thumbnail sequence (Th (An), Th (An + 1), Th
(An + 2), Th (An + 3), Th (An + 4)) 60 and audio type strings (ACn, ACn + 1, ACn + 2, ACn + 3, ACn + 4) 61 Is displayed. Press the left skip button 62,
Time code string one span before the currently displayed time code string (An-5, An-4, An-3, An-2, An-
1), audio classification point thumbnail row (Th (An-5), Th (An
-4), Th (An-3), Th (An-2), Th (An-1)) and audio classification type strings (ACn-5, ACn-4, ACn-3, ACn-2, ACn-
1) is displayed. Similarly, when the right skip button 63 is pressed, the time code sequence (An + 5, An + 6, An + 7, An + 8, An
+9), audio classification point thumbnail row (Th (An + 5), Th (A
n + 6), Th (An + 7), Th (An + 8), Th (An + 9)), and audio classification type strings (ACn + 5, ACn + 6, ACn + 7, ACn + 8, ACn +
9) is displayed.

In the frame unit search browsing browsing described in the first embodiment, the browsing display in the audio class browsing as shown in "Relationship between audio classification type point time code and frame unit time code" in FIG. And time intervals in the audio classification point browsing display (for example, the switching point from the moderator's voice to the musician's performance)
An application is also possible in which is specified and the display is made finer by frame-based browsing display from that location.

FIG. 19 shows an example of a search / browsing / playback screen using the audio category browsing browser and the frame unit browser alone or in combination.

The operation of the audio classification browsing browser 64 is the same as that described with reference to FIG. 17. The switching point of the scene by the audio classification is displayed by the thumbnail and the audio classification type, and the reproduction start / end position can be designated. I have. With the thumbnail at the time position selected on the audio classification browsing browser 64, the playback start time point button
By pressing 42, the time code is displayed in the playback start point display area 41 and the playback start time point button
The corresponding thumbnail is displayed on 42. Similarly, by pressing the reproduction end time point button 44, the time code is displayed in the reproduction end point display area 43, and at the same time, the corresponding thumbnail is displayed on the reproduction end time point button 44. Next, by pressing the playback start button 45, the section playback from the designated start point to the end point by specifying the audio classification type switching point is started on the playback display screen 46. When only the reproduction start position is specified and the reproduction start button 45 is pressed, the section from the specified start point to the end of the content is reproduced. When the reproduction start button 45 is pressed without specifying the reproduction start / end positions, the entire section from the beginning to the end of the moving image content is reproduced.

The equal-interval browser 39 for performing frame-unit display and the like operates in the same manner as in the first embodiment, and a description thereof will be omitted.

In the fourth embodiment of the present invention, thumbnail search and browsing are performed by using the cut point search information described in the first embodiment and the audio classification information described in the third embodiment in combination. In addition, advanced section playback by designating the time as a playback start / end point is performed on a local host.

In FIG. 20, search information storage device 65,
The operations other than the search unit 66, the control unit 67, and the search result browsing display unit 68 are the same as those in FIG. The search information storage device 65 includes cut point search information and audio classification information. The search unit 66, the control unit 67, and the search result browsing display unit 68 include all functions of each unit described in the first embodiment and the third embodiment.

FIG. 21 shows a composite browsing display example of the cut point browser and the audio classification browsing browser. FIG.
In 1, the cut point time code string 30 and the cut point thumbnail string 31 operate in the same manner as in FIG. Cut point time code strings (Cn, Cn + 1, Cn + 2, Cn + 3, Cn + 4) 30 and cut point thumbnail strings (Th (Cn), Th (Cn + 1), Th (Cn + 2),
Th (Cn + 3), Th (Cn + 4)) 31 and audio classification type strings (AC (Cn), AC (Cn) corresponding to the cut point timecode strings
n + 1), AC (Cn + 2), AC (Cn + 3), AC (Cn + 4)) 65 are displayed in synchronization. When the left skip button 32 is pressed, the time code string (Cn-5, Cn-4, Cn-3, Cn-2, Cn-1) one span before the currently displayed time code string is cut. Point thumbnail sequence (Th (Cn-5), Th (Cn-4), Th (Cn-3), Th (Cn-
2), Th (Cn-1)), and audio classification type sequence (AC (Cn-
5), AC (Cn-4), AC (Cn-3), AC (Cn-2), AC (Cn-1)) are displayed. Similarly, when the right skip button 33 is pressed, the time code string one span after the currently displayed time code string (Cn + 5, Cn + 6, Cn + 7, Cn + 8, Cn + 9) , Cut point thumbnail sequence (Th (Cn + 5), Th (Cn + 6), Th (Cn + 7), Th (Cn + 7)
n + 8), Th (Cn + 9)), and audio classification type sequence (AC (C
n + 5), AC (Cn + 6), AC (Cn + 7), AC (Cn + 8), AC (Cn + 9)) are displayed. Here, the method of acquiring the audio classification type AC (Cn) at the cut point time Cn is represented by the following equation (5) using m (m-th audio classification point) satisfying the following equation (4). ACm (m-th audio classification type). Am ≦ Cn <Am + 1 (4) AC (Cn) = ACm (5)

FIG. 22 shows an example of a search / browsing / playback screen using a combination of the cut point browser, the audio classification browsing browser, and the frame unit browser. The cut point search / audio classification combined browsing browser 66 operates in the same manner as described with reference to FIG. 21. The switching point of the scene by the cut point is displayed by the thumbnail and the audio classification type, and the reproduction start / end position can be specified. I have. By pressing the playback start time point button 42 with the thumbnail at the certain time position selected on the cut point / audio classification combined browsing browser 66, the time code is displayed in the playback start point display area 41 and the playback is started at the same time. The corresponding thumbnail is displayed on the time point button 42. Similarly, by pressing the reproduction end time point button 44, the time code is displayed in the reproduction end point display area 43, and at the same time, the corresponding thumbnail is displayed on the reproduction end time point button 44.
Next, by pressing the playback start button 45, the section playback from the designated start point to the end point by specifying the cut point is started on the playback display screen 46. When only the reproduction start position is specified and the reproduction start button 45 is pressed, the section from the specified start point to the end of the content is reproduced. When the reproduction start button 45 is pressed without specifying the reproduction start / end positions, the entire section from the beginning to the end of the moving image content is reproduced.

The browser 39 for displaying the frames at equal intervals performs the same operation as that of the first embodiment, so that the description is omitted.

The fifth embodiment of the present invention uses a telop search information described in the second embodiment and the audio classification information described in the third embodiment in combination to search and browse thumbnails. By specifying the time as a reproduction start / end point, advanced section reproduction is performed on the local host.

In FIG. 23, the search information storage device 67,
The operations other than the search unit 68, the control unit 69, and the search result browsing display unit 70 are the same as those in FIG. The search information storage device 67 includes telop search information and audio classification information. The search unit 68, the control unit 69, and the search result browsing display unit 70 include all functions of each unit described in the second embodiment and the third embodiment.

FIG. 24 shows a composite browsing display example of the telop search / browsing browser and the audio classification browsing browser. In FIG. 24, the telop start point time code column 49 and the telop start point thumbnail column 50 operate in the same manner as in FIG. Subtitle start point time code string (Tn, Tn + 1,
Tn + 2, Tn + 3, Tn + 4) 49 and the telop starting point thumbnail sequence (Th (Tn), Th (Tn + 1), Th (Tn + 2), Th (Tn + 3), Th (Tn +
4)) Along with 50, the audio classification type sequence (AC (Tn), AC (Tn + 1), AC (T
n + 2), AC (Tn + 3), AC (Tn + 4)) 71 are displayed in synchronization.
When the left skip button 51 is pressed, the time code string (Tn
-5, Tn-4, Tn-3, Tn-2, Tn-1), telop start point thumbnail sequence (Th (Tn-5), Th (Tn-4), Th (Tn-3), Th (Tn -2), Th
(Tn-1)) and audio classification type column (AC (Tn-5), AC
(Tn-4), AC (Tn-3), AC (Tn-2), AC (Tn-1)) are displayed. Similarly, when the right skip button 52 is pressed, the time code string (Tn + 5, Tn + 6, Tn + 7, Tn + 8, Tn + 9) one span after the currently displayed time code string in time. , Telop start point thumbnail sequence (Th (Tn + 5), Th (Tn + 6), Th (Tn + 7), Th
(Tn + 8), Th (Tn + 9)) and audio classification type column (AC
(Tn + 5), AC (Tn + 6), AC (Tn + 7), AC (Tn + 8), AC (Tn + 9)) are displayed. Here, the method of acquiring the audio classification type AC (Tn) at the telop start point time Tn uses m (m-th audio classification point) that satisfies the following equation (6), as in the fourth embodiment. And ACm (m
Th audio classification type). Am ≦ Tn <Am + 1 (6) AC (Tn) = ACm (7)

FIG. 25 shows an example of a search / browsing / playback screen using a telop search / browsing browser, an audio classification browsing browser, and a frame unit browser in combination.

The telop search / audio classification combined browsing browser 72 operates in the same manner as in FIG. 24, and displays scene switching points such as news at the telop start point in the form of thumbnails and audio classifications, and starts playback / reproduction.
The end position can be specified. By pressing the playback start time point button 42 with the thumbnail at a certain time position selected on the telop search / audio classification combined browsing browser 72, the time code is displayed in the playback start point display area 41 and the playback is started at the same time. The corresponding thumbnail is displayed on the time point button 42. Similarly, by pressing the reproduction end time point button 44, the time code is displayed in the reproduction end point display area 43, and at the same time, the corresponding thumbnail is displayed on the reproduction end time point button 44. Next, by pressing the playback start button 45, the section playback from the designated start point to the end point by specifying the telop start point is started on the playback display screen 46. When the reproduction start button 45 is pressed without specifying the reproduction start / end positions, the entire section from the beginning to the end of the moving image content is reproduced.

The equal-interval browser 39 for performing frame-unit display and the like operates in the same manner as in the first embodiment, and a description thereof will be omitted.

A sixth embodiment of the video browsing and searching device is as follows.
The cut point search information described in the first embodiment and the second
By using the telop search information described in the third embodiment and the audio classification information described in the third embodiment in combination, thumbnail search and browsing are performed, and advanced by specifying the time as a reproduction start / end point. The section reproduction is performed on the local host.

In FIG. 26, search information storage device 73,
The operations other than the search unit 74, the control unit 75, and the search result browsing display unit 76 are the same as those in FIG. The search information storage device 73 includes cut point search information, telop search information, and audio classification information. Search unit 74, control unit 75,
The search result browsing display unit 76 includes all the functions of each unit described in the first embodiment, the second embodiment, and the third embodiment.

FIG. 27 shows an example of a search / browsing / playback screen that uses a combination of a cut point search / browser browser, a telop search / browser browser, an audio classification browser, and a frame unit browser.

The cut point search / telop search / audio classification combined browsing browser 77 has a search browsing method selection menu.
In 79, by selecting a search and browsing method (cut point search, telop search, audio classification browsing), the operation of the internal browsing display unit 78, the operation of the left skip button 80, and the operation of the right skip button 81 change. .

The operation of the internal browsing display unit in each search and browsing method is the same as in FIG. 21 for cut point search, the same as in FIG. 24 for telop search, and FIG. 17 for audio category browsing. Is the same as

When a thumbnail at a certain time position is selected on the cut point search / telop search / audio classification combined browsing browser 77 and the playback start time point button 42 is pressed, the time code is displayed in the playback start point display area 41. Is displayed, and at the same time, the corresponding thumbnail is displayed on the reproduction start time point button 42. The frame skip buttons 82 and 83 are used for fine adjustment of the reproduction start point in frame units.
By pressing the reproduction end time point button 44 in the same manner as the reproduction start point designation, the time code is displayed in the reproduction end point display area 43, and at the same time, the corresponding thumbnail is displayed on the reproduction end time point button 44. The frame skip buttons 84 and 85 are used for fine adjustment of the reproduction end point in frame units. Next, by pressing the playback start button 45, section playback from the designated start point to end point by specifying the cut point, the telop start point, and the audio classification type switching point is started on the playback display screen 46. When only the reproduction start position is specified and the reproduction start button 45 is pressed, the section from the specified start point to the end of the content is reproduced. Start playback button 45 without specifying playback start / end position
When is pressed, the entire section from the beginning to the end of the moving image content is reproduced. The operation of the edit section addition button 86 will be described in the following first embodiment of the present invention.

The equally-spaced browser 39 for displaying frames or the like operates in the same manner as in the first embodiment, and a description thereof will be omitted.

A seventh embodiment of the video search / browsing device is as follows:
Advanced section playback by specifying thumbnail search and browsing using cut point search, telop search, audio classification, frame unit search described in the first to sixth embodiments, and specifying the time as a playback start / end point Is applied to a configuration of a server and a client on a network.

In FIG. 28, a moving image storage device 88 has
Moving image contents are stored in advance. Cut point search information, telop search information,
Alternatively, various search information such as audio classification information is detected in advance and stored in the search information storage device 87. Here, it corresponds to the time code included in various search information.
Viewing thumbnails such as JPEG files do not need to be extracted or saved. The description of the various search classification information is the same as in the first to sixth embodiments.

Retrieval and search on client side WWW browser
Various search start commands are specified from the playback command input unit 98 by specifying a video file name.
When input to the control unit 92 via the control unit 93, the control unit 92 outputs various search classification start time codes and the number of thumbnails for browsing various search classifications to the search unit 89, and the search unit 89 based on the input information. Then, from the various types of search information stored in the search information storage device 87, a browse time code string corresponding to the thumbnail to be browsed is obtained. In the control unit 92,
The obtained browsing time code sequence is passed to the thumbnail extracting unit 90, and the corresponding thumbnail sequence is extracted, and the control unit 92 and the WW
The search result browsing display unit 97 on the WWW browser 96 via the W server 93 and the network 95 is browsed and displayed as various search classification results.

When searching and browsing more detailed contents in the time direction from various search classification point positions or arbitrary positions, the time code of the detailed search classification browsing start position and the display in the time direction such as frame units and second units are displayed. Accuracy and the number of thumbnails retrieved for browsing
The search / playback command input unit 98 is input to the control unit 92 via the network 95 and the WWWServer 93. Control unit 92
Then, after generating a browsing time code sequence at regular time intervals from the obtained information, the browsing time code sequence is passed to the thumbnail extracting unit 90, the corresponding thumbnail sequence is extracted, and the control unit 92, the WWW server 93, and the network It is passed to the search result browsing display unit 97 via 95 and browsed and displayed as search results at regular time intervals.

Regarding the section reproduction of the video, in the browse display of the search result browsing display unit 97 on the WWW browser,
The time code to start and end the reproduction of the video is acquired, and the reproduction start command including the information of the reproduction start / end time code and the video file name is transmitted from the search / reproduction command input unit 98 on the WWW browser 96 to the network. Input to video server 94 via 95. Then, the video server 94 operates as a moving image content storage device.
From 88, the data of the section specified by the frame precision of the specified video content is extracted and transmitted to the video reproduction display unit 99 on the WWW browser 96 via the network 95. At this time, if the transmission band of the extracted moving image data exceeds the transmission band of the output network, the bit rate conversion gateway 91 performs bit rate conversion so as to conform to the network transmission band, and then outputs the data to the network. Here, with regard to the moving image bit rate conversion method, for example, a patent application “Rate conversion method and apparatus of image coded data by the present applicant” is disclosed in Japanese Patent Application Laid-Open No. 8-251587.
The method described in Japanese Unexamined Patent Publication (Kokai) No. HEI 9-301 can be used. The video reproduction display unit 99 performs frame reproduction from the reproduction start point to the end point with frame accuracy by sequentially reproducing the received data.

An embodiment in the case where the system is actually implemented as a system will be described below. The control unit 92 and the search unit 89 are provided with an ASP (Active) that can be dynamically applied to an interactive operation with the WWW browser 96 via the WWW server 93.
A thumbnail extraction unit 90 implemented by a server module, a script language such as JavaScript) and implemented by a library module such as ATL (Active Template Library) that can be called from the ASP is controlled by the ASP of the control unit 92. The search information storage device 87 accumulates search information in a database in order to support high-speed search, and retrieves an ODBC (Open Da
A search request is issued in SQL format using the tabase connectivity) function. The video storage device 88 is provided with a RAID (Redundant Arrays of Inexpensive Diode) in consideration of the performance improvement at the time of a large number of simultaneous random accesses from the video server 94 and the thumbnail extraction unit 90.
sks) Use HDD with configuration.

Next, a first embodiment of the present invention will be described. In this embodiment, the editing start point / end is efficiently and accurately performed using the thumbnail search and browsing results using the cut point search, telop search, audio classification, and frame unit search described in the first to sixth embodiments. A point is designated, and a moving image is edited.

In FIG. 29, the thumbnail extracting unit 14, the search result browsing and displaying unit 76, and the video storage device 12 operate in the same manner as in FIG. Cut point search, telop search,
From the search results of the audio classification and the frame unit search, an edit start point / end point with frame accuracy is found, and its time code and file name are input to the control unit 102 via the edit section addition command input unit 104. The control unit 102 extracts the thumbnail of the edit start point / end point and the content start time code together with the added edit section time code by the thumbnail extracting unit 14 and additionally displays the thumbnail on the edit list display unit 105. After the editing section adding operation is repeated, splice editing (partial editing of the seam portion) and splice reproduction (partial portion of the seam portion) are performed in order to check the visual consistency at the time of image reproduction at each editing point (each seam portion). Playback).

Here, the splice editing / playback processing will be described. A splice playback (partial playback of a joint portion) command is input from the splice playback command input unit 106 via the control unit 102 together with the file name of the content corresponding to the editing point corresponding to the splice editing unit 101 and the editing start / end points. Then, the splice editing unit 101 reads data for a fixed time (for example, 3 seconds, which can be changed by setting) before and after the editing point from the corresponding moving image content and performs partial editing. Next, the partially edited moving image data is reproduced by the splice reproducing unit 103 and displayed on the splice view 108.

Unnecessary edit sections are deleted by splice reproduction confirmation, and fine adjustment is performed on edit sections that need to be changed. After confirming the consistency of all the editing sections and all the seams, an editing start command is issued from the editing start command input unit 107, thereby performing the editing processing of the image content.

Here, the MPE in the frame unit editing unit 100
The editing processing of the G content in units of frames will be described.
As shown in FIG. 30, the baseband editing unit 109 uses a method of processing the compressed data as it is, except that the first half of the edit point 110 and the second half of the edit point 111 are partially decoded into baseband and edited / recompressed. Therefore, it is possible to prevent the deterioration of the image quality, save the temporary storage disk by not performing the full decompression, and realize the high-speed processing.

Hereinafter, the frame unit editing method on the MPEG compressed data will be described in detail with reference to FIGS. In addition, the arrangement of the frame display in the figure is all DisplayOrd
er. Regarding the difference between DisplayOrder and StreamOrder, as shown in FIG. 31, the sequence of frames on the MPEG-compressed stream is StreamOrder,
The sequence of frames after decoding and playback is Displa
yOrder.

FIG. 32 shows a case where the first half of the editing point is cut after the P frame 112 and the second half of the editing point is cut before the P frame 113. For the first half of the edit point, the bit allocation of all frames (corresponding to 110) before the P frame 112 in the same GOP including the P frame 112 is changed, and for the second half of the edit point, the same P in GOP
The bit allocation of all frames after the frame and the first two B frames belonging to the next GOP (corresponding to 111) is changed. In this case, there is no change in the picture type, but VBV (Video B
buffering verifier) The amount of bits allocated to each frame, that is, the quantization scale (Quan
Adjust tizer Scale). Hereinafter, the processing in the first half of the editing point and the processing in the second half of the editing point will be described in detail by dividing into cases.

First, the processing in the first half of the editing point will be described in detail. FIG. 33 shows a case where the first half of the edit point is cut after the I frame 114. Since the GOP is closed at the edit point, the picture type of the I frame 114 is changed to the P frame 115, and the bit allocation of all frames belonging to the immediately preceding GOP is changed without changing the picture type. When changing an I frame to a P frame, the I frame is decoded once and re-encoded as a P frame.
Since the bit amount is a fraction of the number of the I frame, the bit amount is redistributed for the remaining frames of the GOP so as to cover the bit amount reduced because the I frame has become a P frame.

FIG. 34 (a) shows that the first half of the editing point is a P frame 116.
Shows the case where it is cut after the. The picture type of the P frame 116 is not changed, and the same GO including the P frame 116
The bit amount redistribution of all frames (corresponding to 110) before the P frame 116 in P is performed. At this time, since the B frames B28 and B29, which have much less bits than the P frames, are cut, the bit amount is reduced again so that the I frame of the next GOP is connected and buffer underflow does not occur. Distribute.

However, in the example shown in FIG. 34 (b) where the edit point is cut after the same P frame, the edit point is in the first half of the GOP,
Because the bit allocation operation within the same GOP is difficult, the previous GOP
And redistribute bits. In this case, by changing the picture type from I16 to P'16, a 15-frame
The GOP structure is changed to a 21-frame GOP structure.

FIG. 35A shows that the first half of the editing point is a B frame 117.
Shows the case where it is cut after the. B frame 117 is P
The picture type is changed to frame 118 and B frame 117
All frames before the B frame 117 in the same GOP (1
(Equivalent to 10). When changing the picture type of a B frame to a P frame, basically the B frame is decoded once and re-encoded as a P frame, but the motion compensation is performed only in the forward direction in the B frame.
For B (Macro Block), motion compensation is performed in the same direction even if it becomes a P frame, so re-encoding is performed using the motion vector as it is, and a new motion search process is performed for all MBs. The processing speed can be improved as compared with.

FIG. 3 shows an example of cutting after the same B frame.
In the case of 5 (b), the editing point is in the first half of the GOP,
Since it is difficult to perform the bit allocation operation within the GOP, the bits are redistributed including the immediately preceding GOP. In this case, by changing the picture type from I18 to P'18, the GOP structure having 15 frames is changed to the GOP structure having 20 frames. B19 to B '
At the time of the change to 19, not only the change of the bit allocation, but also the reference frame used for the backward motion compensation is changed from P21 to P′20, so that the MB using the bidirectional motion compensation or the backward motion compensation is used. For, it is necessary to re-search for a motion vector.

The processing in the first half of the editing point has been described so far. The processing in the second half of the editing point will be described in detail below. FIG.
Indicates a case where the frame is cut short of the I frame 119. All frames after the I frame 119 in the GOP including the I frame 119 and the first B frames B31 and B32 (111
Is changed).

FIG. 37 (a) shows that the latter half of the editing point is a P frame 120.
Shows the case where it is cut short before. New from edit point
Since the GOP starts, the P frame 120 is changed in picture type to the I frame 121, and the P frame 120 included in the same GOP is changed.
B16, B17 (1
(Equivalent to 11). When changing a P frame to an I frame, the P frame is decoded once and re-encoded as an I frame.However, since the I frame usually has several times the bit amount of the P frame, for the remaining frames of the GOP, The bit amount is redistributed so that the bit amount increased due to the P frame becoming the I frame can be absorbed. However, in the example shown in FIG. 37B in which the frame is cut before the same P frame, the editing point is in the latter half of the GOP, and it is difficult to perform the bit allocation operation in the same GOP. GOP
And redistributes bits including the first two B frames belonging to. In this case, by changing the picture type from P15 to I'15, the GOP structure having 15 frames is changed to the GOP structure having 16 frames.

FIG. 38A shows that the latter half of the editing point is a B frame 122.
Shows the case where it is cut short before. New from edit point
Since the GOP starts, an I frame is required near the beginning, and P6
Is changed to I'6. Although a method of changing the picture type from the B frame 122 to the I frame is conceivable, the conversion from the B frame to the I frame is not suitable because the I frame has a large bit amount and is used as a reference image requiring image quality. As for the other frames, all the frames after the B frame 122 in the same GOP including the B frame 122 and the first B16 and B17 (1
(Equivalent to 11). B4 → B'4, B5 →
In the B'5 conversion, not only the bit amount is adjusted, but also the I3 frame referred to in the forward motion compensation disappears.Therefore, for an MB using bidirectional or forward motion compensation, the motion vector Re-searching is required.

FIG. 3 shows an example in which the frame is cut just before the same B frame.
In the case of 8 (b), the edit point is in the latter half of the GOP, and
Since the bit allocation operation within the GOP is difficult, the bits are redistributed including the next GOP and the two first B frames of the next GOP (corresponding to 111). In this case, by changing the picture type from I18 to P'18, the GOP structure having 15 frames is changed to the GOP structure having 17 frames. B14 to B'14
At the time of the change to, not only the change of the bit allocation but also the reference frame P12 used for the forward motion compensation is lost, so that the MB using the bidirectional motion compensation or the forward motion compensation is used.
For, it is necessary to re-search for a motion vector.

Next, an example of a screen used when performing frame unit editing using the thumbnail search and browsing result of a moving image will be described.
First, on the search / browsing / playback screen using the cut point search / browsing browser, telop search / browsing browser, audio classification browsing browser, and frame unit browser in FIG. 27, the playback start time / playback end specified and section playback confirmed. When the time is set as the edit start / end point and the edit section addition button 86 is pressed, the edit start / end point is set to FIG.
A section (for example, section 1) selected by the section selection button 123 on the editing control screen is added as an editing section.

In the representative thumbnail display area 124, a time ttop1 (ttop1 = 0) representing the inserted moving image content is displayed.
In this case, the thumbnail Th (ttop1) at the beginning of the content is displayed, and the file name of the content is displayed in the file name display area 125. The edit start point thumbnail display area 126, the edit start point timecode display area 127, the edit end point thumbnail display area 128, and the edit end point timecode display area 129 respectively have an edit start point thumbnail Th (tin1) and an edit start point. Time code tin1, edit end point thumbnail Th (tout1), edit end point time code t
out1 is displayed. By the same operation, the editing section is repeatedly added to the section n.

If the edit section list cannot be displayed at once, the left skip button 130 and the right skip button 131
By pressing, the next editing section can be displayed. To delete an editing section, the section deletion button 134 is pressed while the section to be deleted is selected. Further, when it is desired to fine-tune the editing start point / end point of the section, the section is selected and the section change button 135 is pressed, so that the corresponding editing section is displayed at the playback start time point / reproduction end time point in FIG. The display and the time code are displayed, and the start point / end point can be finely adjusted again.

In FIG. 39, after all the editing sections have been inserted, if it is desired to confirm the visual consistency at the editing point before performing the editing operation, the splice view display button 132 of the editing point to be confirmed is pressed. This makes it possible to partially edit the seam portion from the data before and after the edit point in a short time and display the moving image on the splice view 133. After confirming the consistency of the seam of each edit point by the splice view, if an edit section needs to be deleted or changed, it is corrected as needed. Finally, when all the editing sections are determined, the editing start button 137 is pressed to start the editing work.
The progress of the editing process is displayed by an editing process indicator 138.

If the EDL addition button 136 is checked at the start of editing, as shown in FIG. 40, a list of editing sections is displayed together with the moving image editing in SMPTE (Society of Motion Pic).
EDL (Edit Dele
cision list) 139. In addition, by reading this EDL file from the outside, you can reproduce the editing operation that was done in the past,
It becomes possible to fetch and edit an edit list created using an external VTR. This method is effective as a method that allows the user to instruct the work only by the edit list without exchanging large-capacity moving image contents when performing the editing work between remote places.

Next, a second embodiment of the present invention will be described. In this embodiment, the frame unit editing using the thumbnail search and browsing result of a moving image described in the first embodiment is applied to a server-client configuration on a network.

In FIG. 41, the thumbnail extracting unit 90, the search result browsing and displaying unit 97, and the video storage device 88 operate in the same manner as in FIG. Cut point search, telop search,
From the search results of the audio classification and the frame unit search, the editing start point / end point of frame accuracy is found, and its time code and file name are input from the editing section addition command input unit 146 to the network 144 and the WWW server 143.
Is input to the control unit 142 via the.

The control unit 142 extracts the thumbnails of the edit start point / end point and the content start time code together with the added edit section time code by the thumbnail extracting unit 90, and sends the edited list display unit via the WWW server 143 and the network 144. 147 is additionally displayed. After the editing section adding operation is repeated, splice editing (partial editing of the seam portion) is performed via a network in order to check the visual consistency at the time of image reproduction at each edit point (each seam portion).
Further, it is possible to confirm by splice regeneration (partial regeneration of a seam portion).

Here, the splice editing / playback processing via the network will be described. Splice playback (joint portion) from the splice playback command input unit 148 via the network 144, the WWW server 143 and the control unit 142 together with the file name of the content corresponding to the edit point corresponding to the splice edit unit 141 and the edit start / end points When a (playback) command is input, the splice editing unit 141 reads data for a fixed time (for example, 3 seconds, which can be changed by setting) from the corresponding moving image content before and after the editing point, and performs partial editing. Next, the partially edited video data is transferred to the WWW server 1.
After being transmitted to the splice reproducing unit 145 via 43, it is reproduced and displayed on the splice view 150.

Unnecessary edit sections are deleted by splice reproduction confirmation, and fine adjustment is performed on edit sections that need to be changed. After finally checking the consistency of all edit sections and all seams, the edit start command input unit 149 issues an edit start instruction to the control unit 142 via the network 144 and the WWW server 143, so that the frame unit is Editor 140
Perform the editing process of the image content.

As a result, it is possible to efficiently perform search / browsing / editing of a moving image and confirmation before editing by a splice view even from a remote place via a network. Also, the edited moving image can be confirmed and reproduced by the search / browse / reproduce system on the network described in the seventh embodiment.

Although the first and second embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to these first and second embodiments. Even if there is a design change or the like within a range not departing from the gist of the present invention, it is included in the present invention.

For example, in the above-described first and second embodiments, for example, the thumbnail extracting program of the thumbnail extracting units 14 and 90, the frame editing program of the frame editing units 100 and 140, and the splice editing of the splice editing units 101 and 141. The program is recorded on the computer-readable recording medium shown in FIG. 42, and the thumbnail extraction program, the frame unit editing program, and the splice editing program recorded on the recording medium 300 are read into the computer 200 shown in FIG. May be executed. Here, the thumbnail extracting program, the frame unit editing program, and the splice editing program are programs for realizing the functions of the thumbnail extracting unit, the frame unit editing unit, and the splice editing unit shown in FIGS. 29 and 41, respectively.

The computer 200 shown in FIG. 42 includes a CPU 201 for executing the transmission program, an input device 202 such as a keyboard and a mouse, and a ROM (Read On) for storing various data.
ly Memory) 203 and a RAM (Ran
dom Access Memory) 204, a reading device 205 that reads a transmission program from the storage medium 300, an output device 206 such as a display or a printer, and a bus BUS that connects each unit of the device.

The CPU 201 reads the thumbnail extracting program recorded on the recording medium 300 via the reading device 205, and executes the thumbnail extracting program, the frame unit editing program, and the splice editing program, respectively. The thumbnail extraction process, frame unit editing process, and splice editing process are performed. The recording medium 300 includes not only a portable recording medium such as an optical disk, a floppy (registered trademark) disk, and a hard disk, but also a transmission medium such as a network for temporarily recording and holding data.

[0117]

As is clear from the above description, according to the present invention, the following effects can be obtained. (1) Efficient and accurate designation of the edit start point / end point using cut point search, telop search, audio classification, thumbnail search and browsing results using frame unit search, and editing of the moving image frame unit Becomes possible. (2) When performing frame-based editing of MPEG content, except for decoding and re-compressing only part of the data before and after the edit point to the baseband, the processing is performed as is with the compressed data. Prevention, saving of temporary storage disk by not fully decompressing, and high-speed processing can be realized. (3) By confirming by splice editing (partial editing of the seam part) and splice reproduction (partial reproduction of the seam part), the moving image at each editing point (seam part) is edited before the frame unit editing of the moving image is performed. It is possible to check the consistency of the appearance at the time of reproduction. (4) When editing a moving image, the list of edit sections is changed to SMPTE (Socie
ty of Motion Picture & Television Engineers (EDL) that can be output to a file as an Edit Decision List (EDL) or read externally to perform new editing work based on past editing operations, external VTRs, etc. Can be used to import and edit the edit list created by using, and when editing work between remote locations, remote instruction of work only with the edit list without exchanging large volume of moving image content It becomes possible to do. (5) The processing of (1) to (4) can be performed in a state where moving image data is shared not only on the local host but also on the network. (6) It is possible to provide a computer-readable recording medium that records a program capable of preventing deterioration of image quality due to not decompressing MPEG contents, saving temporary storage disks, and achieving high-speed processing. (7) It is possible to provide a computer-readable recording medium that records a program capable of performing frame unit editing and spline editing.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of a first embodiment of a video search and browse device.

FIG. 2 is an explanatory diagram of cut point search information description.

FIG. 3 is a block diagram illustrating functions of a thumbnail extracting unit.

FIG. 4 is an explanatory diagram of an MPEG frame structure.

FIG. 5 is an explanatory diagram of a thumbnail extracting operation.

FIG. 6 is a diagram illustrating a display example of browsing for browsing cut point search.

FIG. 7 is a diagram showing an example of a frame unit search browsing browsing display.

FIG. 8 is an explanatory diagram showing a relationship between a cut point time code and a frame unit time code.

FIG. 9 is a diagram illustrating a display example of a search / browse / reproduction screen according to the first embodiment.

FIG. 10 is a block diagram showing a schematic configuration of a second embodiment of the video search / browsing device.

FIG. 11 is an explanatory diagram of a telop search information description.

FIG. 12 is a diagram illustrating an example of a telop search / browsing browsing display.

FIG. 13 is an explanatory diagram showing a relationship between a telop start point time code and a frame unit time code.

FIG. 14 is a diagram illustrating a display example of a search / browse / reproduction screen according to the second embodiment.

FIG. 15 is a block diagram showing a schematic configuration of a third embodiment of the video search / browsing device.

FIG. 16 is an explanatory diagram of audio classification information description.

FIG. 17 is a diagram showing an example of audio category browsing browsing display.

FIG. 18 is an explanatory diagram showing a relationship between an audio classification point time code and a frame unit time code.

FIG. 19 is a diagram illustrating a display example of a search / browse / reproduction screen according to the third embodiment.

FIG. 20 is a block diagram showing a schematic configuration of a fourth embodiment of the video search / browsing device.

FIG. 21 is a diagram showing a composite display example of a cut point browser and an audio classification browser.

FIG. 22 is a diagram illustrating a display example of a search / browse / reproduction screen according to the fourth embodiment.

FIG. 23 is a block diagram illustrating a schematic configuration of a fifth embodiment of the video search / browsing device.

FIG. 24 is a diagram showing a composite display example of a telop search / browsing browser and an audio classification browser.

FIG. 25 is a diagram illustrating a display example of a search / browse / reproduction screen according to the fifth embodiment.

FIG. 26 is a block diagram illustrating a schematic configuration of a sixth embodiment of the video search / browsing apparatus.

FIG. 27 is a diagram illustrating a display example of a search / browse / reproduction screen according to the sixth embodiment.

FIG. 28 is a block diagram illustrating a schematic configuration of a seventh embodiment of the video search / browsing device.

FIG. 29 is a block diagram showing a schematic configuration of the first embodiment of the present invention.

FIG. 30 is an explanatory diagram showing frame unit editing on compressed data of MPEG content.

FIG. 31 is a diagram illustrating a relationship between a picture arrangement of a stream and a picture arrangement of a display in MPEG.

FIG. 32 is an explanatory diagram showing frame unit editing on MPEG compression.

FIG. 33 is an explanatory diagram showing frame unit editing (the first half of an editing point) on MPEG compression.

FIG. 34 is an explanatory diagram showing another frame-based editing (the first half of an editing point) on MPEG compression.

FIG. 35 is an explanatory diagram showing another frame unit editing (the first half of an editing point) on MPEG compression.

FIG. 36 is an explanatory diagram showing another frame unit editing (the latter half of the editing point) on MPEG compression.

FIG. 37 is an explanatory diagram showing another frame unit editing (the latter half of the editing point) on MPEG compression.

FIG. 38 is an explanatory diagram showing another frame-based editing (the latter half of an editing point) on MPEG compression.

FIG. 39 is a diagram showing an example of an edit control screen of the first embodiment.

FIG. 40 is an explanatory diagram of an editing process using an EDL file.

FIG. 41 is a block diagram showing a schematic configuration of a second embodiment of the present invention.

FIG. 42 is a block diagram illustrating a schematic configuration of a computer from which a program of the present invention is read.

[Explanation of symbols]

11, 47, 54, 65, 67, 73, 87 ... search information storage device, 12 ... video storage device, 13 ... search unit, 1
4, 90 ... Thumbnail extraction unit, 15, 102, 142
..Control unit, 16 ... Video playback unit, 17, 76, 97 ... Search result browsing display unit, 18 ... Search / playback command input unit, 19 ... Video display unit, 22 ... Moving image partial decoding unit, 23 ... cache, 24 ... thumbnail generation unit, 3
8 ... cut point browser, 39 ... equally spaced browser, 53
··· Telop search and browsing browser, 64 ··· Audio classification browsing browser, 66 ··· Cut point search and audio classification combined browsing browser, 72 ··· Telop search and audio classification combined browsing browser, 100 and 140 ··· Frame unit editing unit, 101, 141 ... splice editing unit, 10
3. Splice playback unit, 104, 146 ... edit section addition command input unit, 106, 148 ... splice playback command input unit, 107, 149 ... edit start command input unit, 108, 150 ...・ Splice view, 143 ・
..WWW server, 144 ... network.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI theme coat ゛ (Reference) H04N 5/262 H04N 5/91 N 5/76 5/781 510F 5/765 5/92 H 5/781 G11B 27/02 B 5/92 (72) Inventor Masaru Kanno 2-1-15 Ohara, Kamifukuoka-shi, Saitama F-term in K.D. Laboratory (reference) 5B075 KK07 KK13 KK33 KK38 ND12 ND14 ND18 ND23 NR02 NR16 NR20 PP03 PP13 PP30 PQ02 PQ04 QS03 UU40 5C023 AA02 AA14 AA34 5C052 AA01 AC08 DD04 5C053 FA14 FA23 GA11 GB09 GB37 HA29 HA30 KA24 LA06 LA14 5D110 AA13 BB06 CA05 CA16 CA42 CD02

Claims (8)

[Claims] 1. A moving image search / browsing / editing apparatus for editing digital moving image contents such as MPEG, wherein a thumbnail search / browsing result using a cut point search, a telop search, or an audio classification in combination with a frame unit search. A moving image search / browsing / editing apparatus characterized by comprising means for designating an editing start point / edit end point by using a mouse. 2. The moving image search / browsing / editing apparatus according to claim 1, wherein when performing the frame unit editing processing of the MPEG content,
A moving image characterized by comprising means for decoding only part of data before and after the edit point to the baseband to perform editing and recompression, and processing other than the partial data before and after the edit point as compressed data as it is. Search, browse and edit device. 3. The moving image search / browsing / editing apparatus according to claim 1, wherein before performing frame-by-frame editing of the moving image, visual consistency at the time of reproducing the moving image at each editing point (joint portion) is checked. A moving image search / browsing / editing device comprising means for checking by splice editing (partial editing of a seam portion) and splice reproduction (partial reproduction of a seam portion). 4. The moving picture search / browsing / editing apparatus according to claim 1, wherein a list of editing sections is edited in SMPTE (Society of Society) at the time of moving picture editing.
E based on Motion Picture & Television Engineers)
A moving image search / browsing / editing device comprising: means for outputting as a DL (Edit Decision List); and means for reading the EDL from the outside and reproducing the editing operation. 5. The moving picture search / browsing / editing apparatus according to claim 1, wherein all processing is performed via a network using a web or the like. . 6. A process for decoding only partial data before and after a designated editing point using a thumbnail search and browsing result to a baseband when editing a frame unit of MPEG content, and editing at the editing point. A computer-readable recording medium recording a program for performing a process and a process of recompressing the decoded data. 7. A computer-readable recording medium on which a splice edit / reproduction processing program for realizing a visual consistency check of a joint portion at each edit point before executing an edit processing is recorded. 8. When a moving image is edited, a list of editing sections is stored in an SM.
PTE (Society of Motion Picture & Television Engine
computer-readable recording medium that records a processing program that can be input / output as an EDL (Edit Decision List) compliant with ers).