JP2001119661A - Dynamic image editing system and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To effectively utilize dynamic image data captured in a computer or the like by using a still picture, so as to allow a user to immediately confirm the contents of the dynamic image data. SOLUTION: A dynamic image analysis section 12 detects an image pattern changing point, at which a video image is greatly changed by analyzing moving picture data and records a decoded image corresponding to the image pattern changing point to a still image storage memory 13 as a still image of a bit map form. Thus, a still image after decoding processing is recorded to the still image storage memory 13 as a shot of the moving image data. A still image simultaneous display section 15 displays a plurality of still images stored in the still picture storage memory 13 together as the index images of the dynamic image data. Furthermore, a fast forward/rewinding reproducing section 17 consecutively reproduces the plurality of still image recorded in the still image storage memory 13 at equal intervals so as to execute fast forward reproduction of the dynamic image data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture editing system and a recording medium, and more particularly to a moving picture editing system used for displaying an index of moving picture data and a recording medium recording a computer program therefor.

[0002]

2. Description of the Related Art In recent years, with advances in semiconductor technology and digital image processing technology, personal computers (PCs) have been developed.
In such cases, capture moving image data from the camera,
Processing such as recording / editing / reproduction can be easily performed.

[0003] Since moving image data functions as a video only by displaying and reproducing images for each frame continuously, in order to confirm the contents of the moving image data, it is necessary to perform the reproduction processing of the moving image data without fail. Required. For this reason, for example, when a plurality of moving image data are recorded in a PC, the contents cannot be confirmed without reproducing the individual moving image data. It takes time.

[0004] In general, VTRs and the like have functions such as fast-forward playback / fast-forward reverse playback (rewind playback) as functions for quickly confirming the contents of video. By realizing this playback / fast-forward reverse playback function on a PC, the time required for confirming the content of the video can be reduced to some extent.

However, the conventional fast-forward playback / fast-forward reverse playback function plays back all frames at a high playback speed such as double speed or triple speed. For this reason, even if a monotonous video scene continues for a long time, it is not possible to check the content of the video unless you keep watching it, and in the part where the screen change is large, the display time of each frame is short, so This makes it difficult to confirm the contents of the video.

In the case of fast-forward playback of moving image data that has been compression-encoded by MPEG or the like, the only data that can be played back is I-data that can be decoded independently without using a reference image.
Since it is limited to only the picture, there is a problem that it is difficult to confirm the motion of the video.

[0007]

As described above, in the related art, even if moving image data is taken into a PC or the like, a normal V
Only the same reproduction control as that of the TR can be performed, and it is practically difficult to effectively utilize the captured moving image data. In particular, there is a problem in that the content of the moving image data must be reproduced in order to confirm the content, which requires a lot of time.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-described circumstances, and enables a user to immediately check the contents of moving image data by displaying a list of still images and continuously reproducing the still images, and It is an object of the present invention to provide a moving image editing system capable of sufficiently utilizing image data and a recording medium on which a computer program for the moving image editing system is recorded.

[0009]

In order to solve the above-mentioned problems, a moving image editing system according to the present invention comprises a detecting means for analyzing input moving image data to detect a screen change point; Still image recording means for recording an image at a screen change point as a still image, and display means for displaying a list of a plurality of still images recorded by the still image recording means as index images.

In this moving image editing system, an image at a screen change point where a video motion is large is extracted from input moving image data, and is recorded as a still image. Then, since the plurality of recorded still images are collectively displayed as an index image in a list, without playing a moving image,
The contents of the input moving image data can be checked quickly.

[0011] Further, by further providing an index file storing means for storing an index image composed of a plurality of still images displayed in a list as one file, an index image of moving image data can be easily attached to an e-mail. And can be used as a preview video of moving image data on a homepage or the like.

Further, there is further provided a number setting means for setting the number of still images to be displayed as a list of index images in accordance with a user operation, and the number setting means of the plurality of still images recorded by the still image recording means. It is preferable that the still images for the number of images set as described above are displayed in a list as index images. This makes it possible to display an index image composed of a predetermined number of still images regardless of the number of screen change points, and to display the index image in a form that is easy for the user to see.

[0013] Further, for each still picture recorded by the still picture recording means, means for recording a corresponding position on the input moving picture data as position information, and among the plurality of still pictures displayed in a list, When a predetermined still image is selected from, the position information corresponding to the selected still image is obtained from the position information recording means, and the input moving image data is obtained from the position corresponding to the selected still image. And a moving image reproducing means for starting the reproduction of the moving image can be easily reproduced from a position corresponding to each still image.

Further, the moving picture editing system of the present invention comprises a detecting means for analyzing input moving image data to detect a screen change point, and a still image for recording an image of the screen change point detected by the detecting means as a still image. It is characterized by comprising image recording means, and reproducing means for continuously reproducing a plurality of still images recorded by the still image recording means.

In this moving image editing system, an image at a screen change point is extracted from input moving image data and recorded as a still image. Then, by continuously playing back a plurality of recorded still images, fast-moving portions and fast-moving reverse playback display a large number of still images in a portion having a lot of motion and a still image in a portion having a slow motion. This makes it possible to more efficiently check the contents of a video in a short time as compared with the case where fast forward playback / fast forward reverse playback is performed at double speed for all frames.

In addition to a mode in which a still image at each screen change point of input moving image data is displayed as an index image, still images at predetermined time intervals of input moving image data can be displayed according to the time or number of sheets specified by the user. A mode for displaying as an index image may be prepared, and still images may be displayed at predetermined time intervals. As a result, it is possible to obtain an index image that the user desires, regardless of the presence or absence of motion.

[0017]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows a functional configuration of a moving image editing system according to an embodiment of the present invention. The moving image editing system is for performing processing such as editing and reproduction of moving image data, and is used by being mounted on, for example, a notebook or pocket personal computer (PC) with a camera. The moving image editing system has the following functions.

1) A function of extracting and displaying several representative still images for each scene by grouping similar images (scenes). 2) Listing a plurality of representative still images by batch display. Function of presenting the contents flow to the user 3) Function of starting moving image reproduction from the video of the selected scene by selecting the listed still images 4) Continuously reproducing a plurality of representative still images A function for performing fast forward playback / fast forward reverse playback of a moving image in a simple manner will be described below.

As shown, the moving image editing system includes a moving image input unit 11, a moving image analysis unit 12, a still image storage memory 13,
It comprises a time storage memory 14, a still picture batch display section 15, a still picture batch storage section 16, a fast forward / rewind playback section 17, a moving picture playback section 18, and the like. Other units other than the still image storage memory 13 and the time storage memory 14,
Implemented by software.

The moving image input section 11 is for reading a video file captured from a camera or the like and recorded on a disk device or the like in a PC or a video file recorded on a CD-ROM or the like. The video file is composed of moving image data that has been compression-encoded by, for example, MPEG2 or MPEG4. Video input unit 11
Is input to the moving image analysis unit 12, where decoding processing, analysis processing for detecting a screen change point, and the like are performed.

In MPEG, a motion-compensated prediction coding mode is used. In addition to I-pictures coded in a frame, P-pictures for forward prediction and B-pictures for bidirectional prediction Contains a picture. The P picture and the B picture are image data encoded in the motion compensation prediction encoding mode.
A picture is represented by prediction error information between an image obtained by performing motion prediction from a reference frame and an input image, and a motion vector used in motion prediction.

The moving image analysis unit 12 detects a screen change point where the video changes greatly by analyzing the moving image data, and decodes the decoded image corresponding to the screen change point as a bitmap still image in a still image storage memory. Record at 13.
Since a motion vector is large in a frame in which an image greatly changes, a screen change point can be detected based on the value of the motion vector. That is, the moving image analysis unit 12 checks the magnitude of the motion vector for each picture, and records the image as a bitmap still image if the value is larger than a predetermined threshold value. If the value of the motion vector is smaller than a certain value, it is determined that the image has not changed,
The still image is not recorded and the next video is analyzed.

When recording a still image, the moving image analysis unit 12 stores time information indicating the position of the frame image to be recorded as the still image from the top of the moving image in the time storage memory 14. . As the time information, a frame number or the like of a frame image recorded as a still image can be used.

The still image batch display unit 15 displays a list of a plurality of still images stored in the still image storage memory 13 as index images of moving image data. The index image is composed of thumbnail images (reduced images) of a plurality of still images. That is, the still image storage memory 1
3, since the still images after the decoding process are recorded as shots of the moving image data, the still images are collectively displayed in a lump to form a list of shots for each screen change point of the moving image data. Will be displayed.

In this case, the still image storage memory 13
It is not necessary to display the thumbnail images of all the still images stored in the camera as index images. Only the number of still images specified in advance by the user are read out, and a list of those thumbnail images is displayed as an index image. I just need.

The still image batch storage unit 16 collectively stores a plurality of still images listed as index images by the still image batch display unit 15 as one file (index image file). The index image file is stored in a disk device in the PC with a file name obtained by adding a specific symbol to the video file name of the original moving image data.

The fast-forward / rewind playback unit 17 plays back a plurality of still images recorded in the still-image storage memory 13 continuously at a constant interval, so that fast forward playback or fast-forward reverse playback (rewind playback) is performed. Execute The moving image reproducing unit 18 starts reproducing moving image data from a position corresponding to a specific still image specified on an index display screen (shot view) in which a plurality of still images are displayed in a list. The time information corresponding to the specified still image is acquired from the time storage memory 14, and the moving image data is decoded and reproduced from the position specified by the time information.

(Image Extraction Processing) Next, referring to FIG.
The principle of processing for extracting and recording a still image from moving image data will be described.

In the case of moving image data of 30 frames / second,
The analysis processing of the moving image data by the moving image analysis unit 12 is performed in units of time intervals such as 0.5 second intervals. In this case, each unit time interval includes 15 frames of image data. For each set of these 15 frames of image data,
At least one frame having a large motion vector value is selected, and the decoded image is recorded as a still image at a screen change point.

In FIG. 2, the image data from the first frame to the fifteenth frame are divided into frames f2 and f2.
3 and f4 are selected as images having large motion vectors, and the image data from the 16th frame to the 30th frame are selected from frames f18 and f18.
When the 19 image data are the image data from the 31st frame to the 45th frame, the frames f32 and f3
3, a case where the image data of f34 and f35 are selected as images having large motion vectors is shown.

In this case, since the decoded image of each of the selected frames is recorded as a still image, the frames f2 and f2 are displayed on the index display screen as shown in FIG.
3, f4, f18, f29, f32, f33, f34,
The thumbnail images of the still images of f35,... are displayed in a list. When the number of still image thumbnail images is large, all still images can be confirmed by scrolling the screen.

Actually, the time unit of 0.5 second interval described in FIG. 2 is a typical GOP (Group O).
f Picture). The first picture of the GOP is an I picture independently coded without using a reference screen, and is often inserted as a random access point. For this reason, an I picture may be recorded as a still picture, and an image having a motion vector value larger than a predetermined threshold value among the remaining P and B pictures may be recorded as a still picture. As a result, an image of at least one frame can be recorded as a still image at each unit time interval.

(Index Creation Process) Next, a flow of a series of processes for creating an index image from moving image data will be described with reference to the flowchart of FIG.

First, the moving image analysis unit 12 performs a motion vector analysis process on moving image data and a decoding process for decoding each picture (steps S11 and S12).
As for the P and B pictures, the decoded image cannot be obtained without the decoded reference image. Therefore, in the index creation processing, the decoding of the picture that is not actually recorded as a still image is also performed. Since the B picture is not used as a reference image, the decoding process may be performed only when the B picture is recorded as a still image.

Next, the moving image analysis unit 12 compares the value of the motion vector with the threshold value Th, and determines whether or not the image is a screen change point having a large motion (step S13). If the image is a screen change point having a large movement, the moving image analysis unit 12
Saves the decoded image in the bitmap format in the still image storage memory 13 (step S14), and then records time information corresponding to the decoded image in the time storage memory 14 (step S15).

(Index Display Processing) Next, with reference to the flowchart of FIG. 5, a procedure of index display processing for displaying a list of still images as index images will be described.

After reading a plurality of still images recorded therein from the still image storage memory 13 (step S21), the still image batch display unit 15 creates a thumbnail image for each still image and The thumbnail images of the still images are displayed collectively on the index display screen (step S22). When saving (Save) of the index image is designated by the user on this index display screen (YES in step S23), the still image batch saving unit 16 is activated.

The still image batch saving section 16 saves an index image composed of a plurality of thumbnail images displayed on the index display screen as one index image file (step S24). By saving as a single file in this way, the file can be used later to attach an index image that easily represents the contents of the moving image data to an e-mail, or to preview the moving image data on a homepage or the like. It can be used as a video.

(Mode Setting) FIG. 6 shows an example of a mode setting screen for setting the operation mode of the present moving image editing system by GUI operation.

The moving image editing system according to the present invention has a display mode (screen change mode) in which a still image at each screen change point of input moving image data is displayed as an index image. A display mode (time / number designation mode) for displaying images as index images is provided.

The "shot" button in the display mode setting items of FIG. 6 is a button for designating the display of the above-mentioned index display screen. When either the "time designation" button or the "number designation" button is selected with the "shot" button selected, the time / number designation mode is selected.

When the "time designation" button is pressed,
Still images at fixed time intervals specified by the number of seconds input to the input field on the right are displayed as thumbnails on the index display screen. It is possible to specify 0.5 seconds to 60 seconds. As mentioned earlier, regardless of the magnitude of the screen change,
By recording still images one by one at 0.5 second intervals, a list of still images can be displayed at regular time intervals.

When the "designate number of sheets" button is pressed, the still images at the fixed time intervals determined by the value of the number of sheets entered in the input field on the right thereof are stored in the still image storage memory. 13 and is displayed as a thumbnail on the index display screen.

When the "screen change" button is pressed while the "shot" button is selected, the screen change mode is selected. In this screen change mode, only a still image at a screen change point with a large motion is displayed as a thumbnail on the index display screen. Also in this screen change mode, the number of still images to be displayed can be designated.
As a result, even for a moving image having many screen change points, only the required number of still images can be displayed as thumbnails or saved as an index image file.

The setting item of the reproduction speed in FIG. 6 is for specifying the magnification of the reproduction speed, and fast forward reproduction / fast forward reverse reproduction can be performed at the speed designated by the user.
The value of “reproduction frame rate” in the setting item of the reproduction speed indicates the frame rate of the analyzed moving image data. The fast-forward playback / fast-forward reverse playback process is performed using a still image recorded in the still image storage memory 13. This processing will be described later with reference to FIG.

(Index Creation Process 2)
A second example of the index creation processing for creating the index image from the moving image data will be described with reference to the flowchart in FIG.

The above-mentioned "screen change mode" and "time / number designation mode" can be used not only as a display mode but also as a recording mode for recording a still image. That is, the method of processing for extracting and recording an image from moving image data is changed according to the set mode.

In this case, the moving image analysis unit 12 first determines whether the current mode is the “screen change mode” or the “time / number designation mode” (step S31).
When the “time / number-of-sheets designation mode” is selected by the user, the moving image analysis unit 12 extracts an image from the moving image data at fixed time intervals and stores the decoded image as a still image in the still image storage memory 13. Record (Step S3
2). Then, in the index display process, the still image batch display unit 15 reads out still images at fixed time intervals from the still image storage memory 13 according to the designated time or the designated number set by the user, and displays them on the index display screen in the form of thumbnails. (Step S33).

On the other hand, when the “screen change mode” is selected by the user, the moving image analysis unit 12 records a still image in the still image storage memory 13 for each screen change point according to the procedure described with reference to FIG. (Step S34). Then, in the index display processing, the still image batch display unit 15 reads out the still images of the number set by the user from the still image storage memory 13 and displays the thumbnails on the index display screen (step S35). In this case, basically, the first still image in terms of time only needs to be displayed with priority, and if information on the magnitude of the motion vector is recorded in association with each still image, the motion can be further improved. A large still image can be displayed with priority.

(Fast-forward playback / fast-forward reverse playback) Next, FIG.
The fast-forward playback / fast-forward reverse playback process using the still image recorded in the still image storage memory 13 will be described with reference to the flowchart of FIG.

The fast-forward / rewind playback unit 17 reads a plurality of still images recorded therein from the still image storage memory 13 (step S51), and designates these still images on the setting screen of FIG. Reproduction is continuously performed at predetermined time intervals determined by the reproduction speed magnification (step S52). In the case of fast-forward playback, playback may be performed in order from the past in time, and in the case of fast-forward reverse playback, playback may be performed in order from the latest in time.

When the "screen change mode" is designated, in a scene where the motion of the video data is intense, a large number of still images are recorded in the still image storage memory 13 because the motion vector is large, and conversely, the motion is changed. In a gentle scene, a small number of still images are recorded in the still image storage memory 13.
If the still images obtained in this way are played back at fixed time intervals, the user can easily view the contents of the video at that time without pausing the video because there are many still images in the rapidly moving part. Can be confirmed. In addition, in the portion where the movement is gentle, the content of the video can be confirmed in a short time because there are few still images but little movement.

In the video reproduced in this manner, the contents of the video can be more accurately expressed as compared with the normal fast forward. Also, since what is reproduced and displayed is an image after decoding, unlike the case where fast-forward reproduction is performed while decoding encoded moving image data, restrictions such as "reproduction image is limited to I picture" Not at all.

(Moving Picture Reproduction Processing) Next, the moving picture reproduction processing executed by the moving picture reproduction section 18 will be described with reference to the flowchart of FIG.

When a thumbnail image of a specific still image is double-clicked on the index display screen, the moving image reproducing process by the moving image reproducing unit 18 is started. Video playback unit 18
First, the time information corresponding to the still image on the index display screen designated by the double-click operation is obtained from the time storage memory 14 (step S51), and the reproduction start position on the moving image data is determined based on the time information. Is determined (step S52). Thereafter, the moving image reproducing unit 18 starts decoding and reproducing the moving image data from the reproduction start position determined in step S52 (step S53). In addition,
In practice, Since the decoding / playback processing cannot be started from the B picture, the decoding / playback processing is performed from the I picture closest to the playback start position determined in step S52.

By preserving the position from the beginning of the moving image data at the same time as the still image, the original video data can be reproduced from the specified position of the still image.

As described above, in the present embodiment, a list display of a plurality of still images extracted from moving image data,
By realizing fast-forward playback / fast-forward reverse playback using these still images, it is possible to immediately confirm the contents of moving image data.

The method according to the present embodiment can be applied not only to moving image data that has been compression-encoded, but also to raw moving image data. In this case, the detection of the screen change point can be realized by analyzing the moving image data and examining the magnitude of the correlation for each frame.

Further, since the moving picture editing system of the present embodiment is realized by a computer program, the same effect as in the present embodiment can be easily obtained only by introducing the computer program into a normal computer through a recording medium. It becomes possible.

[0061]

As described above, according to the present invention,
By displaying a list of a plurality of still images extracted from the moving image data and performing fast-forward playback / fast-forward reverse playback using the still images, the contents of the moving image data captured by a computer or the like can be immediately confirmed. .

[Brief description of the drawings]

FIG. 1 is a block diagram showing a functional configuration of a moving image editing system according to an embodiment of the present invention.

FIG. 2 is an exemplary view showing the principle of still image extraction processing from moving image data in the moving image editing system of the embodiment.

FIG. 3 is an exemplary view showing an example of an index display screen used in the moving image editing system of the embodiment.

FIG. 4 is an exemplary flowchart illustrating the procedure of an index creation process executed by the moving image editing system of the embodiment.

FIG. 5 is an exemplary flowchart illustrating the procedure of an index display process executed by the moving image editing system of the embodiment.

FIG. 6 is an exemplary view showing an example of a mode setting screen used in the moving image editing system of the embodiment.

FIG. 7 is an exemplary flowchart illustrating a second example of the index creation processing executed by the moving image editing system of the embodiment.

FIG. 8 is an exemplary flowchart showing the procedure of fast-forward playback / fast-forward reverse playback processing executed by the moving image editing system of the embodiment.

FIG. 9 is an exemplary flowchart showing the procedure of a moving image reproduction process executed by the moving image editing system of the embodiment.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 11 ... Video input part 12 ... Video analysis part 13 ... Still image storage memory 14 ... Time storage memory 15 ... Still image batch display part 16 ... Still image batch storage part 17 ... Fast forward / rewind reproduction part 18 ... Video reproduction part

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5C053 FA07 FA14 GA11 GB08 GB29 GB38 GB40 HA24 HA25 HA30 KA04 KA24 LA01 LA11 5C059 KK39 MA00 MA15 NN43 PP01 PP05 PP06 PP07 PP26 SS17 SS18 SS20 TA66 TB03 TC14 TD11 UA05

Claims (10)

[Claims] A detecting means for analyzing input moving image data to detect a screen change point; a still image recording means for recording an image of a screen change point detected by the detecting means as a still image; A display unit for displaying a list of a plurality of still images recorded by the recording unit as index images. 2. The moving image editing system according to claim 1, further comprising: an index file storage unit that stores the index images composed of the plurality of still images displayed as a list as one file. 3. The apparatus according to claim 1, further comprising: a number setting unit configured to set a number of still images to be displayed in a list according to a user operation, wherein the display unit includes a plurality of still images recorded by the still image recording unit. 2. The moving image editing system according to claim 1, wherein the number of still images set by the number setting means is displayed as a list of index images. 4. A position information recording unit for recording, as position information, a corresponding position on the input moving image data for each still image recorded by the still image recording unit; When a predetermined still image is selected from the images, position information corresponding to the selected still image is obtained from the position information recording unit, and the position information corresponding to the selected still image is obtained from the position corresponding to the selected still image. The moving image editing system according to claim 1, further comprising moving image reproducing means for starting reproduction of moving image data. 5. A detecting means for analyzing input moving image data to detect a screen change point; a still image recording means for recording an image of a screen change point detected by the detecting means as a still image; A moving image editing system comprising: a reproducing unit that continuously reproduces a plurality of still images recorded by the recording unit. 6. The input moving image data is composed of a coded sequence subjected to motion compensation prediction coding, and the detecting means detects a screen change point based on a motion vector included in the coded sequence. 6. The moving image editing system according to claim 1, wherein the still image recording unit decodes the encoded sequence and records a decoded image at the screen change point as a still image. 7. A still image recording means for analyzing input moving image data and recording images at predetermined time intervals as a still image, and further comprising a user selecting a plurality of still images recorded by the still image recording means. And a display means for reading out still images at predetermined time intervals specified by the time input from the CPU and displaying a list of the still images as index images. 8. A still image recording means for analyzing input moving image data and recording images at predetermined time intervals as a still image, and a user from among a plurality of still images recorded by the still image recording means. And a display means for reading out still images at predetermined time intervals in accordance with the index display number input from the CPU and displaying the still images as index images in a list. 9. A computer-readable recording medium on which a computer program for processing input moving image data is recorded, wherein the computer program analyzes the input moving image data and detects a screen change point. A still image recording step of recording an image of a screen change point detected by the detection step as a still image; and a display step of listing a plurality of still images recorded by the still image recording step as index images. A recording medium comprising: 10. A computer-readable recording medium on which a computer program for processing input moving image data is recorded, wherein the computer program analyzes the input moving image data and detects a screen change point. A still image recording step of recording an image of a screen change point detected by the detection means as a still image, and a reproduction step of continuously reproducing a plurality of still images recorded by the still image recording step. A recording medium characterized by comprising: