JPH0530464A - Moving image management device - Google Patents

Abstract

PURPOSE:To easily retrieve a cut or a scene in moving images, to easily recognize the constitution of the whole of moving images, and to easily realize the editing work such as cut transposition. CONSTITUTION:The image obtained by reducing one frame out of frames constituting each cut and the picture obtained by reducing the picture indicating the division of a scene are preliminarily stored in a magnetic disk 5 as images for icon, and reduced images for icon indicating each cut or scene are read out from the magnetic disk 5 and are displayed in corresponding icons on a CRT display device 2 at the time of displaying the hierarchical structure of moving images.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image management apparatus for electronically executing input, display, storage, editing, etc. of moving images.

[0002]

2. Description of the Related Art In recent years, moving images have become one of familiar media due to the spread of VTRs and video movies. Such moving images are stored by a video cassette or the like. However, as the number of moving images to be stored increases, the management method thereof becomes a problem. At present, the inefficient method of stacking video cassettes and searching the label of each video cassette is inevitable.

Further, since the video cassette is essentially a sequential access medium, a tape counter is used to search for a desired scene from one video cassette at the time of searching or editing. Or you have to repeat fast forward and rewind.

Therefore, it is not possible to easily find the necessary scenes and cuts, and it is not possible to easily edit the scenes and cuts as a unit. Furthermore, the division of cut units imposes an excessive burden on the user. There is a drawback that

[0005]

SUMMARY OF THE INVENTION As described above, according to the present invention, it is not possible to easily find a necessary scene or cut, and it is not possible to easily edit scenes or cuts as a unit. Furthermore, the division of cut units removes the disadvantage that it imposes an excessive burden on the user, and the moving images can be stored and managed in a hierarchical structure of scenes and cuts, and the necessary scenes and cuts can be easily found. It is also an object of the present invention to provide a moving image management apparatus that can perform editing in units of scenes and cuts, can automatically perform division in cut units, and can reduce the burden on the user. To do.

[0006]

SUMMARY OF THE INVENTION A moving picture management apparatus of the present invention is a detecting means for detecting a cut and a boundary between cuts in an input moving picture from the magnitude of a change amount between frames, and a detection result of this detecting means. Dividing means for dividing into a file in which a cut composed of a plurality of frames is used as a unit, and a file in which the cut divided by the dividing means is used as a unit and an image obtained by reducing one frame in the frame constituting the cut is stored. Storage means, output means for reading out an image for each file in units of cuts stored in this storage means from the storage means, and outputting it, and a scene delimiter in which cuts are grouped by the image output by this output means. Instructing means for instructing, and first processing means for storing in the storage means an image obtained by reducing the image of the scene delimiter in accordance with the instruction of the instructing means. Second processing means for storing one moving image in the storage means in a hierarchical structure of scenes and cuts according to the instruction of the instructing means, and hierarchical structure of the scenes and cuts of the one moving image in the output means. When outputting, a third process of reading an image obtained by reducing one frame in a frame constituting a cut and an image obtained by reducing an image of a scene delimiter from the storage means and performing an output process in a corresponding icon of the output means. It is composed of means.

[0007]

The present invention detects the boundaries between cuts based on the magnitude of the amount of change between frames of an input moving image, and divides the input moving image into a file in units of cuts consisting of a plurality of frames. Then, the storage unit stores the file in units of the divided cuts and the image obtained by reducing one frame in the frames constituting the cuts, and the image for each file in units of the stored cuts is stored in the storage unit. The image is read out and output, and the output image is used to instruct a scene delimiter in which cuts are grouped together, and an image in which the image of the scene delimiter is reduced in accordance with this instruction is stored in the storage means, and the instruction is given. Accordingly, one moving image is stored in the storage unit in a hierarchical structure of scenes and cuts, and when the hierarchical structure of scenes and cuts of one moving image is output by the output unit, The image obtained by reducing the image of the reduced image and scene delimiting one frame in the frame constituting is obtained so as to output the processing in the corresponding icon of the output means reads from the storage means.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail below with reference to the drawings. FIG. 1 is a block configuration diagram of a moving image management apparatus of the present invention. That is, a CPU 1 that performs various controls, a CRT display device 2 that displays a moving image and information for managing the moving image, and a CPU 1
Memory for storing control programs and images displayed on the CRT display device 2, a boundary detection unit 4 for inputting a series of moving images composed of a plurality of cuts, and detecting boundaries of each l-cut, a series of moving images or A TSC (television system committee) from a magnetic disk device 11, a video camera 7, a VTR 8 or the like that handles a magnetic disk (HDD) 5 that stores a file in units of one cut consisting of a plurality of frames cut out by the boundary detection unit 4. ) A video interface 6 for inputting a signal and converting it to a format for handling moving images with this device, a video camera 7 connected to the video interface 6, or a VTR
8 and a mouse 10 (or keyboard) as an input means.

The CPU 1, the memory 3, the boundary detecting section 4,
The video interface 6, mouse 10, and magnetic disk device 11 are connected to the bus 9 and are controlled by access from the CPU 1. If another storage medium such as an optical disk (ODD) is used instead of the magnetic disk 5,
Alternatively, a remote file via a network may be used.

FIG. 2 shows an example of a moving image format in this embodiment. 30 frames per second, 6 for 1 frame
40 × 512 pixels, one pixel is represented by 8 bits of density (Y) and 4 bits of chromaticity (I, Q). In addition, the boundary detection unit 4
In the processing, 640 × 512 pixels are divided into 8 × 8 blocks, and the blocks are used as a unit.

As shown in FIG. 3, the boundary detecting unit 4 detects the presence or absence of motion in the input image in block units within a frame as compared with the preceding frame, and performs motion compensation inter-frame prediction and intra-frame prediction. An encoder 41 for encoding by a method with less error, a boundary detector 42 for detecting a cut boundary between consecutive frames using an intra-frame / inter-frame flag output from the encoder 41, an encoded moving image It is composed of a decoder 43 for reconstructing an image.

The encoded moving image information output from the encoder 41 is divided by the boundaries detected by the boundary detector 42, and one cut is stored in the magnetic disk 5 as one file. The decoder 43 performs the decoding when displaying the file stored in the magnetic disk 5.

As shown in FIG. 4, the encoder 41 has an intra-frame / inter-frame determiner 411, an orthogonal transformer 412,
Quantizer 413, inverse quantizer 414, inverse orthogonal transformer 41
5, frame memory 416, loop filter 417, encoder 418, subtractor 410, changeover switches 408, 40
9 and an adder 407.

The intraframe / interframe determiner 411 is
Inter-frame prediction is performed between the block after motion compensation using the searched motion vector and the input block, and when the prediction error is large, intra-frame prediction is performed in the input block. When the prediction error is large, the selector switches 409 and 408 are switched to the upper side, and the image of the input block is directly output to the orthogonal transformer 412 via the selector switch 409. When the prediction error is small, the selector switches 409 and 408 are switched to the lower side, and the input block of the moving image is subtracted by the subtracter 410 from the loop filter 41.
The image subtracted by the previous image from 7 is the orthogonal transformer 4
It is output to 12.

The quadrature converter 412 has a changeover switch 409.
The image is supplied by the discrete cosine transform (DCT) which is a kind of two-dimensional orthogonal transform. The quantizer 413 quantizes the orthogonal transform content of the orthogonal transformer 412. The inverse quantizer 414 is the quantizer 4
The 13-quantized 8 × 8 coefficient is inversely quantized for motion compensation. The inverse orthogonal transformer 415 restores the contents from the inverse quantizer 414 by inverse orthogonal transform. The frame memory 416 includes an inverse orthogonal transformer 415.
Contents restored by inverse orthogonal transform from frame and frame memory 4
16 to loop filter 417 and changeover switch 40
The image (previous image) added by the adder 407 with the image supplied via 8 is stored and held.

The loop filter 417 is used to reduce the quantization error. The encoder 418 receives the intra-frame / inter-frame from the intra-frame / inter-frame determiner 411.
Codeword divided by inter-frame block instruction, quantization index and quantizer instruction for transform coefficient from quantizer 413, motion vector from frame memory 416, and loop filter on / off instruction from loop filter 417 Is output as a code.

With such a configuration, the intra-frame / inter-frame determining unit 411 performs inter-frame prediction between the block after the motion compensation using the searched motion vector and the input block, resulting in a prediction error. Is large, the changeover switches 409 and 408 are switched to the upper side, and when the prediction error is small, the changeover switches 409 and 408 are changed to the lower side. As a result, when the prediction error is large, the image of the input block is directly output to the orthogonal transformer 412 via the changeover switch 409, the orthogonal transformer 412 performs two-dimensional orthogonal transform (DCT), and the quantizer 413. Then, the contents of the orthogonal transformation are quantized and output to the encoder 418.

The quantization content of the quantizer 413 is restored by the inverse quantizer 414 and the inverse orthogonal transformer 415 and supplied to the adder 407. The adder 407 adds the restored image from the inverse orthogonal transformer 415 and the previous image from the frame memory 416, and the contents of the frame memory 416 are updated by this image.

This allows the encoder 418 to
Intra-frame / inter-frame block instruction from inter-frame determiner 411, quantization index and quantizer instruction to transform coefficient from quantizer 413, frame memory 4
Motion vector from 16 and loop filter 417
The code word assigned by the loop filter on / off instruction from is output as a code. Next, a method of dividing a moving image composed of a continuous series of frames into several cuts will be described.

Since there is a high degree of similarity between consecutive frames within one cut, inter-frame coding has less error, and therefore, there are fewer blocks to be subjected to intra-frame coding. On the contrary, between the last frame of one cut and the first frame of the next cut, the inter-frame correlation is low, the intra-frame coding is advantageous, and the blocks performing the intra-frame coding increase locally. Therefore, by performing such a determination, a series of moving images can be cut out into individual cuts.

As shown in FIG. 5, the boundary detector 42 includes an adder 421, a comparator 422, and FF circuits 423-4.
26 and logic gates 427 to 433. The adder 421 is initialized to "0" by the frame end signal (or frame start signal) from the CPU 1, and if it is within a frame by looking at the inter-frame / in-frame flag from the encoder 41 for each block. It is an adder that adds "1" and operates so that it does not change between frames. The comparator 422 is a comparator that compares the number of blocks coded in one frame with a threshold value given in advance by the CPU 1 and outputs a flag indicating change if the number of blocks is larger. ..

The FF circuits 423 to 426 are four-stage circuits which latch the flag indicating that there is a change from the comparator 422 in response to the frame end signal from the CPU 1 and hold information regarding the presence or absence of change in the past four frames. Logic gates 427-4
In 33, the presence / absence of changes in five consecutive frames is “None *
* "(* May or may not be present)," presence or absence * ",
It is a logic circuit that determines that it is a cut boundary and outputs a boundary detection flag when it is either “absent or not”.

With such a configuration, the adder 4 is supplied for each frame by the frame end signal from the CPU 1.
21 is initialized to “0”, and the encoder 4 is set for each block.
If the inter-frame / in-frame flag from 1 is within the frame, the adder 421 adds "1", and the adder 42
The addition content of 1 is output to the comparator 422. Comparator 42
When the content added by the adder 421 by 2, that is, the number of blocks encoded in one frame is larger than a predetermined threshold value, a change flag is output to the FF circuit 423.

As a result, the FF circuits 423 to 426 sequentially latch the change flags from the comparator 422 in response to the frame end signal from the CPU 1, and hold the information on the presence or absence of changes in the past four frames. FF circuit 423
When the latch contents of ~ 426 are "presence / absence **", "presence / absence / absence *", and "presence / absence / presence", it is determined to be a cut boundary by a logical operation by the logic gates 427 to 433. A boundary detection flag is output from the logic gate 428. That is, the case where the state with change continues for 3 frames or less and there is no change before and after this circuit is regarded as the cut boundary.

An example of the judgment is shown in FIG. It is assumed that the horizontal axis indicates the number of frames, and the vertical axis indicates the number of blocks subjected to intra-claim coding in each frame. Peak 1
Is a frame that is higher than the threshold value and has no change before and after the frame, so that the boundary detector 42 shown in FIG. Therefore, the first frame to the frame immediately before the peak 1 is cut out as cut 1 and stored as a file. Similarly, since the boundary of the peak 2 is also judged, the cut 2 is cut out from the peak 1 to immediately before the peak 2. On the other hand, peak 3 is not judged to be a boundary because the frames with change continue for “4” or more.

By such a determination, when there is an object moving in front of the lens of the video camera 7 or when a rapid pan operation is performed, good segmentation can be obtained without making an erroneous determination as a cut boundary.

In this embodiment, it is determined that a continuous portion in the order of "1" or more without change, "3" or less with change, and "1" or more with no change is the boundary. By changing the gate, it is possible to determine the case where no change continues for Tf times or more, there is a change for Tw times or more, and no change continues for Ta times or more.

The decoder 43 has the same structure as the encoder 42. In addition, the encoder 42 may be configured to operate as an encoder and a decoder, and the decoder 43 may not be provided.

As shown in FIG. 7, the video interface 6 is composed of an A / D converter 61 and a format converter 62. The A / D converter 61 converts the TSC signal from the video camera 7, VTR 8 or the like into a digital signal and outputs it. The format converter 62 converts the digitized TSC signal from the A / D converter 61 into the format shown in FIG.

The input signal may be another system instead of TSC. In this case, the format may be converted according to the resolution of the system. Alternatively, it may be converted into the format of this embodiment by another system in advance and input by using a network or FDD (floppy disk).

Next, the operation of this structure will be described. First, an instruction from the video cassette set in the video camera 7 or the VTR 8 is given to the magnetic disk 5 by an instruction from the mouse 10 or the keyboard. Then, 1 from the video camera 7 or VTR 8
A frame-by-frame moving image is supplied to the boundary detection unit 4 via the video interface 6 and the bus 9. Boundary detection unit 4
Encodes the input moving image and detects the cut boundary.

As a result, one cut composed of a plurality of frames is stored in the magnetic disk 5 by the magnetic disk device 11 as one cutting file. At this time, on the magnetic disk 5, as shown in FIG.
The moving image name, scene number, cut number, cut length (seconds), icon image, high-speed display image, and encoded moving image information consisting of multiple frames for one cut are stored as files for cutting. To be done. At this time, the cut numbers are serially assigned in the order of cutting.

As the icon image, an image in which an appropriate frame selected from the cuts to represent the cut is reduced to the size of the icon and decoded is stored. Since it is difficult to select the representative frame by considering the content of each cut, it is sufficient to decide the first frame, the predetermined frame from the beginning, the middle frame, and the like. Since a large change does not occur between frames within one cut, such a simple selection is sufficiently effective.

Further, in general, important information often appears in the central portion of one frame, so trimming may be performed centering on the center of the frame to make the icon size.

In the high-speed display image (representative image), an appropriate frame selected so as to represent the cut is decoded and stored as in the case of the icon. This image may be reduced in advance according to the size of the high-speed display window. This image is displayed in the high-speed display window as a representative image of the cut in the mode in which the entire moving image or the scene is viewed at high speed.

In order to take out a predetermined frame from the encoded moving image, it is necessary to decode from the first frame, which takes time. If a decoded image for high-speed display is stored in advance as in this embodiment, it can be read out in the shortest time.

Further, one scene is stored in the magnetic disk 5 by the magnetic disk device 11 as one scene file. At this time, on the magnetic disk 5, as shown in FIG. 9A, a moving image name, a scene number, an icon image, and a high-speed display image are stored as scene files. As a result, the scene can be displayed at high speed only by the scene.

The image for an icon is an image obtained by reducing the image of a scene delimiter to the size of the icon, and the image for high speed display is an image obtained by reducing the image of a scene delimiter to the size of the high speed display window. The icon image and the high-speed display image are registered in advance by the decoder 43.

Further, the operator uses the mouse 10 or the keyboard to instruct the reading of the image registered in the magnetic disk 5. Then, the CPU 1 sequentially reads only the first frame of each cut of the corresponding moving image name and displays it on the CRT display device 2. In response to this display, the operator uses the mouse 10 or the keyboard to instruct whether or not the cut division is correct, and the division of the scene (several continuous cuts as a semantic and temporal unit).

The CPU 1 assigns a scene number to each cut of the magnetic disk 5 and changes the cut number to a serial number for each scene. When it is instructed that the cut division is not correct, the CPU 1 re-registers the cuts before and after the division of the magnetic disk 5 as one cut.

Therefore, the relationship between each scene and each cut for one continuous moving image on the magnetic disk 5 is as follows.
Registered in a tree structure (hierarchical structure) configured as shown in (a) and (b) of FIG. 8, in which there are a plurality of scenes including a plurality of cuts, and one continuous moving image is formed by the plurality of scenes. To be done.

When the operator gives an instruction for image editing or the like with the mouse 10 or keyboard and a moving image name, an image showing a tree structure (hierarchical structure) corresponding to the moving image name is output from the magnetic disk 5. It is read and displayed in the window 21 of the CRT display device 2 as shown in FIG. In this window 21, a structure in which one moving image is processed and classified into scenes and cuts is displayed, and each scene and cut is displayed by an icon composed of images representing each scene.

Then, when the operator gives an instruction to reproduce a predetermined icon with the mouse 10 or the keyboard,
Another window 22 opens, and the scene or cut indicated by the icon is read from the magnetic disk 5 and displayed in the window 22.

On the other hand, when the operator instructs the moving image or the scene to be reproduced at high speed with the mouse 10 or the keyboard, the images for high-speed display of cuts located under the moving image or the scene are read from the magnetic disk 5 one after another. As shown in FIG. 11, the high-speed display windows 23 of the CRT display device 2 are displayed. By doing so, the operator can quickly and easily understand the content of the moving image or the scene.

As described above, the input moving image is
The boundaries between cuts are detected based on the amount of change between
A frame consisting of multiple frames according to the detection result.
The file is divided into files in units of
The file of each unit and the image for high-speed display are magnetic.
It is stored in a disc, and the unit of this stored cut is
Read and output images for each file from the magnetic disk
Then, cuts were put together by this output image.
Instruct scene breaks with mouse or keyboard
Then, according to this instruction, one moving image is
Stored in the above magnetic disk in a hierarchical structure to output high-speed moving images.
Image for high-speed display of each of the above cuts when force is specified
To read and output from the above magnetic disk continuously.
It is the one.

As a result, moving images can be stored and managed in a magnetic disk in a hierarchical structure of scenes and cuts.
Necessary scenes and cuts can be easily found, editing can also be easily performed in units of scenes and cuts, and divisions in cut units can be automatically performed to reduce the burden on the user.

In addition, one frame in each cut is reduced in advance, and the decoded image and the image indicating the boundary between scenes are reduced, and the decoded image is recorded on the magnetic disk as an image for an icon. Stored and displaying a hierarchical structure of a moving image, a reduced image for an icon indicating each cut or scene is read from a magnetic disk and displayed in a corresponding icon of a CRT display device. Is.

As a result, the cuts or scenes in the moving image can be easily searched, the structure of the entire moving image can be easily grasped, and editing work such as rearranging the cuts can be easily realized.

[0049]

As described in detail above, according to the present invention,
You can store and manage moving images in a hierarchical structure of scenes and cuts, easily find the necessary scenes and cuts, and easily handle editing in scene and cut units. It is possible to provide a moving image management apparatus that can automatically divide a unit and reduce the burden on the user.

[Brief description of drawings]

FIG. 1 is a schematic block diagram showing the overall configuration of an embodiment of the present invention.

FIG. 2 is a diagram showing an example of a moving image format.

FIG. 3 is a block diagram showing a configuration of a boundary detection unit in FIG.

FIG. 4 is a diagram showing the structure of the encoder of FIG.

5 is a diagram showing a configuration of the boundary detector of FIG.

FIG. 6 is a diagram for explaining the operation of the boundary detector of FIG.

7 is a block diagram showing the configuration of the video interface of FIG.

FIG. 8 is a diagram for explaining a moving image management structure.

9 is a diagram showing a configuration example of a scene file and a cut file stored on the magnetic disk of FIG. 1. FIG.

10 is a diagram showing an example of a display screen on the CRT display device of FIG.

11 is a diagram showing an example of a high-speed display screen in the CRT display device of FIG.

[Explanation of symbols]

1 ... CPU, 2 ... CRT display device, 3 ... Memory, 4 ... Boundary detecting unit, 5 ... Magnetic disk, 6 ... Video interface, 7 ... Video camera, 8 ... VTR, 9 ...
Bus, 10 ... Mouse, 11 ... Magnetic disk device, 22 ...
High-speed display window, 41 ... Encoder, 42 ... Boundary detector, 43 ... Decoder, 411 ... In-frame / interframe determiner, 421 ... Adder, 422 ... Comparator, 423 ...
426 ... FF circuit, 427-433 ... Logic gates.

Claims (1)

Claim: What is claimed is: 1. A detection means for detecting a boundary between cuts based on the magnitude of a change amount between frames of an input moving image, and a plurality of frames according to a detection result of the detection means. Dividing means for dividing the cut into units of files, and storage means for storing a file divided by the dividing means as a unit and an image obtained by reducing one frame in a frame constituting the cut, Outputting means for reading out the image for each file in units of cuts stored in the storage means from the storage means and outputting the same, and an instruction for instructing a scene division in which the cuts are grouped together by the image output by the output means. Means, first processing means for storing, in the storage means, an image obtained by reducing an image of a scene delimiter in accordance with an instruction from the instruction means, Second processing means for storing one moving image in the storage means in a hierarchical structure of scenes and cuts in accordance with an instruction from the means, and hierarchical structure of scenes and cuts in the one moving image are output by the output means. When, 1 in the frame that constitutes the cut
And a third processing means for reading the reduced frame image and the reduced scene separation image from the storage means and outputting them in the corresponding icon of the output means. Image management device.