JPH06253255A - Video index display device - Google Patents

Abstract

PURPOSE:To easily retrieve a required scene by collecting plural index pictures obtained from video image comprising plural consecutive scenes for each scene and displaying the collected picture onto an edit pattern. CONSTITUTION:A scene change detection section 2 detects a scene from a digital moving picture. When a scene change is detected by the detection section 2, information such as a detected frame picture and a scene number is managed by a scene change management section 3. On the other hand, a characteristic frame detection section 4 matches a shape pattern stored in a material management section 5 as to a digital moving picture to detect a characteristic frame in an index picture. Then the scene change management section 3 manages the index detected by the scene change detection section 2 and the characteristic frame detection section 4 for each scene and outputs the index information to an index display section 6 and it is displayed on an edit pattern and recorded on a recording section 9.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention uses several representative frame images extracted from a video as an index when editing a video captured by a video camera or extracting only a necessary portion from the captured video. The present invention relates to a video index display device displayed as an image.

[0002]

2. Description of the Related Art Conventionally, when editing a video image shot by a video camera or extracting only a necessary portion from the shot video image, a plurality of representative frame images extracted from the video image are edited as index images. The necessary parts are searched by displaying them on the screen over time and following those index images. As the index image, for example, a frame image of a scene-changed portion in a video or a characteristic portion in one scene is extracted.

However, in the index image display method based on this method, a plurality of index images extracted from a video composed of a plurality of continuous scenes are
Since they are simply arranged in time, the relationship between the index images is unclear. For this reason, it is not possible to distinguish the boundary between the index images obtained from the same scene and the connection between the plurality of scenes, which is insufficient for grasping the content of the video.

[0004]

As described above, in the conventional video index display method, since the index images are displayed only with time, the connection between the index images and the connection between the scenes are made. It was not possible to distinguish the boundaries, and it was not possible to fully understand the contents of the video from the display screen.

The present invention has been made in view of the above points, and a plurality of index images obtained from a video composed of a plurality of consecutive scenes are indexed in each scene. It is an object of the present invention to provide a video index display device that performs a display that enables the user to easily understand the video contents.

[0006]

According to the present invention, one or more index images per scene are extracted from a video including a plurality of scenes, and the index images are included in each scene for each index image. It is characterized in that it is displayed in a form that expresses the relationship between them.

[0007] Also, a management means for dividing and managing a video composed of a plurality of consecutive scenes by detecting a frame image corresponding to a scene change in the video, and a characteristic frame from each scene. And a detection unit that detects an image as an index image.

According to the present invention, a scene change detecting means for detecting a frame image corresponding to a scene change as an index image from a digital moving image composed of continuous frame images, and an index of a characteristic frame image from the digital moving image. A scene change management unit that manages, for each scene, a characteristic frame detection unit that detects an image, an index image that is detected by the scene change detection unit, and an index image that is detected by the characteristic frame detection unit, and the scene change. Index display means for displaying the index images in a form expressing the relationship of the index images managed by the management means.

Further, the index display means arranges, with respect to the representative index image for each scene, which is a representative index image in the scene, the index images included in each scene according to the appearance order in the digital moving image. Is displayed in a subordinate form.

[0010]

According to this structure, the index image obtained from the video composed of a plurality of continuous target scenes is displayed hierarchically or only the index image that should be the representative of the scene is displayed on the editing screen. By displaying in such a form, the contents of the video can be easily grasped by looking at the index image on the editing screen, and the required video portion can be efficiently searched.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of a video index display device according to an embodiment of the present invention. As shown in FIG. 1, the video index display device includes a digital moving image input unit 1, a scene change detection unit 2, a scene change management unit 3, a characteristic frame detection unit 4, a material management unit 5, an index display unit 6, and an operation control. It is composed of a unit 7, an operation unit 8 and a recording unit 9.

The digital moving image input section 1 inputs a digital moving image composed of continuous frame images, which is generated by digitizing a video signal captured by a video camera or the like.

The scene change detection unit 2 detects a scene change from the digital moving image input by the digital moving image input unit 1, and outputs the scene number together with the frame image of the scene changed portion to the scene change management unit 3. Here, the changed portion of the scene may be a portion where the screen is switched or a portion where a characteristic object appears in the video.

The scene change management section 3 is effective in retrieving an image by the frame image (index image) and scene number of the scene change portion detected by the scene change detection section 2 and the characteristic frame detection section 4 described later. A characteristic frame image (index image) detected as a characteristic frame image is input and managed for each index image included in one scene.

The characteristic frame detection unit 4 detects the digital moving image input by the digital moving image input unit 1.
By using a shape pattern stored in a material management unit 5 described later, a characteristic frame image that is effective when searching a video is detected as an index image and output to the scene change detection unit 3. .

The material management unit 5 is for storing the shape pattern used for detecting the characteristic frame in the characteristic frame detection unit 4. The shape pattern is
This is for detecting a characteristic frame image.For example, when it is necessary to search the display screen displayed on the display device from the video, the pattern of the display screen frame of the display device is stored as a shape pattern. deep. As a result, the frame image captured by the display device can be detected as an index image.

The index display unit 6 has a predetermined display mode described later regarding the index images detected by the scene change detection unit 2 and the characteristic frame detection unit 4 and the index image group managed by the scene change management unit 3. The screen for editing is displayed.

The operation control unit 7 edits the index image group in the recording unit 9 described later in response to an editing instruction on the editing screen displayed on the index display unit 6 via the operation unit 8 described later, and It controls operations such as changing the form of index display.

The operation unit 8 is for inputting an edit instruction for the index group displayed on the index display unit 6, and controls the man-machine interface. The editing instruction input via the operation unit 8 is realized by the operation control unit 7 to the index display unit 6 and the recording unit 9.

The recording unit 9 is for recording the index image group managed for each scene by the scene change management unit 3. The information recorded in the recording unit 9 is changed according to the editing instruction input from the operation unit 8.

Next, the operation of the embodiment will be described.
First, the digital moving image input by the digital moving image input unit 1 is sent to the scene change detection unit 2 and the characteristic frame detection unit 4.

The scene change detector 2 detects a scene from a digital moving image. As a scene change detecting method in the scene change detecting unit, for example, it is detected by taking a difference between frame images, or “DC
Automatic scene change detection method using T conversion coefficient "Atsushi Fuji, Kazuhiro Mori, Masafumi Tamura, 1992 IEICE Spring Conference, D-529, pp7-271. This technology is based on JPEG (Joint Photographic E
Among the information obtained in the image compression process of the xperts Groupe) method, the direct current (DC) component count is DCT (Discrete cosin).
e transform) Note that it corresponds to the average brightness of the image in the block, and by using that value to detect the scene change, the change from a scene with a lot of movement to a scene with little movement, or from a scene with little movement. It is possible to accurately detect a scene change such as a change to a severe scene. When the scene change detection unit 2 detects a scene change, the scene change management unit 3 manages information such as the detected frame image and scene number.

On the other hand, the characteristic frame detection unit 4 matches the digital moving image (continuous frame images) with the shape pattern stored in the material management unit 5, and detects a characteristic frame that can be used as an index image.

The scene change management unit 3 manages the index images detected by the scene change detection unit 2 and the characteristic frame detection unit 4 for each scene, and outputs the index image information obtained thereby to the index display unit 6. It is displayed on the edit screen and is recorded in the recording unit 9.

Here, for example, as shown in FIG.
It is assumed that the index image is detected. That is,
Three scenes (scenes A, B, C) are detected and
Ten index images from the scene (Index # 1 ~ In
dex # 10) is detected. Note that Index # 1 is the index image detected in the scene change portion of scene A, and similarly Index # 2 is the scene B and Index #.
Reference numeral 3 is an index image of scene C. Also, Index #
4 to Index # 6 are index images in the scene A, Index # 7 to Index # 8 are scene B, Index # 9 to Index #.
10 is included in scene C. Here, details of the operation for scene change management in the present embodiment will be described with reference to FIG.
This will be described with reference to the flowchart shown in FIG.

First, the scene change detection unit 2 reads the k-th frame image from the digital moving image input unit 1 (step S1) and detects the scene change for this frame image (step S2).

As a result, when the frame image is detected as the scene change portion (step S3), the scene change management unit 3 displays the index image relating to the n-1th scene held until then as an index. It is displayed on the section 6 (step S4). Further, the scene change management unit 3 updates the value of the parameter (steps S5 and S).
6). Here, the parameter is a scene number (scene n
o. ) N and a scene number (scene no.) M. Here, the value n of the scene number is updated by 1, and the in-scene number is set to m = 0. (Note that the initial value of n = 0). The scene change management unit 3 converts the head frame image of the nth scene A into a detected video index (step S7) and holds it as an index image Index # 1.

Next, if the processing is not completed up to the final frame (step S13), the next frame image is read (steps S14 and S1). In the scene change detection of step S2, when the read frame image does not correspond to the scene change portion (step S3),
The characteristic frame detection unit 4 detects whether the frame is a characteristic frame (step S8).

As a result, if the frame is a characteristic frame, the scene change management unit 3 updates the scene number attribute (scene no. Attribute) n, and the scene number value m (step S).
10, S11). That is, the detected frame image is converted into a detected video index, and this index image is held as the m-th index in the n-th scene (step S12).

By repeating the above processing,
By the time the next (n + 1) th scene B is detected, the index image Index # 1 with the scene number 0 and the index images with the scene numbers 1 to 3 that are the representative index images of the scene are included in the scene A with the scene number 1. Index # 4 ~ Index #
6 is extracted.

As a result, in the case of a video composed of a plurality of scenes as shown in FIG. 2, Index # 4 to Index # 6 are In
Index # 7 to Index # 8, which have dex # 1 as a representative index image of scene A (scene number 1), are Inde
Index # 9 to Index # 10 are groups that have x # 2 as a representative index image of scene B (scene number 2)
Each group is managed as a group having 3 as a representative index image of scene C (scene number 3).

Next, an example of a mode in which the index image managed for each scene is displayed on the edit screen of the index display unit 6 will be described. First, regarding the first embodiment, FIG.
~ It demonstrates using FIG.

FIG. 4 shows an index image obtained for each scene as an index image obtained from the first frame for each scene, for example, Index # 1 for scene A as a representative index image, and below that from within scene A. An example of displaying the obtained index image over time (along the time axis) is shown.

With this display, the rough flow of the video can be grasped by following the representative index images Index # 1 to Index # 3, and the flow in the scene is connected to the index images in each scene. You can figure it out by following.

At the first display stage as shown in FIG. 4, for example, in the scene A, the index image In from the head frame of the scene is used as a representative index image.
dex # 1 is assigned. This representative index image can be changed to another index image in the same scene.

At this time, an instruction for changing the representative index image, designation of a representative index image to be changed, designation of an index image to be a new representative index image, and the like are input from the operation unit 8. The operation control unit 7 changes the display screen, the relationship between the index images in the storage unit 9, and the like according to the input from the operation unit 8.

As a result, the representative index image is changed, for example, as shown in FIG. In FIG. 5, with respect to the scene A, Index # 5 is newly selected as the representative index image and is displayed by being replaced with the previous Index # 1. And Index # 1 is the other two index images In
It is rearranged and displayed along with dex # 4 and Index # 6 in the order of time. Similarly, for scene B, Index #
8 is newly selected as the representative index image and the display is changed.

Further, as shown in FIG. 5, a new representative index image is selected, and unnecessary index images such as the index image of the first frame that is no longer needed can be deleted.

In this case as well, a deletion instruction, designation of a target index image, and the like are input from the operation unit 8.
In FIG. 6, since Index # 5 and Index # 8 are newly selected as the representative index images in FIG. 5, the previous Index # 1
, Shows a display example in which Index # 2 is deleted as an unnecessary index image.

Next, the second embodiment will be described with reference to FIGS. FIG. 7A shows a display example at the first display stage. In FIG. 7A, the index images obtained for each scene are displayed in a stacked shape for each scene. The index image displayed at the top is the representative index image for that scene.

From this edit screen, for example, when it is desired to know the contents of scene A, In displayed at the top of scene A
dex # 1 is designated by the cursor by operating the operation unit 8. In this case, the index display unit 6 is displayed as shown in FIG.
As shown in (b), the index images Index # 4 to Index # 6 extracted in the scene A are displayed along the time axis.
Here, when Index # 1 is instructed again, the display returns to the grouped display for each scene shown in FIG.

FIG. 8 shows a display example when the representative index image is changed. When Index # 5 is selected as a new index image in the state of being displayed as shown in FIG. 7B, the representative index is replaced with the previous Index # 1 as shown in FIG. 8A. Change and display the image.
And Index # 1 is the other two index images Index #
4, along with Index # 6, are rearranged and displayed in chronological order. When Index # 5 is selected in this display state, as shown in FIG. 8B, the index images are displayed together for each scene.

FIG. 9 shows a display example when the index image is deleted. For example, in the edit screen shown in FIG.
dex # 5 is newly selected as the representative index image,
If the previous Index # 1 is no longer needed, use the operation unit 8
When the deletion of dex # 1 is instructed, the index image Index # 1 is changed to the deleted display as shown in FIG. 9A. When Index # 5 is selected in this display state,
As shown in (b), the index images are displayed together for each scene. As shown in FIG. 9A, scene A
The index image included in is displayed in a form in which it is changed into three images.

In this way, when editing a video composed of a plurality of continuous scenes, various index images obtained from the video are displayed together in association with each other. The semantic connection of images in a scene becomes clear, and it becomes easy to grasp the overall flow. Also, by efficiently arranging the representative index image representing the scene obtained from the video on the editing screen, it is possible to easily grasp the content of the entire video.

[0045]

As described above, according to the present invention, by displaying a plurality of index images obtained from a video composed of a plurality of continuous scenes for each scene on the editing screen, the index can be displayed. The required scene can be easily searched by following the image. Further, the boundary between the connection between the index images and the connection between the scenes can be easily distinguished, and the content of the video can be sufficiently grasped from the display screen.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an index image display device according to an embodiment of the present invention.

FIG. 2 is a diagram showing an example of a video composed of a plurality of scenes.

FIG. 3 is a flowchart for explaining details of an operation for scene change management in the present embodiment.

FIG. 4 is a diagram showing an example of an edit screen according to the first mode in the present embodiment.

FIG. 5 is a diagram for explaining an operation of changing the representative index image on the editing screen according to the first embodiment in the present embodiment.

FIG. 6 is a diagram for explaining an operation of deleting an index image on the edit screen according to the first mode in the present embodiment.

FIG. 7 is a diagram showing an example of an edit screen according to a second mode in the present embodiment.

FIG. 8 is a diagram for explaining an operation of changing the representative index image on the edit screen according to the second mode in the present embodiment.

FIG. 9 is a diagram for explaining an operation of deleting an index image on the edit screen according to the second mode in the present embodiment.

[Explanation of symbols]

1 ... Digital moving image input unit, 2 ... Scene change detection unit, 3
... Scene change management unit, 4 ... Characteristic frame detection unit, 5 ... Material management unit, 6 ... Index display unit, 7 ... Operation control unit,
8 ... Operation part, 9 ... Recording part.

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁵ Identification code Office reference number FI technical display location H04N 7/18 U

Claims (4)

[Claims] 1. A form in which one or more index images per scene are extracted from a video including a plurality of scenes, and the index images represent the relationship between index images included in each scene for each scene. A video index display device characterized by displaying. 2. A management means for dividing and managing a video composed of a plurality of consecutive scenes in scene units by detecting a frame image corresponding to a scene change from the video, and a characteristic feature in each scene. The video index display device according to claim 1, further comprising: a detection unit that detects a frame image as an index image. 3. A scene change detecting means for detecting a frame image corresponding to a scene change as an index image from a digital moving image composed of continuous frame images, and a characteristic frame image as an index image from the digital moving image. Characteristic frame detecting means, an index image detected by the scene change detecting means, and a scene change managing means for managing the index image detected by the characteristic frame detecting means for each scene, and the scene change managing means. An image index display device comprising: an index display unit that displays the index image in a form that expresses a relationship between managed index images. 4. The index display means, with respect to a representative index image for each scene, which is a representative index image in the scene, arranges the index images included in each scene in the order of appearance in the digital moving image. 4. The video index display device according to claim 3, wherein the video index display device displays the video index in a subordinate form.