JPH11266449A - Video structural processing device and recording medium storing program for video processing - Google Patents

Abstract

PROBLEM TO BE SOLVED: To apply structural processing to various video images by combining basic processing systems from basis processing groups required for applying structural processing to a video image so as to establish a series of processing systems to realize the structural processing to the video image, charging only the procedure of combination of the processing systems. SOLUTION: The video image structural processing device is provided with a video class discrimination section 7 that designates a kind of a video image, a video image structural processing procedure storage section 3 that stores a structural processing procedure of a corresponding video image for each kind of the video image, a video image structural processing 9 that calls structural procedure corresponding to the class of the video image designated by the video class discrimination section 7 from the video image structural processing procedure storage, section 3 and that executes the video image structural processing, and a video image structural processing procedure storage section 4 that stores information from the video image structural processing 9. The optimum structural processing to each class of the video image is called from the video image structural processing procedure storage section 3, and the video image structural processing 9 executes the processing to obtain the optimum structural information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image structuring apparatus and a program recording medium therefor, and more particularly to a viewer selecting a desired image from a large amount of images in broadcasting such as satellite broadcasting and CATV. To watch the content in a timely manner or to grasp the entire content in a short time,
The present invention relates to a configuration of an apparatus for structuring an image.

[0002]

2. Description of the Related Art In recent years, the number of broadcast channels distributed to homes by satellite broadcasting, CATV, and the like has been steadily increasing. Also, with the development of the multimedia infrastructure in the future, it is expected that in the future, an environment for easily accessing a large amount of video information stored in various places will be prepared.

[0003] Now, when an environment has been established in which a large amount of video information can be viewed, the content of the video information is
It is expected that demand for grasping in a short time and selectively viewing a desired video will increase.

In order to meet the above demands, it is necessary to simplify access to video information. In this case, however, the video is structured in some way, and based on this, the video is reduced. There is a need for a mechanism for presenting in a reduced form or for selectively presenting a part of an image.

[0005] Techniques for reducing and displaying images are disclosed in Japanese Patent Application Laid-Open Nos. 03-214364 and 04-214364.
A cut detection method as disclosed in, for example, Japanese Patent No. 219878 has been proposed. Cut detection is a technique for automatically detecting a transition between images.

However, in order to reduce the image,
In the cut detection, the video can be divided into shots, but there is a problem that the shots cannot be linked to each other to structure the video.

Therefore, in order to access a large amount of video information, it is necessary to create a structuring procedure according to the purpose of the user for each type of video and each program.

[0008] However, the structuring of images depends on the types of images,
It is very difficult to construct structuring rules that vary widely depending on the purpose of the user and can be applied uniformly to all images. It is the reality that no definitive method has been established so far for performing structuring of different types and purposes and adapted to various user needs.

[0009] So far, for example, news programs,
Although individual case studies have been conducted on specific subjects such as sports programs, even if these are integrated, a completely different processing system is required for each subject. Also, a processing procedure is required.

[0010]

As described above, in the conventional video structuring device, different structuring is required for each different video.
It is not a realistic solution. In particular, if each structuring is done in a completely different processing system,
It is not realistic from the viewpoint of easily adapting to user needs, and at present, it is strongly desired to construct a more versatile processing system that can handle various types of video structuring.

Accordingly, the present invention solves the above-mentioned problems of the prior art, includes a basic processing group required for structuring an image, and combines these basic processing. By constructing a processing system that realizes the structuring of video, a video structuring apparatus that can perform structuring of various videos with the same processing system only by changing the procedure of the combination of processes The purpose is to provide.

[0012]

In order to achieve the above object, the present invention provides a designating means for designating an image type, and a procedure accumulating means for accumulating a video structuring procedure corresponding to each image type. A structuring procedure for calling a structuring procedure corresponding to the type of video specified by the specifying means from the procedure storage means, and performing video structuring; and structured information for storing information from the structuring means. And a storage unit.

[0013] Further, the present invention is a recording medium on which a program for structuring a video is recorded so as to be executable by a computer, wherein the program calls a pre-stored structuring procedure corresponding to a specified type of video. In addition, the present invention provides a recording medium characterized by including a procedure for structuring an image in the called structured procedure and storing the structured information.

[0014]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram of a video structuring apparatus according to an embodiment of the present invention.

In the figure, the function of each component is as follows.

The video data storage unit 1 stores various video data.

The input device 5 and the display device 11 constitute a terminal for interactively inputting and outputting information.

The video model script storage unit 2 stores a video model script corresponding to each type of video stored in the video data storage unit 1.

The video model script processing / editing section 6 interactively processes and edits a video model script via the display device 11 and the input device 5, and stores the processed / edited video model script in the video model script storage section 2.

The video type determination unit 7 is a video data storage unit 1
The type of the video data stored in the video data is determined, and a video structuring procedure corresponding to the video data is specified.

The video structuring procedure generator 8 generates a video structuring procedure based on the video model script stored in the video model script storage 2 for each type of video, and stores the video structuring procedure storage 3 To be stored.

The video structuring unit 9 includes a video type determining unit 7
The video structuring procedure specified by the above is called from the video structuring procedure storage unit 3, the video is structured, and the obtained video structuring information is stored in the video structuring information storage unit 4. The video structuring information can be displayed on the display device 11.

The video structuring information processing / editing unit 10
Based on the video structuring information stored in the video structuring information storage unit 4, the video structuring information is interactively processed and edited via the input device 5 while displaying the video on the display device 11.

The above-mentioned video data storage unit 1, video model script storage unit 2, video structuring procedure storage unit 3, video structuring information storage unit 4, input device 5, video model script processing / editing unit 6, video The type determination unit 7, the video structuring procedure generation unit 8, the video structuring processing unit 9, the video structuring information processing and editing unit 10, and the display device 11 are connected to a bus 12 that integrates all of them.

The operation of the above configuration will now be described.

In the present embodiment, the following definitions are applied.

A transition between images is called a cut, and an image in a section sandwiched between cuts is called a shot.

An image structure is represented by nodes. Each node has constituent elements and attribute information.

The components of a node include a node and a series of video sections.

The attribute information of a node includes a node name, a representative frame, or a representative frame group.

In the present embodiment, the structuring does not always adopt a strict tree structure, and a lower node is allowed to include a higher node as a constituent element. It is assumed that the structured video has one upper node.

The video model script describes a video structure model. Then, a video structuring procedure is generated from the video model script. The video model script includes a description of an index and a description of a node.

The index is a frame or a shot that is a clue for segmenting a video into nodes. For the description of the index, for example, various expressions having different levels of abstraction are used, such as using the whole or a part of the image as it is or describing the image using a repeated shot pattern. In the present embodiment, it is also possible to use the whole or a part of the image, information, and character information appearing in the image.

A node is described by a series of video sections specified based on an index given to a video and a combination of nodes.

The structuring of an image is performed according to the procedure shown in the flowchart of FIG.

In the pre-processing routine 13, cut detection processing S1 and shot clustering processing S2 are performed on the video data stored in the video data storage unit 1. These processes are performed in the video model script processing / editing unit 6.

First, the video is divided into shots by detecting a transition of the video in the cut detection processing S1.

Subsequently, a shot clustering process S2 is performed according to the similarity of the images. For this processing, for example, the method described in Japanese Patent Application Laid-Open No. 9-270006 by the present applicant is applied. Due to the clustering of shots, similar shots are given the same label.

Following the processing of the preprocessing routine 13, the processing of the video structuring processing routine 14 is started. Here, the video structuring procedure generation processing S3, the index assignment processing S4, the node creation processing S5, the structuring information recording Processing S
6 is performed.

First, in the video structuring procedure generation process S3, the type of the video is determined automatically or manually through the process of the video type determination unit 7, and the video model script for the video is stored in the video model script storage unit. 2 and generates a video structuring procedure from the video model script through the video structuring procedure generation unit 8. The generated video structuring procedure is stored in the video structuring procedure storage unit 3.

In the following indexing process S4,
Frames or shots that match the repeating pattern of images or shots specified by the video structuring procedure,
Assign an index.

Then, in the node creation processing S5, following the assignment of the index, in accordance with the video structuring procedure,
A node is extracted from the video, and a representative frame or a representative frame group is selected for each node.

The above-described index assignment processing S4 and node creation processing S5 are executed through the processing in the video structuring processing 9.

Finally, in the structured information recording process S6, the structured information obtained as described above is stored and recorded in the video structured information storage unit 4.

The display device 11 displays the video in a structured manner based on the structured information, and interactively with the input device 5
The video desired by the user is sequentially presented.

Next, the flow of processing in the display device 11 will be described.

The display is started from the top node, and the screen displays the name of the top node and a representative frame (group). The user selects a representative frame, plays the video of the node, expands the node,
The expanded node can be restored. When a node is expanded, all nodes included in the node are displayed in a list by name and representative frame or representative frame group.

Hereinafter, by repeating the same procedure,
The user can grasp the content of the video in a short time and selectively view only the desired video.

It should be noted that the structuring process by the video structuring unit 9 alone may cause a case where the structuring result of the video is not in the form desired by the user. Therefore, in the present embodiment, the video structuring information processing / editing unit 10 is provided for the user to interactively change, add, or delete nodes on the screen via the input device. The structure is such that the demands of the user can be reflected.

Hereinafter, taking a video of a news program as an example,
A specific structuring procedure will be described.

For a series of videos, first, cut detection processing and shot clustering are performed.

FIG. 3 shows a news image subjected to cut detection processing, and one frame is arranged for each shot. The symbol at the upper left of the frame indicates the result of the clustering of the shot, and includes the alphabet a,
.. b, c... are shot classes, numbers 1, 2, 3,.
* Indicates the order of appearance.

In the news video, the shot with the highest appearance frequency is considered to be a caster shot. Therefore, the most repeated shot is indexed as a caster shot (b1, b
2, b3 ...).

From the shot of the caster to the shot immediately before the shot of the next caster, the nodes (b1, c)
1, d1, e1, etc.). Among the nodes thus formed, a node containing the largest number of shots similar to shots in other nodes is set as a headline, and other nodes are set as topics.

A node having a shot similar to the shot in the headline node among the topics nodes is added to the headline node.

Next, the opening up to the first caster shot is defined as opening, and the shot after the last caster shot is defined as ending.

Subsequently, all the nodes obtained by the above processing are put together into one node, which is set as the highest node.

Finally, one or a plurality of representative frames are selected from shots or shot groups included in each node, and are set as a representative frame or a representative frame group of the node.

FIG. 4 is an example of a video model script representing the structure of the news video described above. The video structuring procedure generation unit 8 of FIG. 1 performs the following processing according to the description of FIG.

In the indexing process S41, first, in the description of the index definition S42, the caster video is index-defined as the shot that appears most frequently.

In the subsequent node processing S43, first, in the description of the node definition S44, an opening from the first shot to the first caster shot is defined as a representative frame and defined as opening.

Subsequently, in the description of the node definition S45, the nodes from the shot of each caster to the shot of the next caster are defined as topics using the first frame of each shot as a representative frame.

Next, in the description of the node definition S46, the nodes from the shot of the last caster to the last shot are defined as the ending with the last frame of the last shot as the representative frame.

Subsequently, in the description of the node name change S47,
The topic in which the number of shots having links to other shots is the largest is defined as a headline.

Next, the node designated as the headline in the description of the addition processing S48 is added to the headline.

Finally, in the node definition S49, the top node is defined as news, which is defined as a headline, topics, and ending with opening as a representative frame.

According to the above description, a video structuring procedure as shown in FIG. 5 is performed, and video structuring processing 9 obtains video structuring information. Incidentally, the information of FIG. 5 includes the indexing S51 of the caster, the node definition S52 of the opening, and the node definition S of the topics.
53, an ending node definition S54, and a headline node definition S55.

As a result of the above processing, for example, the highest node is expanded as shown in the explanatory diagram of FIG. FIG.
Shows an example in which headlines, five topics, and endings are developed after opening.

In the above processing, the case where a repetitive pattern is used for indexing shots is exemplified. However, the case where indexing is performed using the whole or a part of an image is described.
This will be described below using a weather forecast image as an example.

In the case of the weather forecast, in the same program,
For example, an image having the same composition is used for each topic, such as “tomorrow's weather” and “weekly weather forecast”. Therefore, for each topic, the whole or a part of the image that can identify the topic is stored, and by using this,
By performing indexing, structuring can be easily performed.

FIG. 7 shows an example of a video model script representing the structure of a weather forecast video. The video structuring procedure generation unit 8 of FIG. 1 performs the following processing according to the description of FIG.

First, in the indexing process S71, first, in the description of the index definition S72, a shot including an image of today's weather forecast is index-defined as today's weather.

Subsequently, similarly, for example, a weekly weather image, a tomorrow's weather image, and the like are successively index-defined, and in the description of the index definition S73, a shot including the image of the world weather forecast is defined as: The same process is repeated until an index is defined as the weather of the world.

In the next node processing S74, in the description of the node definition S75, from every shot having an index to the next shot having an index,
Define as untitled.

Subsequently, in the description of the node definition S76, the nodes from the shot of each caster to the shot of the next caster are defined as topics using the first frame of each shot as a representative frame.

Next, in the description of the node name change S77, the index of the first shot is set as the node name of every node.

According to the above description, a video structuring procedure as shown in FIG. 8 is performed, and video structuring processing 9 acquires video structuring information. Incidentally, the information in FIG. 8 includes an indexing process S81 and a node process S82.
It consists of.

In the above-described image structuring, it is necessary to appropriately create an image model script for each image. However, it is not so easy to create a video model script that structures the video to the desired shape.

On the other hand, in this embodiment, by using the video structuring information processing / editing unit 10, first, a video model script serving as a model is created, and based on this, the video is structured, and the video is structured. By repeating the procedure of correcting the video model script with reference to the result, it is possible to relatively easily create a structural model script required to realize a desired structuring.

Each of the above-described procedures is performed by a computer as a program that can be executed by various recording media, for example, an optical disk such as a CDROM, a magnetic disk / medium such as a floppy disk, or a magneto-optical disk such as an MD. Etc. can also be recorded.

[0082]

As described above, the video structuring apparatus of the present invention can realize various video structuring by the same processing system by preparing a video structuring procedure for each video type. There is an effect that an environment in which the user can grasp the contents of the video in a short time and selectively view only the desired video can be easily constructed.

[Brief description of the drawings]

FIG. 1 is a block diagram of a video structuring apparatus according to an embodiment of the present invention.

FIG. 2 is a flowchart for explaining the operation of the configuration of FIG. 1;

FIG. 3 is an explanatory diagram in which cut detection processing is performed on a news video, and one frame is arranged for each shot.

FIG. 4 is an explanatory diagram showing an example of a video model script representing the structure of a news video.

FIG. 5 is an explanatory diagram showing an example of information of a video structuring procedure based on the description in FIG. 4;

FIG. 6 is an explanatory diagram showing an example of expanding a top node obtained as a result of the processing shown in FIGS. 4 and 5;

FIG. 7 is an explanatory diagram showing an example of a video model script representing a structure of a weather forecast video.

FIG. 8 is an explanatory diagram showing an example of information of a video structuring procedure based on the description of FIG. 7;

[Explanation of symbols]

 Reference Signs List 1 video data storage unit 2 video model script storage unit 3 video structuring procedure storage unit 4 video structuring information storage unit 5 input device 6 video model script processing / editing unit 7 video type determination unit 8 video structuring procedure generation unit 9 video structure Processing 10 Video structured information processing and editing unit 11 Display device 12 Bus

────────────────────────────────────────────────── ─── Continued on the front page (51) Int.Cl. ⁶ Identification symbol FI H04N 5/91 N (72) Inventor Tsune Aoki 1 Komukai Toshiba-cho, Saiwai-ku, Kawasaki-shi, Kanagawa Prefecture Toshiba R & D Center Co., Ltd.

Claims (6)

[Claims] 1. A designating means for designating a type of video, a procedure storing means for storing a structuring procedure of a video corresponding to each type of video, and a structuring corresponding to the type of video specified by the designating means An image structuring apparatus, comprising: a structuring unit that calls a procedure from the procedure storing unit and implements video structuring; and a structured information storing unit that stores information from the structuring unit. 2. Display means for displaying a video based on the video structured information stored by said structured information storage means.
The video structuring device according to claim 1, further comprising: 3. The video structuring apparatus according to claim 2, further comprising: means for storing a video model script describing a video structure model; and a procedure generation unit for generating a video structuring means based on the video model script. . 4. The video structuring apparatus according to claim 3, further comprising video model script processing / editing means for processing / editing the video model script. 5. The video structure according to claim 2, further comprising video structure information processing and editing means for interactively processing and editing the structured information obtained by said structuring means while displaying said structured information on said display means. Device. 6. A recording medium on which a program for structuring an image is recorded in a computer-executable manner, wherein the program calls a pre-stored structuring procedure corresponding to a specified type of image. A recording medium characterized by including a procedure for also structuring an image in the called structured procedure and storing the structured information.