JP2002271739A - Video device and reproduction control information distribution method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video device that can retrieve a desired video image in a short time. SOLUTION: A digest video scenario table generating section 104 generates a digest video image scenario to identify a frame to be included in a digest video image on the basis of video priority denoting an importance degree by each frame. The generating section 104 generates a digest video scenario by each program and a digest video image scenario table storage section 105 stores the scenario. When a user designates a plurality of programs to be reproduced, a multi-program scenario table generating section 110 combines corresponding digest video scenarios so as to be contained within a prescribed time and provides an output of the result to a video display control section 107. The video display control section 107 allows a video reproduction section 108 to reproduce the video image based on the received digest video scenario.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video apparatus and a reproduction control information distribution method for performing display using a summary video created from video data.

[0002]

2. Description of the Related Art There are many sources of video data from professional use to general consumer use. It takes a lot of time and effort to find what you want to watch from these large amounts of video data. For example, in fast-forward playback of a video tape recorder, finding a program to watch requires more time than expected.

To cope with this problem, there have been proposed methods and apparatuses for creating a summary video and grasping the contents of the rough video through reproduction of the summary video. These methods and apparatuses are described in, for example, JP-A-10-32773, JP-A-11-239322, and JP-A-11-1760.
No. 38, for example.

As disclosed in Japanese Patent Application Laid-Open No. 10-32773, a method of roughly grasping the contents of a video can be roughly classified into two methods. One is a method in which video information having a property of being continuous in time series is developed and displayed in space. For example, this corresponds to a case where one screen is divided and a plurality of important frames are displayed as still images on a plurality of divided screens.

[0005] Another method is to display a video in a time series without spatially developing it. The fast forward reproduction described above is the most primitive example.

An example of the latter technique for displaying important frames in time series is disclosed in Japanese Patent Application Laid-Open No. H11-176.
As disclosed in Japanese Patent Publication No. 038, only key frames having high importance are displayed in a fast-forward manner, and when an operation of selecting any key frame is performed during viewing, normal reproduction is started from a position corresponding to the key frame. A way to enter is proposed.

[0007]

The key frame is:
It is obtained by extracting a frame of high importance from video data.

In some cases, a human may manually select from video data to designate a key frame, but in general, when managing a large amount of video data, the key frame is specified by an automatic method. I have to rely on.

When a key frame is automatically designated, there is a problem in that the number of frames determined to have high importance differs for each video source. As a result, in fast-forward playback of a key frame, it is difficult to keep the playback time of the summary video within a fixed time length desired by the user.

Therefore, when displaying a summary video, it is necessary from the user's standpoint to absorb the difference between the video sources and perform the reproduction with a certain fixed time length.

[0011] In addition, after viewing a certain summary video, there may be a demand that the user wants to view a summary video having a longer time to understand even more detailed contents in order to determine whether the user is a desired video source. Therefore, it is necessary to create a plurality of summary videos having a specified time length from one video source.

An object of the present invention is to provide a video apparatus and a reproduction control information distribution method that enable a user to reach a desired video in a short time through reproduction of a summary video.

[0013]

According to the present invention, there is provided a video apparatus, comprising: a search means for searching for a key frame having a predetermined priority in order from the head of a video sequence including a plurality of frames arranged in time series; A frame set that is a plurality of frames that are successively arranged in a time sequence in a video sequence and that include a selected key frame, and that are a plurality of frames corresponding to a predetermined continuous playback time, is used as a summary video. Selecting means for selecting, determining whether or not the search has been completed to the end of the video sequence, determining means for reducing the priority when the search has been completed to the end of the video sequence and causing the search means to search again, When the total playback time of the summary video exceeds a predetermined total playback time, there is provided time determination means for completing the creation of the summary video.

[0014] Preferably, the video apparatus further includes a priority accumulating means for accumulating the priority corresponding to each frame, which is referred to by the searching means, and the selecting means stores information for specifying the frame selected as the summary video. Output scenario including

Preferably, the video apparatus reads scenario corresponding to a desired video sequence from the scenario storage means in response to an external request, and stores the scenario in a network. And a scenario sending unit for outputting.

Preferably, the video apparatus according to claim 1, wherein the frame set includes a key frame at the beginning of a time series.

According to another aspect of the present invention, a video apparatus stores video information storing means for storing a video sequence including a plurality of frames, user input means for receiving an instruction from a viewer, and a summary corresponding to the video sequence. Video display control means for controlling the display of the summary video in accordance with the video playback control information and the output of the user input means, wherein the playback control information corresponds to each of the plurality of summary videos having different total playback times. Including a scenario of a plurality of layers including information for specifying a frame to be used for a summary video from a video sequence, reading data corresponding to the frame from the video information storage means in accordance with an output of the video display control means, and reproducing the data into a video signal The apparatus further includes a video playback unit.

Preferably, in the video apparatus, the user input means includes: first input means for inputting an instruction to reproduce a summary video including more detailed contents than the currently reproduced summary video; Inputting an instruction to reproduce a summary video including contents coarser than the summary video of the second.
In the scenario of a plurality of layers, a lower layer corresponds to a summary image representing more detailed contents, and the video display control means responds to an input from the first input means,
The scenario to be used as the reference for controlling the display of the video sequence is changed from the currently used scenario to the scenario one level higher, and is used as the reference for controlling the display of the video sequence in response to an input from the second input means. Scenario,
Change the scenario currently in use to the scenario one level lower.

Preferably, when the scenario is changed, the image display control means sets the video sequence to a frame specified by the changed scenario and which is currently being reproduced by the scenario before the change. Is controlled so that the reproduction is continued from the frame whose position on the time series is close.

Preferably, the video device further comprises a receiving means for receiving the reproduction control information through the network and outputting the information to the video display control means.

According to still another aspect of the present invention, there is provided a reproduction control information distribution method, wherein a step of searching for a key frame having a predetermined priority in order from the beginning of a video sequence including a plurality of frames arranged in time series. And a plurality of frames that are sequentially arranged in time series in the video sequence and include the selected key frame, and
Selecting a frame set, which is a plurality of frames corresponding to a predetermined continuous reproduction time, as a summary video, and determining whether or not the search has been completed to the end of the video sequence, and when the search has been completed to the end of the video sequence. To reduce the priority and search again from the beginning of the video sequence, and the total playback time of the summary video is:
When the predetermined total playback time has been exceeded, completing the creation of the summary video, outputting a scenario that is information specifying a frame used for the summary video from the video sequence, and saving the scenario to a recording medium. Reading the playback control information from the recording medium in response to a request from the user terminal and delivering the playback control information to the user terminal.

[0022]

Embodiments of the present invention will be described below in detail with reference to the drawings. In the drawings, the same reference numerals indicate the same or corresponding parts.

[Explanation of Overall Configuration] FIG. 1 is a block diagram showing a configuration of a video apparatus 10 using a summary video according to an embodiment of the present invention.

Referring to FIG. 1, a video apparatus 10 receives and records a program such as a television program obtained from a communication means such as a broadcast or the Internet, or a video taken by a user with a camera or the like. When a viewer searches for a video desired to be reproduced and gives a reproduction instruction, the recorded original video corresponding to the reproduced video is reproduced. .

The video device 10 has a video input unit 100 for inputting program data, a video storage unit 101 for storing input data, and a video included in the input data. And a video priority information storage unit 103 that stores the video priority created by the video priority determination unit 102. Note that the video priority is information for specifying a portion that is considered to be important for simply expressing the content from the input video.

The video apparatus 10 further includes a summary video scenario table creation unit 104 that determines which part of the input video is to be a summary video based on the video priority created by the video priority determination unit 102. A summary video scenario table storage unit 105 that stores a summary video scenario table output by the summary video scenario table creation unit 104
Reading a summary video scenario table corresponding to a plurality of programs from the summary video scenario table storage unit 105, selecting necessary information, and generating a multi-program scenario table in which a plurality of programs are linked to each other Unit 110.

The video device 10 further includes a user input unit 106 for inputting a video to be viewed by the user while viewing the summary video, and a video based on a multi-program scenario table in accordance with the user's input. A video display control unit 107 for controlling reading and stopping of the video, a video reproduction unit 108 for reading a video from the video storage unit 101 based on an instruction from the video display control unit 107, performing necessary processing, and reproducing the video, and a video reproduction And a video display unit 109 for displaying the video data processed by the unit 108. For example, when the data is compressed by the MPEG method or the like, the processing performed by the video reproduction unit 108 includes a decoding process and the like.

When a user requests digest viewing of a plurality of programs, the request is sent to the video display control unit 1.
07 to the multi-program scenario table creation unit 110. Then, the scenario table of each program is read from the summary video scenario table storage unit 105, and the multi-program scenario table creation unit 11 is read.
0 creates a multi-program scenario table.

[Explanation of Video Data] In this specification, FIG.
A unit of an image obtained from a communication means such as a television broadcast or the Internet or an image captured by a user with a camera or the like, which is input to the image device, is referred to as a program. For example, if the input data is a TV program,
One television program corresponds to one program. Further, in the case of a video taken by a user with a camera or the like, for example, a graduation ceremony video, an athletic meet video, and the like correspond to the respective programs. In many cases, audio information is added to video, and in this case, the program includes not only a video sequence as video data but also an audio sequence as audio data.

FIG. 2 is a diagram showing how the video data included in the program in this specification is structured.

Referring to FIG. 2, video sequence 200
Is an image group included in the one program described above. Video sequence 200 includes a plurality of scenes, for example, scenes 211 and 212.

Here, a scene refers to a group of images having the same semantic content. For example, a video sequence corresponding to a program called one news program includes one scene called news of earthquake information and another scene called weather forecast. And are included. One scene includes a plurality of shots.

Here, a shot refers to a group of physically continuous images. For example, an image in which a human moves in the background with respect to the same background is one shot. A boundary between shots is called a cut. For example, scene 211 includes shots 221-225. In other words, the scene 211 is cut 226-2
29 divides shots 221 to 225.

Each shot is composed of a corresponding frame. For example, the shot 221 has a plurality of frames 23
Contains 0. Here, a frame refers to one piece of image data displayed on a screen.

[Explanation of Video Priority] FIG. 3 is a block diagram showing the configuration of the video priority determination unit 102 in FIG.

Referring to FIG. 3, video priority determining section 102
Is composed of an input unit 300 that receives data stored from the video storage unit 101 in FIG. 1, and a video analysis unit 301 that receives data from the input unit 300, derives a feature amount included in video information, and performs analysis. Including. When the program includes audio information and additional data, such as a television program, the video analysis unit 301 also analyzes them.

The purpose of the analysis performed by the video analyzing unit 301 is as follows.
The purpose is to find the priority of each frame when creating a summary video, and to perform cut detection and scene detection as necessary. Various proposals have been made for these analysis methods, for example, "Structure and Retrieval of Information, by Nishio et al., Iwanami Shoten, March 27, 2000, 157-159.
Page ".

As a method of cut detection necessary for specifying a shot, for example, there is a method of obtaining a difference in gray value of a corresponding pixel between two temporally adjacent frames.

The scene is extracted by taking into account the continuity of these information, such as video information such as color, audio information such as BGM, audio information spoken by humans, and important words included in text information such as telops. Can be considered.

The video priority determining unit 102 further receives a result of analysis by the video analyzing unit 301 to determine a final frame priority, and outputs an output of the frame priority determining unit 302. And an output unit 303 for outputting to the video priority information accumulation unit 103 in FIG.

FIG. 4 is a schematic diagram showing the priorities analyzed for each frame by the video priority determining section 102.

Referring to FIG. 4, the priority of each frame corresponds to a histogram 4 corresponding to a video sequence 400.
It is represented by 01. For example, a rule that determines that a frame is important if a face is included, a rule that is determined if a moving object is included, or a rule that if short shots are consecutive, those frames and the frame immediately after them are important The priority is determined by a plurality of defined rules. The video sequence 400 is analyzed according to such rules, and the priority is counted for each frame to obtain the histogram 401. Of the plurality of rules, a frame determined to conform to more rules can be considered to have a higher priority.

Here, the priority is derived for each frame. If a shot is specified, for example, the first frame in one shot is set as a representative frame, and the priority of only this representative frame is calculated. By setting the priority for the entire shot, the amount of calculation can be reduced.

When the shots, cuts, and scenes are specified by the video analysis unit 301 in FIG. 3, for example, it is possible to view the summary video in scene units. It is also possible to obtain the priority of a scene based on the priority of each frame, and this can be used when viewing a summary video.

[Explanation of Summary Video Creation] FIGS. 5 and 6 are flowcharts showing the operation of the summary video scenario table creation unit 104 in FIG.

Referring to FIG. 5, before the processing is started in step S500, summary video scenario table creating section 104 holds the total playback time of the summary video and the continuous playback time as parameters in advance. It is assumed that

Here, the total playback time is a parameter indicating how long the user wants to put the summary video as a whole. In addition, the continuous playback time is a parameter indicating how many frames are to be played continuously starting from the key frame in order to recognize an extracted key frame having a high priority as a moving image.

In this description, it is assumed that the video priority information is accompanied by an index number for specifying a shot.

First, the video priority information storage unit 103 shown in FIG.
Is input to the summary video scenario table creation unit 104, and the process is started in step S500.

In step S501, the summary video scenario table creation unit 104 initializes an index number I indicating a shot to be processed from now on to zero. Further, the summary video scenario table creation unit 104 sets the variable W indicating the priority of the shot to be processed from now on to the highest priority.

Next, in step S502, the summary video scenario table creation unit 104 determines whether the priority of the shot number I is equal to the priority W currently being processed. The priority of the shot number I is determined by the priority of the frame included in the shot. The priority of the frame having the highest priority may be set as the priority of the shot, or the priority of the frame located at the beginning of the shot may be set as the priority of the shot for simplicity of calculation.

If the priority of the shot number I is equal to the priority W currently being processed, the flow advances to step S503 to mark the representative frame of the shot I as a key frame, and via the connector A to step S504 in FIG. Proceed to.

At this time, the summary video scenario table creation unit 104 sets the variable J for storing the index number of the frame uniquely allocated from the head of the video to the number of the representative frame of the shot I.

If the priorities are not equal in step S502, the flow advances to step S512. Step S512
Then, the shot number I is incremented. Then, the process advances to step S509 to determine whether or not the last shot in the program currently being processed has been processed. If the processing has been performed up to the last shot, step S510
Then, the shot number I is initialized to 0, and the priority W is reduced by one. Priority W in step S510
Is decreased, the process proceeds to step S502 again.

If the last shot has not been reached in step S509, step S510 is skipped and the process proceeds to step S502 again.

That is, steps S512, S509, S
In 510, when the search is performed up to the last shot of the program, a process of reducing the priority by 1 is performed.

Referring to FIG. 6, in step S504,
The index J of the frame number is incremented. Subsequently, in step S505, it is determined whether the frame currently being processed is the head of the shot or whether the index J has been incremented to the last frame.

The case where the frame corresponds to the head of the shot is the case where the index J is incremented and the position of the frame currently being processed enters the shot next to the shot to which the key frame belongs. . If the shots are different, for example, a frame of a sports program and a frame of a commercial may be included during continuous playback time. In this case, since the video is not smooth even during the continuous reproduction time, the mark of the frame is interrupted.

When the index J is incremented to the last frame of the program, it is impossible to mark a subsequent frame.

Therefore, when these conditions are satisfied, the flow advances to step S508. On the other hand, if the condition is not satisfied, the process proceeds to step S506, and it is determined whether or not the total playback time of the frame marked continuously with a certain key frame at the beginning exceeds the continuous playback time. If the total playback time of the marked frames exceeds the continuous playback time, the process proceeds to step S508.

On the other hand, if not exceeded, step S
Proceeding to 507, frame J is marked as an additional frame to the already marked key frame without consideration of priority.

That is, by the processing of steps S505 to S507, after an important key frame, several frames specified as a continuous reproduction time are continuously reproduced to maintain smoothness as a video. If a cut exists between these several frames, continuous reproduction is terminated by making a determination in step S505.

In step S508, step S508
It is evaluated whether or not the sum of the reproduction times of the key frames and the additional frames marked in step 503 and step S507 exceeds a predetermined total reproduction time of the summary video.

If the predetermined total playback time has elapsed, the flow advances to step S511 to complete the creation of the summary video scenario table. If not, step S50
Proceed again to 2 and repeat the loop.

FIG. 7 is a diagram for explaining that the summary video scenario table is created according to the flowcharts shown in FIGS.

Prior to this processing, it is assumed that the video priority determining unit 102 has previously performed processing for setting the priority of the frame with the highest priority in each shot to the priority of that shot.

Referring to FIG. 7, a case where shots 1 to 10 are displayed in time series in one video sequence will be described. Priority is given to each shot,
It is assumed that the priority has 10 levels, with 10 being the highest priority. For each shot, shot 1 has priority 2,
Shot 2 has priority 3, shot 3 has priority 4, and shot 4 has priority 10.

Furthermore, shot 5 has a priority of 9, shot 6 has a priority of 8, shot 7 has a priority of 3, and shot 8 has a priority of 10. Also shot 9
Has priority 9 and shot 10 has priority 7.

In FIG. 7, one circle represents a plurality of frames corresponding to one second of reproduction time. The plurality of frames are called a frame set.

Hereinafter, the process of selecting frames for the summary video will be described. A case where a frame set for a total reproduction time of 10 seconds is selected as a summary video from the frame sets included in shots 1 to 10 will be described.
5 and 6 when such data is given.
An example of how the process is performed will be described with reference to the flowchart of FIG.

First, in step S501, the priority W is set to 10, which is the highest priority, in the subsequent processing. Shots 1-3 have a priority lower than 10, so
It is skipped in the loop of steps S502, S512, and S509.

Subsequently, when the shot number I becomes 4, since the shot 4 has the priority 10, the steps S502 and S502
At 503, the representative frame, which is the first frame of shot 4, is marked as a frame included in the summary video. Here, as can be seen from FIG. 7, the shot 4 also includes a frame set 552 following the frame set 551.

FIG. 8 is a diagram for explaining the structure of the frame sets 551 and 552. Referring to FIG. 8, frame 571 which is the first frame of frame set 551
Is the representative frame at the beginning of the shot, and
This is the key frame marked in step S503.
Note that a key frame is a frame that is marked first in a shot. As shown in FIG. 7, the priority of the entire shot is determined by the priority of the highest priority frame included in the shot, or the priority of the representative frame which is the first frame is determined. If so, the priorities of the frames included in the shot are all equal, so the key frame is a marked representative frame.

In order to recognize the key frame as a moving image, several frames following the key frame are automatically included in the digest. For example, a continuous reproduction time of 2 seconds is continuously included in the digest. Frames that are marked continuously with the key frame at the beginning in this manner are referred to as a continuous playback frame set.

Assuming that the frame rate is 30 frames / second, the number of frames for 2 seconds is 60 frames. Therefore, 59 frames following the key frame are selected as additional frames used for the summary video. In this case, the frame from the frame 571 to the frame 572 corresponding to the 60th frame is marked as being included in the digest in step S507.

Referring again to FIGS. 6 and 7, when a desired continuous reproduction frame set is marked for one shot,
The process proceeds from step S506 to step S508.

Subsequently, in step S508, it is determined whether or not the total reproduction time of the frames marked so far exceeds the total time of the desired summary video. At this time, the frame sets 551 and 552, that is, two seconds from the frames 571 to 572 in FIG. 8 are selected, and the desired total reproduction time is less than 10 seconds.
08 to step S502.

By the above processing, the frame set at the time when the first loop of FIG. 7 has been rotated is selected. At this point, the frame sets 551 and 552 have been selected.

Subsequently, the leading frame of the next shot of priority 10 is selected as a key frame. This frame is included in frame set 555.

As in the first loop, the key frames and the additional frames included in the two frame sets 555 and 556 are marked as frames included in the summary video. With these processes, the second loop ends.

The frame sets marked when the second loop ends are frame sets 551 and 5
52, 555 and 556. Even at this point, since the total playback time is less than 10 seconds, the process proceeds to the third loop.

Since there are no more shots of priority 10, the processing proceeds to the last shot 10, and the shot number I is initialized to 0 in step S510. Then, the priority is changed from 10 to 9.

In the third loop, the frame set 55
3 will be marked. Here, step S50
Although the frame number J is incremented at 4, the shot number 6, which is the next shot, is reached before the continuous reproduction time of 2 seconds is satisfied. Therefore, the process proceeds from step S505 to step S508. Even at this point, the total playback time is less than 10 seconds, so the loop is repeated. At the end of the third loop,
The selected framesets are 551, 552, 553,
555 and 556.

The loop is repeated in this manner. Finally, the frame set 558 is selected, and the total reproduction time is 10
For more than a second. At this time, from step S508 to step S
Proceeding to 511, the creation of the summary video scenario record ends.

The above processing is repeatedly executed to create a plurality of scenario records corresponding to a plurality of total reproduction times, for example, a scenario record for 30 seconds, 1 minute, etc.

FIG. 9 is a schematic diagram showing an example of the summary video scenario table created by the summary video scenario table creation section 104 of FIG.

Referring to FIG. 9, a summary video scenario table is created corresponding to original video 600. This summary video scenario table contains the scenario record 60
1,602,603. Scenario record 601,
602 and 603 indicate a total playback time of 30 seconds and 1
These are scenarios with different lengths corresponding to minutes and minutes.

The scenario record 601 includes continuous playback frame sets 601.1 and 601.1 whose continuous playback time is 3 seconds.
1.2,.

The scenario record 602 has a continuous reproduction frame set 602.
1, 602.2, 602.3, and so on.

The scenario record 603 has a continuous playback frame set 603.
1, 603.3, 603.4 and a continuous playback frame set 603.2 with a continuous playback time of 3 seconds.

The contents reproduced by the scenario record 601 include shot number 1, ie, 0 of the original video.
It contains information such as reproduction from minute 0 second to 3 seconds, and reproduction from shot number 29, that is, normal elapsed time 7 minutes 23 seconds for 3 seconds. This information may be the time information itself as shown in FIG. 9 or may be designated by a pointer such as a frame number.

Similarly, scenario records 602 and 603
However, a set of a normal elapsed time of a video and a continuous reproduction time from that position is described, and is expressed as a reproduction scenario of a summary video.

Such a scenario record can be sequentially obtained by giving the total reproduction time and the continuous reproduction time as parameters in the flowcharts shown in FIGS. Further, in FIG. 9, the total playback times are all different in time length, but a plurality of scenario records having different continuous playback times and the same time length may be included.

[Description of Video Reproduction] FIG. 10 is a display example for explaining the video display unit 109 in FIG.

Referring to FIG. 10, in this example, a PDA (Personal Digital Assembler) is used as one mode of video display unit 109.
(i.e., istant: personal portable information terminal) 700 shows an image being displayed.

The PDA 700 is provided with a touch panel 701 also serving as a display area and operation buttons 702 to 706. An original video and a summary video are displayed on the touch panel 701 according to a user's instruction. By operating the touch panel 701, the user can give an instruction to start reproduction and view a detailed summary video. In addition, the user can control the reproduction status using the operation buttons 702 to 706. For example, it is possible to switch from a summary video that requires a short playback time to coarse content, which has a long playback time but contains more detailed content, or to switch from a detailed summary video to a coarse summary video.

Touch panel 70 also serving as display area
1 corresponds to the image display unit 109 in FIG. Also, the touch panel 701 and the operation buttons 702 to 706
Corresponds to the user input unit 106.

Specifically, when the operation button 702 is pressed,
The PDA 700 returns from the video being played back to the top of the summary video, and restarts playback of the summary video with the shortest playback time.
When the operation button 703 is pressed, the PDA 700 returns to the previous shot of the currently reproduced summary video and resumes the reproduction. When the operation button 704 is pressed, the PDA 700 stops playing the video. When the operation button 705 is pressed, the PDA 7
00 reproduces the original video of the currently viewed summary video. When the operation button 706 is pressed, the PDA 700 shifts from the currently reproduced summary video to a coarser summary video.
The flow of these operations will be described in more detail using a flowchart.

FIGS. 11 and 12 are flowcharts for explaining the operation of the video display control unit 107 in FIG.

Referring to FIG. 11 and FIG. 12, step S8
At 00, the processing routine is started. FIG. 13 is a diagram for explaining the contents of the video stored in the video storage unit 101 of FIG. 1 at the start.

Referring to FIG. 13, image storage unit 10 shown in FIG.
It is assumed that a program 1, a program 2, a program 3,... For example, when the program is a television program, the program 1, the program 2, and the program 3 are recorded in a non-viewed state by scheduled recording or the like. Each program contains multiple shots. For example, program 1 has shots 1-3
Including multiple shots.

As described above, in the case where the video is stored in the video storage unit 101 in order from the program 1, the user specifies the program that he / she wants to view.

Referring again to FIG. 11, step S800
, A processing routine is started. In step S801, a multi-program scenario table for three layers is created from a program requested to be reproduced. This hierarchy indicates the level of detail of the summary video, and is called a digest level.

FIG. 14 is a conceptual diagram for explaining the creation of a multi-program scenario table.

Referring to FIG. 14, it is assumed that the scenario table shown in FIG. 9 corresponding to each program has already been created by summary video scenario table creating section 104 in FIG. Creating this scenario table
This is performed based on the flowcharts shown in FIGS.

The multi-program scenario table creating section 110 in FIG. 1 creates a multi-program scenario table by selecting and combining necessary scenario records from the scenario tables of programs that the user wants to view. The multi-program scenario table includes three levels of multi-program scenario records with digest levels of levels 1 to 3 from short to long playback time.

For example, the multi-program scenario table creation unit 110 determines that the total playback time is level 1 from the scenario table of program 1, the scenario table of program 2,...
The appropriate scenario records are selected and combined so as to match the playback time of the program, and a level 1 multi-program scenario record is created.

Also, multi-program scenario records of level 2 and level 3 that include a video that is more detailed than level 1 and have a long reproduction time are created in the same manner. Then, a multi-program scenario table including the multi-program scenario records of level 1 to level 3 is created.

In this way, a multi-program scenario table is created according to the user's viewing desire.

Referring again to FIGS. 11 and 12, in step S802, the video display control unit 107 waits until there is an instruction from the user input unit 106 to start reproduction. This instruction is performed by touching touch panel 701 in FIG.

Subsequently, in step S803, reproduction of the coarsest summary video, that is, a summary video requiring a short reproduction time is started. In the previous example, level 1 in FIG.
The video specified by the multi-program scenario record is reproduced.

Next, in step S804, the video display control unit 107 confirms whether or not the user has issued an instruction to select a summary video that is more detailed than the currently reproduced summary video. Proceed to S805
If no selection has been made, step S805 is skipped and the process proceeds to step S806. It should be noted that the user needs to
By touching the touch panel 701, the user can instruct to reproduce a more detailed summary video than the current layer.

In step S805, it is determined whether or not the video currently being reproduced is the most detailed summary video, that is, in FIG. 14, whether or not the digest level corresponds to level 3, and if not, the flow proceeds to step S809. . On the other hand, if it is the most detailed summary video, the process skips step S809 and proceeds to step S806.

In step S809, the reproduction is restarted from the same shot that is currently being reproduced, based on the more detailed summary video scenario in which the level in FIG. 14 is one level lower, and then the flow proceeds to step S806.

Next, in step S806, the video display control unit 107 confirms whether or not the user has given an instruction to select a summary video coarser than the currently reproduced summary video.
If a selection instruction has been made, the process proceeds to step S807. On the other hand, if the selection instruction has not been issued, step S807 is skipped and the process proceeds to step S812. In addition, the user
By touching the 0 operation button 706, it is possible to instruct the reproduction of the summary video whose level is one level higher than the currently reproduced summary video.

In step S807, it is determined whether or not the currently reproduced video is the coarsest summary video.
If the video is not the coarsest summary video, the process proceeds to step S810. If the video being reproduced is the coarsest summary video, the process skips step S810 and proceeds to step S812.

In step S810, the reproduction is restarted from the next shot closer to the same shot currently reproduced, based on the coarser summary video scenario corresponding to the next layer.

When the processing in step S810 ends, the flow advances to step S812. In step S812, it is determined whether or not the reproduction of the currently reproduced program has been completed. If the processing has been completed, the process proceeds to step S813. On the other hand, if not completed, step S813 is skipped and step S8
Proceed to 08.

In step S813, the process moves to the next program, and the shortest digest having the highest level is reproduced again.

When the processing in step S813 ends, the flow advances to step S808. In step S808, it is confirmed whether or not all data to be reproduced has been reproduced. If there is no data to be reproduced, the flow advances to step S811 to end reproduction.

In step S808, it is also checked whether or not a stop instruction has been given by the user. If there is a stop instruction, the process similarly proceeds to step S811 and ends.

Next, the reproducing operation will be described with reference to a flow chart using a simple schematic diagram.

FIG. 15 is a schematic diagram for explaining reproduction when digest levels are prepared for three layers.

Referring to FIG. 15, at digest level 1 where the reproduction time is the shortest, reproduction is started from point P0 which is the head of program 1. Here, it is assumed that a request that the user wants to view a more detailed digest is generated at the point PA.

Then, steps S804 and S80 in FIG.
5. The digest level of the reproduced video transitions from level 1 to level 2 via S809. Then, the digest having a longer reproduction time is reproduced.

Further, it is assumed that a similar request is made at point PB of shot 2 while the digest video of digest level 2 is being reproduced. Then, the digest level changes from level 2 to the most detailed level 3 through similar steps, and the reproduction is continued.

When the reproduction of the digest video at the digest level 3 advances to the point PC which is the end of the last shot 3a of the program 1, the reproduction of the current program 1 is completed, and the steps from the step S812 to the step S812 in FIG.
Through 813, the reproduction of the level 1 summary video, which is the shortest scenario of the program 2, is continued.

Next, a case where the operation button 706 of FIG. 10 is operated will be described. FIG. 16 is a schematic diagram for explaining an operation when the operation button 706 is pressed.

Referring to FIG. 16, it is assumed that shot 1 of a level 3 video, which is the most detailed digest of program 1, is being reproduced at the request of the user.

If the user presses the fast-forward button 706 in FIG. 10 at the point PD of the next shot 2 after the reproduction of the shot 1 and the shot 1a is completed, the digest level shifts from the level 3 to the level 2 and the next shot 3 Video playback is continued from the beginning.

Further, when the fast-forward button is pressed at point PE during the reproduction of shot 3 of level 2, the transition is made to digest level 1, which is the shortest digest, and reproduction is continued from the beginning of program 2.

[Operation Example in Network] In the present embodiment, it is assumed that the summary video scenario table is created in the video device. This example is conceivable in a normal television broadcast or the like as described above. On the other hand, when additional data (metadata) is transmitted in addition to video and audio data as in digital television broadcasting, a summary video scenario table may be included as part of this metadata. . In such a case, the video device serving as a receiver does not need the video priority determination unit 102, the video priority information storage unit 103, and the summary video scenario table creation unit 104 of FIG. The circuit configuration is simplified.

A so-called electronic program guide (EPG: Elec) displays a guide for selecting a TV broadcast program on a display screen.
trical Program Guide) has been put to practical use. The EPG is described in, for example, JP-A-8-275077. The EPG is a type of information about a television program supplied to the user from the outside.

As in the case of the EPG, if the distribution of the summary video scenario table corresponding to the television program is received from the outside, the video device on the user side can have a simpler configuration. For example, it is conceivable that information is supplied from a provider via a network such as the Internet. Such a provider will be referred to as an EPG provider.

FIG. 17 is a schematic diagram showing an example in which additional data of a video and a summary video scenario table are transmitted from an EPG provider.

Referring to FIG. 17, metadata including a summary video scenario table is transmitted from provider 900, while ordinary video and audio information is transmitted as broadcast waves from broadcast station 901 to the user.

[0137] The metadata and broadcast wave information are sent to a video device arranged in the user's home 902. In this case, the video device serving as a receiver does not need the video priority determination unit 102, the video priority information storage unit 103, and the summary video scenario table creation unit 104 of FIG. Becomes easier.

FIG. 18 is a block diagram showing a first system configuration example in a network operation example.

Referring to FIG. 18, a scenario table is transmitted from server 905 located at the EPG provider to user's video device 906 via network 908. In other words, the user's video device 906 plays a role as a kind of terminal that instructs data transmission and receives data. In addition, a video source is transmitted from the video transmission device 907 disposed at the television broadcasting station to the user's video device 906 via the network 909. Note that the network 909 is transmitted by radio waves from an antenna of a broadcasting station to an antenna of a user in a normal broadcast, but may be a wired one such as a cable television.

The server 905 includes a video input unit 910 for inputting a video for creating a scenario table, a video storage unit 911 for storing input data, and a video included in the input data. A video priority determining unit 912 for determining the priority, a video priority information storage unit 913 for storing the determined video priority, and a summary video scenario for creating a summary video scenario table based on the video priority information Table creator 914 and summary video scenario table storage 9 for storing the summary video scenario table output by summary video scenario table creator 914
15 and a scenario table sending unit 921. The scenario table sending unit 921 sends the scenario table to the network 908.

[0141] Video transmitting device 9 arranged in a television broadcasting station
07 is a video storage unit 926 for storing video to be broadcast.
And a video transmission unit 924 that reads video data from the video storage unit 926 and transmits the video data to the user's video device. The video transmission unit 924 supplies program data including video and audio information to the user via the network 909.

A video device 906 placed at the user's home
Includes a scenario table receiving unit 922 that receives a scenario table from the network 908, a summary video scenario table storage unit 923 that stores a scenario table of the received summary video, and a user based on the summary video scenario table corresponding to each program. Includes a multi-program scenario table creating section 920 that creates a multi-program scenario table for viewing.

The video device 906 further includes a user input unit 916 on which a touch panel and operation buttons operated by the user are arranged, and a summary video based on a multi-program scenario table according to input information from the user input unit 916. Display control unit 917 for controlling the display of the image, a video receiving unit 925 for receiving the program data via the network 909, a video storage unit 927 for storing the received video, and control of the video display control unit 917. A video playback unit 918 that receives data from the video storage unit 927 and plays back the video based on the video playback unit 927, and a video display unit 919 that receives the output of the video playback unit 918 and displays the video so that the user can recognize the video.

FIG. 19 is a flowchart for explaining the operation of creating a scenario table for a server located in the EPG provider.

The server creates a plurality of scenario records having different total reproduction time lengths and continuous reproduction time lengths. For example, 1
Consider creating zero scenario records.

Referring to FIG. 19, first, in step S93,
When the process is started in step 1, a program is obtained in step S932. Figure 1 when acquiring a program
8 is a video priority information storage unit 9
13 stores video priority information. The summary video scenario table creation unit 914 initializes the index K of the scenario record to 1 (step S933).

Subsequently, a summary scenario table is created (step S934). The creation of the summary scenario table in step S934 is performed by the series of processes described with reference to FIGS. Therefore, description of the processing in step S934 will not be repeated.

If one scenario record is created in step S934, then in step S935
The index number K is incremented.
The process advances to step S936 to determine whether all records have been created. This determination is made, for example, based on whether or not the index number K matches 10.

If the creation of all records has not been completed, the flow returns to step S934, and the next scenario table is created.

On the other hand, when all records have been created, that is, when ten scenario records having different total playback time lengths have been created, the flow advances to step S937 to store a scenario table including these scenario records on a recording medium. That is, the summary video scenario table is shown in FIG.
8 is stored in the summary video scenario table storage unit 915. Then, the process proceeds to step S938, and the process of creating the scenario table ends.

FIG. 20 is a flowchart illustrating a procedure for a client to obtain a scenario table created by an EPG provider.

Referring to FIGS. 18 and 20, when the server 905 of the EPG provider starts processing (step S
951), and enters a reception waiting state of waiting for a request from the client (step S952).

The video device 906, which is a client, records the program (step S956), and upon completion of the recording, requests the server 905 on the EPG provider side to transmit a scenario table (step S95).
957). This request may be at the start of the recording of the program instead of at the end.

The server of the EPG provider reads the scenario table from the recording medium in response to a request from the video device (step S953). The recording medium is, for example, a hard disk or the like included in the summary video scenario table storage unit 915 in FIG. Then, the server 905 transmits the scenario table to the video device 906 (Step S954).

The video device 906 is in a reception waiting state when the scenario table acquisition request is made in step S957 (step S958), and is received when the scenario table is transmitted from the EPG provider side, and the local recording medium is received. (Step S959). The local recording medium corresponds to the summary video scenario table storage unit 923 in FIG.

Then, the process of obtaining the scenario table ends (step S960). FIG. 21 is a block diagram for explaining another example of the operation system using the network.

Referring to FIG. 21, this system is different from the system shown in FIG. 18 in that an EPG provider server 905a and a user's video device 906a are included instead of server 905 and video device 906. .

The server 905a is different from the server 905 shown in FIG.
Instead of the multi-program scenario table creation unit 97
0 and multi-program scenario table sending section 97
1 in that the server 905a transmits a multi-program scenario table to the network 908.

The video device 906a includes a scenario table receiving unit 922, a summary video scenario table storage unit 9
23 and multi-program scenario table creation unit 9
20 is different from the video device 906 shown in FIG. 18 in that a multi-program scenario table receiving unit 972 is included in place of 20 and a multi-program scenario table is received from the network 908. Other configurations are the same as those shown in FIG. 18, and description thereof will not be repeated.

FIG. 22 is a flow chart for explaining the operation of the system shown in FIG. 21 for acquiring the scenario table.

Referring to FIGS. 21 and 22, when the server 905a on the EPG provider side starts processing (step S981), it waits for a request from a user (step S982).

When the processing is started in the video device 906a (step S985), the user enters the user input unit 91
6 to request viewing of a summary video of a plurality of programs (step S986). Then, the video display control unit 91
7. This request is transmitted to the EPG provider via the multi-program scenario table receiving unit 972 and the network 908 (step S987).

When the server 905a on the EPG provider side receives the request (step S982), the multi-program scenario table creation unit 970 creates a multi-program scenario table from the scenario table of each program (step S983). Then, the multi-program scenario table is transmitted to the video device 906a (step S984). When transmission ends, EP again
The server 905a of the G provider enters a reception waiting state (step S982).

The video device 906a waits for reception after requesting the acquisition of the multi-program scenario table (step S988).

When the multi-program scenario table is transmitted from the EPG provider, the video is reproduced based on the multi-program scenario table (step S989). Video reproduction is performed based on the processing described with reference to FIGS. 11 and 12, and therefore, detailed description will not be repeated.

If there is an input from the user (step S990), the program returns to the beginning of the program if necessary.
The reproduction is started again (step S989).

If there is no particular input from the user, it is determined in step S991 whether or not the reproduction has been completed.
If the reproduction has not been completed, the process returns to step S990 again to monitor the user's input. On the other hand, when playback ends,
Proceeding to step S992, the process ends.

The embodiments disclosed this time are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

[0169]

As described above, according to the present invention,
Even if there are many unviewed programs, the user can grasp the outline in a short time by reproducing the summary image that fits within the predetermined time. In addition, when the user wants to know the contents of the summary image a little in detail during the reproduction of the summary image, the digest level can be changed at that point, so that the desired image can be easily found in the summary image.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a video apparatus 10 using a summary video, which is one embodiment of the present invention.

FIG. 2 is a diagram showing how video data included in a program in the present specification is structured.

FIG. 3 is a block diagram showing a configuration of a video priority determining unit 102 in FIG.

FIG. 4 is a schematic diagram showing priorities analyzed for each frame by a video priority determining unit 102;

FIG. 5 is a first half of a flowchart showing the operation of the summary video scenario table creation unit 104 in FIG. 1;

FIG. 6 is the latter half of the flowchart showing the operation of the summary video scenario table creation unit 104 in FIG. 1;

FIG. 7 is a diagram for explaining that a summary video scenario table is created according to the flowcharts shown in FIGS. 5 and 6;

FIG. 8 is a diagram illustrating the structure of frame sets 551 and 552.

9 is a summary video scenario table creation unit 10 of FIG.
FIG. 9 is a schematic diagram showing an example of a summary video scenario table created in 4.

FIG. 10 is a display example for explaining the image display unit 109 in FIG.

11 is the first half of a flowchart for explaining the operation of the video display control unit 107 in FIG. 1;

FIG. 12 is the latter half of the flowchart for explaining the operation of the video display control unit 107 in FIG. 1;

FIG. 13 is a diagram for explaining the contents of a video stored in the video storage unit 101 of FIG. 1 at the start.

FIG. 14 is a conceptual diagram for describing creation of a multi-program scenario table.

FIG. 15 is a schematic diagram for explaining reproduction when digest levels are prepared for three layers.

FIG. 16 is a schematic diagram for explaining an operation when a fast-forward button is pressed.

FIG. 17 is a schematic diagram showing an example in which additional data of a video and a summary video scenario table are transmitted from an EPG provider.

FIG. 18 is a block diagram showing a first system configuration example in an operation example in a network.

FIG. 19 is a flowchart illustrating an operation of creating a scenario table of a server arranged in an EPG provider.

FIG. 20 is a flowchart illustrating a procedure in which a client acquires a scenario table created by an EPG provider.

FIG. 21 is a block diagram for explaining another example of the operation system using the network.

FIG. 22 is a flowchart for explaining an operation of acquiring a scenario table in the system shown in FIG. 21;

[Explanation of symbols]

10,906,906a Imaging device, 100,910
Video input unit, 101, 911, 926, 927 Video storage unit, 102, 912 Video priority determination unit, 103, 9
13 video priority information storage unit, 104, 914 summary video scenario table creation unit, 105, 915 summary video scenario table storage unit, 106, 916 user input unit, 107, 917 video display control unit, 108, 918
Video playback unit, 109, 919 Video display unit, 110
Multi-program scenario table creation unit, 200 video sequences, 211 scenes, 221 shots,
226 cuts, 230, 571, 572 frames,
300 input unit, 301 video analysis unit, 302 frame priority determination unit, 303 output unit, 551-558 frame set, 600 original video, 601-603
Scenario records, 601.1 to 601.2, 602.
1 to 602.3, 603.1 to 603.4, continuous playback frame set, 701 touch panel, 702 to 706
Operation buttons, 900 providers, 901 broadcast stations,
902 home, 905, 905a server, 907 video transmission device, 908,909 network, 920 multi-program scenario table creation unit, 921 scenario table transmission unit, 922 scenario table reception unit, 923 summary video scenario table storage unit, 924
Video transmitting unit, 925 Video receiving unit, 970 Multi-program scenario table creating unit, 971 Multi-program scenario table transmitting unit, 972 Multi-program scenario table receiving unit.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 5/76 H04N 5/91 N 5/765 L

Claims (9)

[Claims] 1. Search means for searching for a key frame of a predetermined priority in order from the beginning of a video sequence including a plurality of frames arranged in time series,
A plurality of frames including the selected key frame, and a selection means for selecting a frame set, which is a plurality of frames corresponding to a predetermined continuous playback time, as a summary video; and Judgment means for judging whether or not the search has been completed, and when the search has been completed up to the end of the video sequence, a judgment means for reducing the priority and causing the search means to search again. A video device comprising: time determination means for completing creation of a summary video when the total exceeds a predetermined total playback time. 2. The image processing apparatus according to claim 1, further comprising: a priority accumulating unit that accumulates the priority corresponding to each of the frames, which is referred to by the searching unit, wherein the selecting unit stores information that specifies the frame selected as the summary video. Outputting a scenario including the information.
An imaging device according to claim 1. 3. A scenario storing means for storing the scenarios corresponding to a plurality of video sequences, and reading the scenario corresponding to a desired video sequence from the scenario storing means and outputting the scenario to a network in response to an external request. The video device according to claim 2, further comprising a scenario sending unit. 4. The video device according to claim 1, wherein the frame set includes the key frame at the beginning of a time series. 5. A video information storage means for storing a video sequence including a plurality of frames, a user input means for receiving an instruction from a viewer, reproduction control information of a summary video corresponding to the video sequence, and the user input means. Video display control means for controlling display of a summary video in accordance with the output of the video sequence, wherein the playback control information corresponds to each of a plurality of summary videos having different total playback times from each other, and Video playback means for reading data corresponding to a frame from the video information storage means in response to an output of the video display control means and playing back the video data in response to an output of the video display control means. A video device. 6. The user input means includes: first input means for inputting an instruction to reproduce a summary video including more detailed contents than a currently reproduced summary video; and Second input means for inputting an instruction so that a summary video including coarse content is reproduced, wherein the multi-layer scenario corresponds to a summary image representing more detailed content as the hierarchy is lower. The video display control means, in response to an input from the first input means, changes a scenario used as a reference for controlling the display of the video sequence from a currently used scenario to a scenario one level higher, In accordance with an input from the second input means, a scenario as a reference of control for displaying the video sequence is set to one of the scenarios currently used.
The video device according to claim 5, wherein the scenario is changed to a scenario lower in the hierarchy. 7. The video display control means, when changing the scenario, adds a frame specified by the changed scenario to a frame currently being reproduced by the changed scenario in the video sequence. 7. The video apparatus according to claim 6, wherein control is performed such that reproduction is continued from a frame whose position on the sequence is close. 8. The video device according to claim 5, further comprising a receiving unit that receives the reproduction control information via a network and outputs the information to the video display control unit. 9. Searching for key frames of a predetermined priority in order from the beginning of a video sequence including a plurality of frames arranged in a time series, wherein the key frames are arranged continuously in a time series in the video sequence;
Selecting a frame set, which is a plurality of frames including the selected key frame and a plurality of frames corresponding to a predetermined continuous reproduction time, as a summary video, and performing the search until the end of the video sequence. Determining whether the search has been completed, and, when the search has been completed up to the end of the video sequence, reducing the priority and performing the search again from the beginning of the video sequence. If the sum exceeds a predetermined total playback time, completing the creation of the summary video and outputting playback control information including a scenario for specifying a frame used for the summary video from the video sequence; Storing control information on a recording medium; and storing the control information from the recording medium in response to a request from a user terminal. Reading the reproduction control information and distributing the reproduction control information to the user terminal.