JPH1169281A - Summary generating device and video display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve a summary extract speed and to make a summary display form sharp. SOLUTION: This summary-generating device extracts reference frame data FS from compressed video data. A video software configuration element changes position, such as changeover of cut is detected based on the reference frame data, and a representative image VD is extracted for the position to obtain a summary VE. A video essence player 100, where the summary generating device and the video display device are realized on one device, splits a display screen 26 into plural area and the same image is displayed for one second each, for example, in the order of each area and a succeeding image is displayed on an original area and the image is being displayed in a way such that slide images were being moved. In this case, the size of the areas is changed, so that the middle area is larger and the edge areas are smaller. The middle area is used to display a present image and the left/right area is used to display past/future images. in this case, the reproduction speed become faster the farther away the image is from the present.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a summary generating apparatus and a video display apparatus, and more particularly, to a summary generating apparatus suitable for use in quickly finding what a viewer wants from a large amount of circulated video. The present invention relates to a video display device. In this specification, an image corresponding to one frame of a television broadcast is referred to as an "image" or a "screen", and a series of a plurality of images is referred to as an "image". A product obtained by adding an audio signal, a caption code (caption code), and the like to this is referred to as “video software (video software for short)”.

[0002]

2. Description of the Related Art The applicant has previously filed applications for summarizing images (Japanese Patent Application Nos. 8-334479 and 9-1504).
45, a video search device filed on July 17, 1997). For the details of these inventions, the descriptions of the respective application specifications and drawings will be cited here. Broadly speaking, in each of these inventions, attention is paid to changes in various elements constituting the video software, and The image related to the position is extracted as a representative image to be a summary.

[0003] In today's world, when more and more video software is being supplied in large quantities, and in the future it will increase, summaries will summarize video software that is useful to each viewer. It becomes a powerful weapon to find out precisely in time. This is because video software looks at the flow of time and usually requires a considerable amount of time to see it, but if it is in the form of a summary like this, the time it takes to see it This is because it is extremely short, and it does not take much time for each viewer to select what he or she wants, even if there is much to be supplied.

[0004]

The technical development has a universal theme. For example, improvement of processing speed, reduction of cost, improvement of usability, and the like. For example, in the case of a microprocessor, in recent years, the operation speed has been rapidly increased, and the improvement of the processing speed in many devices largely depends on the improvement of the operation speed of the microprocessor. However, for example, if the processing procedure is simplified, the processing speed of the apparatus can be improved even if the same microprocessor is used. If you don't need to speed things up, you can use slower and cheaper processors. From this viewpoint, the invention of claim 1 aims at improving the processing speed of the abstract extraction or reducing the cost associated therewith.

[0005] The summary is displayed so that the viewer can grasp the outline of the video software. It is extracted according to the story (flow) of the original video software, and naturally reflects the original story development. Therefore,
The more useful this is, the more useful the summary will be. From the above viewpoints, the inventions of claims 2 to 4 aim to improve the display format of the abstract in particular.

[0006]

In order to achieve the above object, according to the present invention, reference frame data extracting means for extracting reference frame data from compressed video data, and a component of the video represented by the reference frame data are included. A change position detecting means for detecting a position at which any change occurs, and a summary generating means for extracting a representative image relating to the detected position and generating a summary of the video (claim 1).

In addition, a display means having a plurality of display areas and an image sequentially drawn from a desired position of a series of image strings are sequentially displayed on one of the plurality of display areas with reference to a desired position. And display control means for sequentially extracting images from another position before or after a desired number of frames from the position and sequentially displaying the images in a display area different from the display area (claim 2).

[0008] Also, the video represented by the image drawn from another position according to claim 2 has a higher reproduction speed as the position is further away from the reference position (claim 3). . Further, the area of the plurality of display areas according to claim 2 or 3 is gradually increased or decreased (claim 4).

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below in detail based on illustrated embodiments. FIG. 1 shows a block configuration of a video essence sprayer 100 according to the embodiment. The video essence sprayer 100 implements the summary generation device and the video display device described in the claims on a single device.

In FIG. 1, reference numeral 11 denotes an optical disk player, which includes a laser disk, a DVD, a CD-ROM, and others.
The video software stored in the storage medium using the laser light is read. Reference numeral 12 denotes a TV broadcast receiving unit, which includes a broadcast satellite radio wave, a cable television network, VHF, and UH.
F Receive digital and analog television broadcasts transmitted via terrestrial waves and others. A video tape deck 13 reads digital or analog video software from the video tape. Reference numeral 14 denotes an Internet connection unit which receives video software on the Internet via a public line or a dedicated line.

The video software received by each section is taken into the apparatus 100 via the input interface 16 and stored in the original video storage section 17. However, what is stored is DV
There is only a television broadcast without a tangible medium such as D. What is stored on a DVD or other medium is directly taken in from the medium at the time of summary generation processing described later. The viewer determines which and which of the storage targets are to be captured. This operation is performed using the user operation unit 18 in the same procedure as that for recording a video tape deck. According to the broadcast form of the program, etc., it is set to be stored only once, stored daily, stored weekly, or the like. In addition, summaries can be seen in a short time (for that reason, we proposed a method called summarization). Therefore, programs that are of business or personal interest,
All broadcast channels and the like may be stored.

The video software is stored in the original video storage 17 as a file, for example, divided into units of 10 minutes. At this time, table of contents information (index) necessary for later reading, such as reception date, reception start hour, minute, second, and channel number, is also recorded. Voice information is also recorded. Video software is being digitized and compressed (M
PEG, JPEG, MUSE and others). The invention of claim 1 is directed to such kind of data-compressed video software. Here, this is stored in the original video storage unit 17 as it is. Although the analog one is not a processing target of the first aspect of the invention, it is digitized by the input interface 16 and stored in the original video storage unit 17 in order to unify the handling.

Recording (accumulation) is preferably performed cyclically. For example, the storage capacity of the original video storage unit 17 is
It is possible to record for 3 consecutive days for 5 channels. When the viewer has time, he checks the summary, the original video software, and deletes the unnecessary ones. In this way, usually, the storage areas of the original video storage unit 17 and the summary storage unit 19 have a margin. Broadcast contents are recorded continuously for a certain period of time, during which time the viewer does not check summaries and the like, so that the storage capacity may be exhausted. In this case, the next new video signal is overwritten on the oldest one of the recorded video. This way,
It is not necessary to specify the date, time, and the like for recording, and only the desired channel (the channel to be monitored) need be specified by the user operation unit 18.

Referring back to FIG. 1, reference numeral 21 denotes a reference frame data detecting unit. When the data is compressed, the frame data of the video software is referred to as reference frame data FS.
And difference frame data FD (FIG. 2). The reference frame data FS alone contains the relevant image (screen).
Can be restored (decompressed). The difference frame data FD represents only a difference from a previous image, and the image cannot be restored unless the previous data is referred to.
This restoration process requires a high-speed image processing chip.

The summary VE is a set of several representative images VD extracted from the original video software VS (FIG. 3). These representative images VD respond to changes in various constituent elements (elements of interest) of the video software VS, such as switching of cuts (scenes, scenes), appearance of characters in images, and cheering. In any case, one or several of these representative images VD are extracted from a continuous image of 300 frames (10 seconds) or 600 frames (20 seconds). (Frame = frame, image).

In the present invention, attention has been paid to the abstract extraction ratio and the fact that the image data is compressed. That is,
In any case, if the extraction is performed at such a ratio, the difference frame FD is ignored from the beginning, and the summary extraction processing may be performed only on the reference frame FS. This eliminates the need for any arithmetic processing on the difference frame FD, and can simplify and speed up the processing. The same code FS is used for the reference frame itself and its data. The same applies to the difference frame FD.

The reference frame data extracting unit 21 has such a purpose that if the video software VS received at that time or taken in from the optical disc player 11 or the like is compressed data, From
Only the reference frame data FS is extracted. Note that the reference frame data FS and the difference frame data FD are provided with identification symbols ID determined by the compression method (FIG. 2).
Extraction uses this as a landmark. When the data is not compressed, it is preferable to simply read out the image data one by one such as one in four frames. FIG. 3 is a quote from the comic "Chibi Maruko-chan". The code CP is the extracted subtitle code. Hereinafter, the reference image VD and the like are omitted. In addition, each “representative image” includes not only one still image but also a moving image of several seconds related to the change position.

The extracted reference frame data FS or the image data extracted at intervals is supplied to a change position detecting section 22. The change position detecting unit 22 monitors the state change of the designated element of interest in the image data sequentially supplied in this manner, and notifies the representative image extracting unit 23 of the change when it occurs.

As described above, the summary is a set of several representative images extracted from the original video software (FIG. 3). Thus, it is determined whether it is necessary to view the original video software in detail. For that purpose, it is desired that each representative image represents a key point of the video software. Here, the video software (broadcast program) includes various components (features) that can be detected. As defined at the beginning, video software is made up of video and audio, among which audio includes human voices, instrument sounds, cheers, pulsating sounds, and other detectable components. In the video, there are several detectable components such as changing cuts, appearance of characters, sudden change in image brightness, sudden change in color of the entire image, stationary, etc. .

Which of these, or which and which, should be noted is determined according to the characteristics of each broadcast program. When desired by the viewer, it can be set by the user operation unit 18 and when no viewer is specified, a default value is used. The default value is determined with reference to the following example. The change position detecting section 22 detects a state change of the element of interest designated in this way.

Examples of the element of interest are shown below. This has been described in the previous applications. For details, those relating to the present invention will be described here by citing the descriptions of those specifications. First, those that can be applied regardless of the genre of the video software (broadcast program) will be described. “Regarding Video” (1) Extraction of Leading Image The image at the start of the original video software is simply extracted as a title (representative image). If the start time is generally clear, such as the time of news at 7:00 in the morning, noon, or 7:00 in the evening, or the beginning of a program such as a program desired by the viewer is known, the reference frame FS at the beginning is used. Let it be a representative screen. If the above-mentioned division is performed in units of 10 minutes, the reference frame FS at the beginning of the division is extracted. (2) Cut-out for each cut The cut is determined, and a representative image of the number of frames is appropriately taken out for each cut. Although the detailed procedure will be described later, this representative image extraction method is to find a position where the element greatly changes over time, and take out the image by paying attention to this. In this case, it is better to first detect the start (change) of the cut, examine the time from that to the next cut, and take out the image at the intermediate time (PC in FIG. 4) as a still image or a short moving image. Seem. (3) Extraction focused on the appearance of characters In response to the appearance of a telop, subtitle, or the like in a part of the image, the image is extracted as a representative. Subtitle detection will be described later.

The above-described cut determination (2) is performed, for example, as follows. (1) Pixel Consolidation Processing An image has a two-dimensional spread, and is a collection of pixels (pixels, points) such as, for example, approximately 250 dots × 525 lines in the NTSC system, and each pixel has brightness and saturation. , And a hue, and a television transmits 30 images of such a set of pixels per second.

Needless to say, the number of pixels is large. Therefore, in order to speed up and simplify the processing, pixel aggregation is performed here. For example, when the sum or average value of 4 × 4 pixels is obtained, the original video has a data amount of 1/16. When the same processing is performed on 8 × 8 pixels, the video data is reduced to 1/64. You.

This aggregation eliminates gradual changes in the image and makes it easy to determine the cut switching. That is, the cut switching determination basically compares the pixels of the two preceding and succeeding images with each other and determines whether or not there is a large difference therebetween. Specifically, for example, comparing the data of both pixels in several areas of the image, if many are in common, there is a single cut of, for example, that the person has moved without changing the background Think of it as the image inside.

However, for example, when there is a zoom-in / zoom-down or a slow change in the direction of the camera, comparing the pixels of the preceding and succeeding images without performing the above-described aggregation processing, the appearance is slightly small, but the pixel level is large. In this case, the data is different in the portion of the data. Therefore, even during the actual cutting, most of the data is determined to be the cut switching position.

In this regard, in the coarse pixels after the aggregation processing, the data up to a certain number of frames does not change because the pixels are coarse. Therefore, these are determined to be images belonging to the same cut. Therefore, as pre-processing for this, the values of those coarse pixels, a (t, x, y)
Take out. Here, t: time x, y: coordinates of the image after the aggregation processing a: the color value of the point (x, y). a
Is to adopt R, G, B as they are, or C ₁ · R + C ₂ ·
G + C ₃ · B (R, G, B are values of three primary color information, and C ₁ , C ₂ , C ₃ are weighting coefficients).

(2) Aggregation processing in the time direction Next, regarding the video data on which the above-described aggregation processing has been performed,
The difference between each pixel of each image and each pixel of the previous image is obtained, and it is determined whether or not the size is large enough to be a cut switching position. To simplify the processing, a fixed threshold value is set for the magnitude of the difference at this time. If a large difference exceeding a threshold value occurs for a certain number of pixels among all the pixels in the image, it is determined that the image has changed there, that is, a cut has been switched.

Even if the cuts belong to the same cut, the magnitude of the difference between the pixels differs depending on the contents. The difference between the preceding and succeeding images is large for a cut in which one frame changes rapidly, and the difference is small for a slow one. Therefore, the simple processing as described above is one method, but here, the following processing is further added in order to more accurately extract the representative image. "Process of temporal difference comparison of video" A temporal difference is obtained for the data a (t, x, y) of each point constituting one image. That is, d (t, x, y) = a (t, x, y) −a (t− △ t,
x, y). Where Δt is an appropriate time width. this is,
The difference (change amount) between pixels of two images that are temporally adjacent to each other by Δt is shown.

The difference d (t, x, y) is calculated for a predetermined period.
For example, it is determined for each image during one minute. In terms of the formula,

(Equation 1) Becomes

FIG. 4 shows an example of the difference Da (t) for an aggregated video image of, for example, one minute obtained in this way. The position where the value is large, that is, the position CS1 or CS2 where the difference between the images is large indicates that there is some large change in the image there, and there is a high possibility that the cut is switched here. Therefore, one cut is estimated between CS1 and CS2, and one representative image is extracted from the cut. Although different depending on the genre, a still image that simply represents a cut is generally located near the center of the cut. Therefore, in the example of FIG. 4, the middle position PC of the cut is set as the representative image extracting position. These are the “temporal difference comparison processing of video” referred to in the above heading.

For each image constituting the video software, attention is paid to, for example, image data of one horizontal scanning line near the center of the image, and a difference accompanying the change of the image is obtained with respect to the image data to detect the change of the cut. You may do it. Specifically, for example, the video signal for one horizontal scanning line is divided into N sections, and a sum or an average value is obtained for each section. Then, for each section, a difference is obtained by comparing with the average value of the previous image in the section. The difference is summed for each image, and a position where the value is large, that is, a position corresponding to CS1 or CS2 in FIG.
This is a cut switching position similar to the above.

The cutout focusing on the character appearance of the above (3) is executed by applying, for example, the kanji OCR method. This is enough if it is possible to recognize that there are some characters in the image, so for example, focusing on the lower part or both ends of the image, which is generally the display position of telops and subtitles, and roughing the image The presence of a character is determined based on the resolution and whether any character can be read therefrom. This speeds up processing.

Referring again to FIG. 1, the change position detector 22 detects that the state of the element of interest has changed as described above.
Is detected, the representative image extracting unit 23 extracts the reference frame FS at that time as a representative image. The extracted representative image is stored in the summary storage unit 19. In this case, the file records the broadcast start date, month, day, hour, minute, and second of the original video software so that the relationship between the accumulated representative image and the original video software can be understood. Further, each representative image among the images is provided with address information for identifying a take-out position. This can be represented, for example, by starting from the broadcast start date / time / day / hour / minute / second and by the number of elapsed seconds or the number of frames from that point.

When the genre of the broadcast channel to be monitored is determined to some extent, it is preferable to detect the switching of the cut by paying attention to the constituent elements exemplified below (there may be some overlaps with those described above). Components that are not directly processed by the present invention but are not components of the image are also shown for reference). Image genre Featured elements News Cut with pattern Drama Cut with subtitle Cut with audio Documentary Cut with audio English conversation Cut with subtitle Sports Applause, cheering cut and its surroundings (audio climax) Animation Cut with subtitle Long and still Cut with sound Cut with sound TV shopping Cut with subtitles (You can see information such as price) Song program Cut at the beginning of music (Discrimination from voice) Educational program Cut with pattern Variety show Cheer-up cut Orchestra starts music However, (periodic extraction) Weather forecast Stationary cut (pattern (flip) = board on which characters and pictures are written. Speakers use TV)

As described above, the elements to be noticed, the change positions thereof, and the positions of the representative images to be taken out corresponding to them vary depending on the genre of the video software. Therefore, the summary generation process and the check may be performed several times to determine a preferable element of interest, a predetermined position, and the like.

Referring again to FIG. 1, the user operation unit 18 includes various operation buttons generally provided in a DVD player and the like (not shown), including operation buttons necessary for implementing the present invention. Each of these buttons is realized by a screen display or a mechanical switch, and in response to the viewer's operation, the display control unit 24 executes a corresponding process. Less than,
An example of operation button names is shown on the left side, and an example of processing corresponding to the right side is shown. SELECT List of recorded files (list of recording start times for each file) EP / NP Switching between essence play and normal play FWD Manual screen advance. BK Manual previous screen return AUTO Automatic frame advance (continuous playback) WIND Winding (reverse of rewinding). Fast forward REW during image playback. Review during image playback STOP Stop ERASE Erase original video software and its summary PAUSE Pause

The processing corresponding to the operation buttons of WIND, REW, STOP, and PAUSE is the same as that normally performed in a DVD player or the like. Description is omitted. When the SELECT button is pressed, a list of original video software recorded in the original video storage unit 17 is displayed on the display. Move the cursor to the desired file with the cursor key (not shown) and press the return key. Thus, the file is selected, and its summary or original video software is displayed in the following essence play and normal play. If you do not select a file with the SELECT button, the oldest file will be selected.

The essence play means that each representative image to be summarized is read from the summary storage unit 19 and displayed on the display 26. The display control unit 24 performs this processing. The inventions of claims 2 to 4 improve the way of displaying the summary, and FIG. 5 shows a first display form example, and FIG.
2 shows a second display mode example. FIG. 5 divides the screen of the display 26 into four parts and displays a series of representative images in each area (display area).
A display mode is shown in which the images are displayed while being sequentially sent to A to D.
Note that reference numerals A to D are shown only in FIG.

Here, it is assumed that the mode is an AUTO mode, that is, a mode in which the representative images are sequentially read and displayed at a predetermined time interval Δt (for example, 2 seconds), that is, the automatic turning of the summary. For the sake of explanation, assuming that the first image, the second image,... Are in order from the top representative image, the first image is displayed in the area A at the first time t. At this time, nothing is displayed in other areas (FIG. 5A).

After Δt, for example, 2 seconds, the first image is displayed in the area B (FIG. 5B). At this time, the second image is displayed in the original area A. To the viewer, the slide connected to the first image and the second image appears to have moved one frame from the upper left to the right. Further △ t
After the lapse of time, the first image is displayed lower than the second image by one, and the third image is displayed in the first area A. To the viewer, the slides connected to the first image, the second image, and the third image appear to move rightward from the upper left side and bend down one position.

When Δt further elapses, the first image is displayed on the left, the second image is displayed on the lower side, and the third image is displayed shifted by one to the right, and the fourth image is displayed on the first area A (FIG. 5). (D)). To the viewer, the slide just connected from the first image to the fourth image appears to make a U-turn from upper left to right, lower, and left. Thereafter, this display is executed every Δt. The viewer seems to see the connected representative images appearing from the upper left of the screen and sequentially disappearing at the lower left. Needless to say, video software has one stream. Therefore, the representative image also has a flow along it, and when these are sequentially displayed in this manner, the flow is easier to grasp as compared to the case where the flow is simply displayed as in the related art. If the display is to be performed in the middle of the summary, the display may be started from the target position and the four representative images before and after the target position.

The FWD button and the BK button are used to manually execute this essence play one frame at a time. That is, each time the FWD button is pressed, the representative image moves to each area by one frame. 1 each time the BK button is pressed
Go back to the previous area one frame at a time. If you do not press the button, it will stop there. The display time of each representative image in the above-described AUTO operation is set according to the viewer's preference. Time is 1
Short values such as seconds, 2 seconds, and 3 seconds may be used. The setting is performed by a setting button (not shown). When there is no instruction from the viewer, a default value is used.

When the representative image is a moving image, the sending time to the next area is determined in consideration of the length of each moving image as each representative image. For example, assume that the length of each moving image is 1 second. Here again, for the sake of explanation, it is assumed that each movie is numbered sequentially from the top. The operation is AUTO. At first, the first moving image is reproduced in the area A over the one second. Next, it is reproduced in the same manner in the area B. At this time, in the original area A, 2
Play the numbered video. When the reproduction of the two areas is completed, the first, second, and third moving images are reproduced in areas C, B, and A next. The following is the same as when the representative image is one frame. Now, the screen showing the short video appears to move from the upper left to the right, lower, and left.

When the time of the moving image is long, for example, 5 seconds, the reproduction speed is appropriately increased in each area to reproduce the moving image. Since one moving image is repeatedly displayed in four areas, the contents can be sufficiently perceived even by slightly fast-forwarding. In this regard,
Conversely, even when each moving image is considerably short, it is considered that the reproduction may be performed as it is. If it is too short and the screen passes over the display 26 instantaneously, the reproduction speed may be reduced.

The same applies to humans and other animals, but the eyes respond more when they are moving than when they are stationary. Therefore, there is a portion where the contents are more easily perceived at this time. Therefore, the display form in an atmosphere where the representative image is a moving image and the screen in which it is reflected is sequentially sent in this way,
It is highly useful as one form of summary display. Frame advance is also useful when the representative image is a moving image. That is, when the FWD button or the BK button is pressed, the display control unit 24 forwards or returns the moving image displayed at that time by one frame in each of the areas A to D. Thus, frame advance is simultaneously performed in the four areas.

However, one moving image as a representative image has 30 frames even for one second. If you send this one frame at a time,
It takes too long. So, for example, if the FWD button is 1
If pressed once, one third of the length of this video, for example, 1
0 frames are played back at the same speed as the original image software, and the screen is stopped there. When the button is pressed again, the next 10 frames are reproduced. This realizes a screen display in which the player moves a little and stops, as in the play of "Dharma has fallen".

In the second display example shown in FIG. 6, the area size and the reproduction speed are devised. Region EK
Are maximum at the center H, and are sequentially reduced toward the left and right. Also, the center H is the uppermost layer, and the layers are sequentially lowered as going left and right. Here, the center area H is assumed to be the current time, and the left area E is assumed to come at a certain point in the future,
The left F and E represent a certain point in the future that comes later respectively. The right side I is at some point before the present,
The right side J, K represents a certain point in the past that has passed each earlier. The playback time there is earlier at a point (area) farther from the present (center area H).

The display of each representative image is specifically as follows. First, assuming that the display time of one frame of the representative image in the central area H (current scene) is T seconds, the left and right areas G, I
Display time is shorter than that. In F and J and E and K next to them, the length is further shortened. In terms of the playback speed of the original video software, the farther from the current, the faster. If the reduction rate of the display time is x, the display time is xT in each of the left G and right I regions of the central region H, x ² T in those left F and right J, the leftmost E and the rightmost. In K, it is x ³ T. That is, the geometric series is shortened. In this case, the underlying display time T
The ratio x and the ratio x are preferably adjustable. In the regions of the leftmost E and the rightmost K, these are effective as a cube, so the ratio x should be finely adjusted.

When the display time (reproduction speed) in each area is changed in this way, the display content of one area catches up with the display content of the other in the same manner as in traveller calculation. Therefore, the display start image of each area at the start of the essence play is determined by assuming this catch-up. A specific example will be given. First, the display time in the current area H is set to T = 2 seconds. Then, 30 frames of the representative image are displayed in the current area H. Also, let x = 0.6.

In this case, the number of frames of the representative image to be displayed in the central left and right regions G and I is 30 / 0.6 = 50 frames, and in the left and right regions F and J, 83 frames and the leftmost E
And 139 frames in the rightmost K (fractions are rounded).
Therefore, when the essence play is started assuming that the position desired by the viewer for a certain summary is to be displayed in the current area H, the display control unit 24 firstly sets the representative image at the position desired by the viewer (from the top of the summary). n) are sequentially displayed in the central area H at intervals of 2 seconds.

This is continued for 30 frames in the central area H.
At the same time, 50 frames are displayed in the left and right areas G and I. Therefore, in the left area G, after 30 frames, that is, n
Display starts from the + 30th representative image. The display time for one frame is 1.2 seconds of 2 × 0.6. In the area F on the left, the display time for one frame is 0.72 from the (n + 80) th frame.
The display starts in seconds, and from the (n + 163) th frame on the leftmost E,
Display starts with a display time of one frame of 0.432 seconds. In the center right region I, the image is started 50 frames before, that is, the (n-50) th representative image. In the area J on the right, display starts from the n-133th, and in the area K on the right, display starts from the n-272. The display time is the same as each corresponding area on the left.

The number of representative image frames of the entire summary is, for example, 150 frames assuming that display is completed in 2 seconds per frame and 5 minutes.
Assuming that the display is completed in one frame per second display and 5 minutes, the number of frames becomes 300 frames. Under the conditions of the specific example shown in the second display example, 544 frames can be displayed in one minute by combining five areas. Therefore, a normal summary can be displayed in about 30 seconds. However, the display is slow in the central area H and the left and right areas G and I. Therefore, if it is desired to see other places slowly, the “current position” corresponding to the central area H is adjusted to the position of the desired representative image, and the essence play is executed again.

The operation of FWD and BK is useful also in the second display mode. However, in this case, the display time of the representative image in each area becomes the same. Therefore, the central area H
Assuming that display is started from the n-th representative image as before, in the left and right regions G and I, for example, 1
Minute future position or one minute past position, and further in the left and right areas F and J, five minutes future or five minutes from the present.
Minutes past, and so on, starting from the representative image of the future or past position as appropriate starting from the current position n, respectively,
Then, the display of the representative image is started.

The time in parentheses in FIG. 6 is an example of this time. In addition, since the information of the extraction position is attached to each representative image, the position and the like in the 5-minute future are searched by using this. The display control unit 24 reads the representative image read in this manner from each of the above positions each time the FWD, BK operation button is pressed and displays the representative image in each area. The essence play, that is, the display of the summary is executed as described above. The summary file that has been read out is recorded for reference in a later erasing operation.

Next, the normal play means that the original video storage 1
7 is to read the original video software stored on the display 7 and display it on the display 27 in the same manner as a normal video tape recorder. When the EP / NP button is pressed, the display control unit 24 switches and executes the normal play and the essence play. As described above, both the original video software and the abstract are indexed with the broadcast date, hour, minute, second, frame number, etc., and the original video storage unit 17 or the abstract storage unit 1 is indexed.
9 is stored. When the EP / NP button is pressed in the essence play state, the display control unit 24 refers to these indices, reads out the original video software that is projected at that time and corresponds to the summary from the original video storage unit 17, and Is displayed on the display 27. Here, the file read out is also recorded to that effect.

In normal play, the original video is reproduced at a normal speed. Operations such as fast forward and rewind are executed in the same manner as a general video tape recorder, DVD player, or the like. The display form examples described with reference to FIGS. 5 and 6 (display form examples corresponding to claims 2 to 4), particularly the display form examples shown in FIG. It can also be applied to the display of. It can be applied to the display corresponding to the operation of FWD and BK for the original video software.

When the desired number of summaries or the contents of the original video software have been confirmed, the ERASE button is pressed. In response to this, the display control unit 24 changes the original video storage unit 17 and the summary storage unit 1
9 is displayed on the display 26 in a list. At this time, the file whose contents have been confirmed is displayed to that effect. If you want to delete them as is, press the return key. Files with the confirmed indication are deleted at once. If you want to keep it for some reason, remove it from the list before erasing.

Hereinafter, modified examples will be described. The present invention may be realized as a dedicated device or as an application program of a personal computer. In that case, floppy disk, CD-ROM
It will be stored and distributed in such as. It may be distributed through the Internet. It is also possible to manufacture a dedicated chip and incorporate it into a personal computer, or incorporate it into a DVD player, VTR, game player, or the like.

The server on the Internet has the function of generating and storing the summary and the function of storing the original video software, transmits this to the viewer side terminal by the access of the viewer, and displays the summary and the original video. Is also good. The original video storage unit 17 and the summary storage unit 19 may use a home video tape or other magnetic tape. If the recording time is not enough, add more decks (drives).

[0060]

As described above, according to the first aspect of the present invention, the reference frame data is extracted from the compressed video data, and the position where any of the components of the video represented by the reference frame data changes is changed. , And a representative image relating to the detected position is extracted to generate a summary of the video software. Therefore, it is possible to increase the processing speed when performing the abstract extraction processing of this type, including the inventions of the earlier applications, or to reduce the cost of this type of apparatus.

According to the second aspect of the present invention, the display screen is divided into a plurality of regions, and images sequentially drawn from the positions are sequentially displayed in one of the plurality of display regions based on a desired position of a series of image strings. At the same time, the image is sequentially extracted at another position before or after a desired number of frames from the reference position and sequentially displayed in a display area different from the display area. Further, in the invention according to claim 3, a video represented by an image drawn from the another position has a higher reproduction speed as the other position is further away from the reference position. In the invention of claim 4, the area of the plurality of display regions is increased or decreased stepwise.
Therefore, according to these inventions, it is possible to visually sense the passage of time, the change of scenes, and the like. Therefore, the contents of the video software can be understood more accurately, and the utility of the summarization method is fully demonstrated.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating an example of an embodiment.

FIG. 2 is a conceptual diagram showing an example of compressed video data.

FIG. 3 is a conceptual diagram showing an example of original video software VS, a representative image VD, and a summary VE.

FIG. 4 is a graph showing an example of a difference between image data.

FIG. 5 is a conceptual diagram showing a first example of display.

FIG. 5 is a conceptual diagram showing a second display example.

[Explanation of symbols]

21: Reference frame data extraction means 22: Change position detection means 23: Summary generation means 24: Display control means 26: Display means 100: Summary generation device A to K: display areas EK: display areas whose areas are gradually increased or decreased FS: reference frame data VD: representative image VE: summary

[Procedure amendment]

[Submission date] December 4, 1997

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction target item name] Brief description of drawings

[Correction method] Change

[Correction contents]

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating an example of an embodiment.

FIG. 2 is a conceptual diagram showing an example of compressed video data.

FIG. 3 is a conceptual diagram showing an example of original video software VS, a representative image VD, and a summary VE.

FIG. 4 is a graph showing an example of a difference between image data.

FIG. 5 is a conceptual diagram showing a first example of display.

FIG. 6 is a conceptual diagram showing a second example of display.

[Description of Signs] 21 ... Reference frame data extraction means 22 ... Change position detection means 23 ... Summary generation means 24 ... Display control means 26 ... Display means 100 ... Summary generation device. A to K: display areas EK: display areas whose areas are gradually increased or decreased FS: reference frame data VD: representative image VE: summary

Claims (4)

[Claims] 1. A reference frame data extracting means for extracting reference frame data from compressed video data, and a change position detecting means for detecting a position at which any of the components of the video represented by the reference frame data changes. And a summary generating means for extracting a representative image relating to the detected position and generating a summary of the video. 2. A display means having a plurality of display areas, and an image sequentially drawn from a desired position in a series of image strings is sequentially displayed in one of the plurality of display areas with reference to a desired position in the series of image sequences. A display control means for sequentially extracting images from another position before or after a desired number of frames from the position and displaying the images sequentially in a display area different from the display area. 3. The reproduction speed of an image represented by an image drawn from the another position is increased as the other position is further away from the reference position. Video display device. 4. The video display device according to claim 2, wherein the plurality of display areas have areas gradually increased or decreased.