JP2000287165A - Image information description method, video image retrieval method, video reproduction method, video retrieval device and video reproduction device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a user to facilitate retrieval by confirming the contents of a video data through representative frame display or variable speed reproduction from compression coded video data in compliance with the MPEG-2 or the like even with a retrieval device having a low CPU capability. SOLUTION: Information such as the frame number of an original video frame and the size of each sampled image frame corresponding to a sampled image frame group is described as sampled image information relating to the sampled image frame group resulting from sampling a video frame group of the original video data 101 into an optional spatial size at an optional temporal interval and the scene change position information of the original video frame group and inter-frame screen change amount information area described together as attached information. Then a database is built up by cross-referencing the resultant time-space sampled video meta data 102 with the original video data 101, and the original video data are subject to representative frame display or variable speed reproduction by using the time-space sampled video meta data 102.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for describing image information, particularly sample image information relating to a sample image frame group obtained by sampling a video frame group at an arbitrary time interval and at an arbitrary spatial size. The present invention relates to a method and apparatus for video search and video playback using sample image information.

[0002]

2. Description of the Related Art Recent advances in semiconductor technology and digital signal processing technology have made it possible to convert video (video) information from analog data to digital data and compress the data in real time. In fact, in digital satellite broadcasting, digital video data compressed and encoded by MPEG-2, which is the international standard for moving picture compression, is distributed. And so on.

[0003] Also, with the increase in the density of optical discs, MP
The technology of recording digital video data compressed by EG-2 or the like has also reached the practical stage. A typical example of such an optical disk medium is a DVD-RAM or a CD-RAM.
There is RW. Although the recording time is shorter than DVD-RAM,
Digital video data can also be recorded in DD. In the future, digital video data recorded on a DVD-RAM or the like will be required to be easily searched, like digitalized character and still image data.

[0004] A classic technique of video search is to define a title and a keyword for each video file such as a movie, and to search by either or both of the title and the keyword. This method
Although the search itself is easy, it is difficult to perform a detailed search according to the content of the video, and the disadvantage is that it is impossible to know whether or not the video is the desired video without actually reproducing and displaying the video.

When recording compressed digital video data, since a video can be treated as a sequence of still image frames, a characteristic image frame called a representative frame is selected from the original video by an image processing technique, and the list is selected. A method of displaying is considered. As a representative frame, a portion called a scene change at which a screen is switched is often used. However, since a scene change occurs only once every few seconds, and sometimes once every several tens of seconds, there is a limit in expressing the contents of a video in a representative frame. The only way to check the video of a frame between scene changes is to decode and display the original video data.

Digital video data compressed according to international standards such as MPEG-1 and MPEG-2 has a mechanism for random access to some extent.
Variable speed reproduction (trick play) such as random reproduction and fast forward reproduction can be performed. However, these variable speed reproductions are performed by manipulating the digital video data itself, so that the processing is heavy, and the processing load is large for a home-use receiving device having a small computer power. Also,
The above-mentioned variable-speed playback in an environment where digital video data is distributed from a server installed at a remote location through a network and received by a home computer or television receiver, such as video-on-demand or browsing on the Internet Is difficult to increase network traffic.

[0007]

As described above, in the conventional general video search, the search is performed only by the title name or keyword assigned to the video file. Is not provided enough.

In a method of selecting a scene change portion from an original video as a representative frame and displaying it in a list,
There is a problem in that it is not possible to check the video of a frame between scene changes.

Further, in the variable speed reproduction mechanism incorporated in the international standard for moving picture compression such as MPEG-1 and MPEG-2, the variable speed reproduction is performed by operating the digital video data itself. There is a problem in that the processing load is heavy on a device with a small computer power, and when trying to perform variable speed playback in an environment where digital video data distributed through a network is received, network traffic increases. .

[0010] It is a main object of the present invention to provide an image information description method capable of performing search and display while confirming the contents of a video.

Another object of the present invention is to enable good image retrieval even when a desired frame exists between scene changes.

It is another object of the present invention to reduce the amount of processing when performing variable-speed playback of video and the like, so that it can be easily realized even on a device with a small computer power or on a network.

[0013]

In order to solve the above-mentioned problems, an image information description method according to the present invention samples a video frame group at an arbitrary time interval and at an arbitrary spatial size. The basic feature is that attribute information for specifying a video frame corresponding to each of the sample image frame groups is described as sample image information regarding the obtained sample image frame group.

Further, in addition to the sample image information including such attribute information, attribute information for specifying a video frame corresponding to each of the sample image frame groups is described.
Further, it is characterized in that the additional information on the video frame group is described.

Here, the attribute information includes one or both of position information indicating a position on the time axis of a video frame corresponding to each of the sample image frame groups and information relating to the size of the sample image frame.

The supplementary information includes one or both of scene change position information of a video frame group and information of a screen change amount between frames of the video frame group.

As the sample image information, image data of a sample image frame group or a pointer to the video frame group of the sample image frame group may be described together.

According to the present invention, there is provided a recording method in which sample image information or sample image information and supplementary information described by the above-described image information description method are stored together with image data of a video frame or separately from the image data. A medium is provided.

Further, according to the present invention, a video search for a sample image frame group is performed as follows using the sample image information or the sample image information and the supplementary information described by the image information description method as described above. And an environment for reproducing video.

That is, a first video search method / apparatus according to the present invention provides a sample related to a sample image frame group obtained by sampling a video frame group at an arbitrary time interval and at an arbitrary spatial size. As image information, at least first position information indicating a position on a time axis of a video frame corresponding to each of the sample image frame groups is described, and the first position information and a given desired video frame Based on the second position information indicating the position on the time axis, the second
A search is made for a sample image frame having the first position information closest to the position information.

As described above, by using the sample image information described according to the present invention, it is possible to easily perform a video search for a desired frame without imposing a load on computer power or traffic.

Another video search method / apparatus according to the present invention comprises:
As sample image information relating to a sample image frame group obtained by sampling the video frame group at an arbitrary time interval and an arbitrary size spatially, at least the time of an image frame corresponding to each of the sample image frame groups The first position information indicating the position on the axis is described, and the scene change position information of the video frame group is also described as additional information, and the first position information and the time of the given desired video frame are described. Based on the second position information indicating the position on the axis and the scene change position information, the second position information and the scene change position information closest to the second position information are determined based on the temporal order of the scene change position information. The method is characterized in that a sample image frame having the first position information closest to the second position information before or after in time is searched.

More specifically, a scene change position closest to a desired frame is detected, and it is determined whether the desired frame is temporally before or after the scene change position. The video frame closest to the desired frame is searched before the change position, and in the latter case, the video frame closest to the desired frame after the scene change position is searched.

By describing the scene change position information as supplementary information in this way, it is possible to search for a sample image frame more similar to a desired frame.

Still another video search method / apparatus according to the present invention provides sample image information relating to a sample image frame group obtained by sampling a video frame group at an arbitrary time interval and at an arbitrary spatial size. As at least, position information indicating a position on the time axis of a video frame corresponding to each of the sample image frame groups is described, a search target image is presented, and a difference between the sample image frame group and the search target image is defined. Is searched for a sample image frame having a predetermined threshold or less. In this case, position information described for a sample image frame whose difference from the search target image is equal to or smaller than a predetermined threshold may be recorded as a search result.

In this way, the difference between the image to be searched and the image of each sample image frame, for example, the sum of the absolute value differences is obtained.
A desired frame can also be searched by searching for a sample image frame having a small value.

According to the video reproducing method / apparatus of the present invention, at least sample image information relating to a sample image frame group obtained by sampling a video frame group at an arbitrary time interval and an arbitrary size spatially is provided. Position information indicating the position on the time axis of the video frame corresponding to each of the sample image frame group is described, and information on the amount of screen change between frames of the video frame group is additionally described as additional information, Using a sample image frame group, variable speed reproduction of video is performed by changing the acquisition position of the sample image frame in accordance with the information on the amount of screen change.

That is, by reducing the reproduction speed when the screen change amount is large, and slowing down the reproduction speed when the screen change amount is small, the variable speed reproduction which is easy to read while the screen change amount is kept constant is sampled. This can be realized for an image frame.

[0029]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows a system architecture according to an embodiment of the present invention. This system is roughly divided into a database 100, a video display engine 104,
It comprises a search engine / specimen image display engine 105, a controller 106 and a display unit 107. The contents of the database 100 are the original video data 10 described in detail later.
1 and the spatiotemporal sample video metadata 102 and a correspondence table 103 (or a correspondence function table) for associating the two.

The database 100 may be centrally arranged at one place or may be dispersedly arranged at plural places. In short, the database 100 can be accessed by the video display engine 104 or the search engine / sample image display engine 105. Good. The original video data 101 and the spatiotemporal sample video metadata 102 may be stored on different media or may be stored on the same media. As the medium, for example, a DVD or the like is used. The original video data 101 may be data transmitted via a network instead of being stored in one place.

The video display engine 104 displays the original video data 101 on the display unit 1 under the control of the controller 106.
A process for displaying at 07 is performed. Further, when the search engine / sample image display engine 105 searches for the original video data 101 based on the spatiotemporal sample video metadata 102, the video display engine 104 displays the searched portion of the original video data 101. Processing for displaying on the display unit 107 is also performed.

Search Engine / Specimen Image Display Engine 10
5 retrieves an appropriate sample image frame in the vicinity of a desired frame of the original video data 101 from the spatiotemporal sample video metadata 102, which will be described in detail later, under the control of the controller 106, and uses them as a representative frame. 107, or spatio-temporal sample video metadata 1
The search for the original video data 101 is performed via the controller 106 using 02.

Here, the difference between the search engine / sample image display engine 105 and the image display engine 104 will be described. The former processes a group of sample image frames in the spatiotemporal sample image metadata 102 having a small capacity. Therefore, a sufficient processing speed can be obtained even if it is implemented as software on a PC having a low capacity built in the receiving device.

Since the latter processes the original video data 101 which is MPEG-2 video data or analog video data, it is often necessary to implement special hardware. Specifically, the original video data 101 is MPEG
-2 compressed video data, a special decode board (MPEG-2 decoder) is used as the video display engine 104. If the original video data 101 is an analog video signal, the video display engine 104 fast-forwards. A video reproducing device such as a VTR that can control rewinding is used.

Note that the original video data 101 is MPEG-1
In the case of compressed video data according to MPEG-4 or MPEG-4, the video display engine 104 can be implemented even by software on a PC, and there is no need to separate them as a system architecture.

The connection between the upper and lower lines in the correspondence table 103 is conceptual, and the correspondence table 103 includes the original video data 101 and the spatiotemporal sample video metadata 1.
02 does not need to be physically connected. Therefore, the medium storing the original video data 101 may be stored in the same housing as the video display engine 104, and the medium storing the spatiotemporal sample video metadata 102
The search engine / specimen image display engine 105 may be housed in the same housing.

The medium storing the spatiotemporal sample video metadata 102 and the search engine / sample image display engine 10
Even if 5 is located at a distance, the line connecting them has a relatively small transmission capacity, for example, 10
A Mbps network is sufficient. On the other hand, the medium storing the original video data 101 and the video display engine 10
The line connecting 4 is 100 depending on the type of media.
It is necessary to prepare a line of Mbps or more.

The advantage of the system architecture as shown in FIG. 1 is that the search is not performed for the original video data 101 but for a smaller data size.
Since the spatial sample video metadata 102 is used as a target, interactive operations can be performed comfortably, and overall traffic can be suppressed low.

FIG. 2 is a conceptual diagram of the original video data 101 and the spatiotemporal sample video metadata 102. The original video data 101 is MPEG-1, MPEG-2, MPEG-4
Digital video data or analog data compressed by the above method, and is composed of a set of video frames (video frame group) constituting a moving image. The original video data 101 is associated with position information indicating a position on the time axis of each video frame, for example, position information such as a media time (hereinafter simply referred to as time) or a frame number. The association between the original video data 101 and the spatio-temporal video metadata 102 is performed by the correspondence table 103 by time or frame number.

The spatiotemporal sample video metadata 102 is mainly composed of sample image information 201 _{1 to} 201 _n .
Further, in the example of the present embodiment, scene change position information 202 and screen change amount information 203 are also included as supplementary information.

The sample image information 201 _{1 to} 201 _n is a sample image frame group obtained by sampling a video frame group constituting the original video data 101 at an arbitrary time interval and at an arbitrary spatial size. And position information (time or frame number) indicating the position on the time axis of the original video frame corresponding to each sample image frame, and the sample image frame such as size information indicating the size of each sample image frame. It consists of attribute information for specifying. Among these pieces of attribute information, position information (time or frame number) indicating the position on the time axis of the original video frame corresponding to each of the former sample image frames is stored in the correspondence table 103.
It is described with reference to FIG.

If the original video data 101 has already been digitized as compressed digital video data, the sample image frames in the sample image information 201 _{1 to} 201 _n of the spatiotemporal sample video metadata 102 The group is created by decoding or partially decoding a desired frame of the original video data 101. If the original video data 101 is analog data, the original video data 101 may be digitized before the sample image frame group is created.

Next, if the original video data 101 is MPEG-2
In the case of compressed video data, the former of the attribute information, that is, position information (time or frame number) indicating the position on the time axis of the original video frame corresponding to each sample image frame will be described. In this case, the MPE
The original video data 101, which is G-2 compressed video data, is decoded and, for example, one frame every 30 frames,
The sample image frame groups 201 _{1 to} 201 _n are reduced by 8
Create In addition, instead of creating a sample image frame group by fixed time sampling and fixed spatial sampling as described above, a sample image frame group can be created by appropriately changing these. It is also effective to perform coarse sampling in the time direction where the screen change amount is small, and fine sampling in the time direction where the screen change amount is large.

In the MPEG-2 compressed video data, there are intermittently frames compressed using only a correlation in a frame called an I picture (intra-frame coded frame). I-pictures are easy to decode because they are not compressed using correlation between frames, unlike P-pictures (forward-predicted inter-coded frames) and B-pictures (bi-directionally predicted inter-coded frames). Therefore, when creating a sample image frame group, only the I picture of the original video data 101 and the DCT of the I picture are used.
(Discrete cosine transform) By decoding only the DC component of the coefficients, a sample image frame group sampled temporally and spatially can be obtained more easily.

Although it is not always guaranteed that I-pictures exist at a constant frame interval, MPEG-2
In order to create a sample image frame group sampled temporally and spatially at a speed equal to or higher than the video rate from the video data compressed by the method described above, a method using an I picture is effective.

The method of creating a sample image frame group from an I picture as described above has an advantage that the processing amount is small and processing can be performed only by software on a PC without using special hardware. Also, when a sample image frame group is created from the original video data 101 via the network, the problem of traffic increase can be easily avoided by using the I picture.

On the other hand, the sampling of the original video data 101 in the spatial direction at the time of creating the sample image frame group does not need to be fixed, and can be appropriately changed. In some cases, not only reduction but also particularly important The frame of the screen may be enlarged. As described above, the sample image information 201 includes the sample image frame group and the attribute information of the sample image frame, and the attribute information includes the size information of the sample image frame group. The image frame can be used after being appropriately converted to a desired size.

FIG. 3 shows a specific description example of the sample image information 201. The sample image information is information described for each frame of the sample image frame group. In this example, (1) the frame number or time of the original video data corresponding to the sample image frame, (2) the sample image frame Size (height x width), (3) number of frames or time to the next sample image frame, (4) image format of sample image frame such as JPEG, RGB, YUV, (5) image data of sample image frame ( Or a pointer to the original video data 101). Where (3), (4),
(5) is not essential, and any of them may be omitted. Further, additional information other than (1) to (5) may be further included.

The sample image frame group may be treated as video data (sample video described later) composed of a temporally continuous frame group. By compressing the video data into, for example, an AVI file or an MPEG-4 file, it is possible to further reduce the size. In that case, the video data is a file pointer to the video frame of the original video data 101 and a frame number. Therefore, an interface for acquiring an image of an arbitrary frame from the video data is required.

FIG. 4 shows a management structure of the metadata 102. In this example, the specimen image information 201 ₁ , 201
₂ , a list structure is used to manage. R
list 4 which is a pointer from sample 401 to sample image information 201 ₁ , 201 ₂ ,.
Connect 02, 403, 404, 405 and End
406 is the final flag. Lists 402, 403,
Numbers ID1, ID2, ID3, ID4 of 404, 405
Are conceptual, and the lists 402 and 40 are listed in this order.
3, 404, 405 are arranged. In this example, the actual sample image information 201 ₁ , 201 ₂ , 202 ₃ , 202 from the lists 402, 403, 404, 405
₄ is pointed to and pointed to.

With such a list structure, it is easy to add and delete specimen image information. When adding a new sample image frame, look up the frame number in order,
Sample image information is added so that the size of the frame number does not reverse. When deleting a sample image frame, the corresponding sample image information may be removed from the list.

The reason for facilitating the addition / deletion of the management of the sample image information 201 in the form of a list structure is that there are many cases where it is desired to add not only the sample image frame initially determined but also the sample image frame later. Take into account. For example, after registering an I picture of MPEG-2 compressed video data as a sample image frame, a scene change position of the MPEG-2 compressed video may be detected, and a frame at the scene change position may be registered as a sample image frame. In this case, the sample image frame from the I picture described above should be registered as a reduced image consisting only of the DC component, and the sample image frame at the scene change position should be registered as a full-size image frame because it is an important frame. Is also possible.

Other examples of description of the sample image information will be described later.

Next, a specific procedure of the description method of the sample image information 201 will be described with reference to FIG.
The case of G-2 compressed video data will be described as an example.
FIG. 5 is a flowchart showing a recording procedure of the spatiotemporal specimen video metadata 102 including the description of the specimen image information 201.

First, a video frame group of the original video data 101 is read (step S11), and the original video frame is temporally sampled (step S12). A scene change position of the original video data is detected (step S1)
3). The scene change position is calculated by, for example, calculating a screen change amount between adjacent frames of the read original video data 101, and detecting a scene change position having a change equal to or more than a predetermined value.

The temporal sampling of the original video data 101 in step S12 can be performed, for example, by sampling a region with a large motion finely and a region with a small motion roughly. In this example, the original video data 101
Is MPEG-2 compressed video data,
At 12, an I picture is extracted to create a sample image frame, and a P picture is extracted to detect a screen change amount.

Next, one sample image frame is created by spatially sampling the I picture extracted in step S12 (step S14). More specifically, in step S14, pixel thinning of the I picture is mainly performed to create a sample image frame including a reduced image. However, if the I picture is an important frame such as a scene change position, the frame of the original video data is used as it is as a sample image frame without reduction, or in some cases, the sample image frame is enlarged by performing pixel interpolation. May be created.

On the other hand, from the P picture extracted in step S12, information on the amount of screen change, that is, information on the magnitude of image change between adjacent screens is obtained (step S15). Since the information of the motion vector from the previous frame is added to the P picture as side information, the screen change amount can be obtained from the size and distribution of the motion vector.

Next, after compressing the sample image frame created in step S14 as necessary (step S14).
16), this compressed sample image frame and step S13
Scene change position detected in step S1 and step S1
2 and 3 using the information of the screen change amount acquired in FIG.
Is recorded (step S17), and the process ends.

That is, in step S17, three pieces of information, ie, sample image information 201, scene change position information 202, and screen change amount information 203 are recorded as the spatiotemporal sample video metadata 102 as shown in FIG. As shown in FIG. 3, the sample image information 201 includes (1) the frame number or time of the original video data corresponding to the sample image frame, and (2) the size (height × width) of the sample image frame. ), (3) the number or time to the next sample image frame, (4) the image format of the sample image frame such as JPEG, RGB, YUV, (5) the image data of the sample image frame (or the original image data 101 Pointer). The image data of the sample image frame of (5) is extracted in step S12 in this example,
This is image data of an I picture that has been subjected to spatial sampling in step S14 and further compressed or uncompressed in step S16 as necessary.

Next, the use of the spatiotemporal sample video metadata 102 thus recorded will be described.

(1) Retrieval of Sample Image Frame Using Scene Change Position Information If it is desired to display a desired video frame, it takes time to retrieve the desired video frame directly from the original video data 101. This is as described above. Instead, if the desired frame is searched by searching the spatio-temporal sample video metadata 102 obtained by sampling the original video data, the processing time can be reduced. However, since the sample image frame group is temporally sampled, an image of a desired frame is not always included. Therefore, the simplest method is to search for and display the sample image frame temporally closest to the desired frame. In the example of FIG. 2, the sample image frame of the sample image information 201 _n that is temporally closest to the desired frame indicated by the broken line is set as the display image frame.

In this case, the shift between the desired frame and the display image frame is determined by how long the sample image frame group is created by sampling. This shift has little problem if the sample image frame group is time-sampled at sufficiently short intervals. However, when there is a scene change, the sample image frame that is temporally closest to the desired frame is not always suitable as an image frame for display. That is,
Desired frame and if there is a scene change between the specimen image frames included in the closest specimen image information 201 _n to, specimen image included in the sample image data 201 _n-1 before one than the specimen image information 201 _n The frame is more suitable as a display image frame similar to the desired frame. According to the present embodiment, this problem can be solved by adding the scene change position information 202 as additional information to the spatiotemporal sample video metadata 102 as shown in FIG.

Referring to the flowchart shown in FIG. 6, a processing procedure for searching for a sample image frame representing a desired frame using the scene change information 202 as described above will be described. Here, the scene change information 20
2 is represented by a frame number at a scene change position (referred to as a scene change frame number).

First, when a frame number of a desired frame to be searched is given, a scene change frame number closest to the frame number is searched (step S2).
1).

Next, it is determined whether or not a desired frame number exists between the start frame number (of the original video data) and the scene change frame number searched in step S21 (step S22).

If it is determined in step S22 that the desired frame number is between the start frame number and the scene change frame number, the time between the start frame number and the scene change frame number is The closest (or spatially) sample image frame is searched (step S23).

If it is determined in step S22 that the desired frame number is not between the start frame number and the scene change frame number, the scene change frame number and the last frame number (of the original video data) are determined. In, a sample image frame temporally (or spatially) closest to the desired frame number is searched (step S24).

Then, the retrieved sample image frame is displayed as an image most similar to the desired frame (step S25), and the process ends.

(2) Retrieval of Sample Image Frame Next, a procedure for retrieving an image similar to a certain image for the spatiotemporal sample video metadata 102 will be described with reference to the flowchart shown in FIG.

First, a search target image R, that is, an image to be searched is presented (step S31).

Next, the spatiotemporal sample video metadata 102
, Sample image frames are sequentially acquired one by one (step S33).

The search target image R is normalized to the size of the sample image frame acquired in step S33 (step S34). This is because the sample image frames have different sizes one by one.

The sample image frame obtained in step S33 and the search target image R normalized in step S34
, For example, the sum of absolute value differences for each pixel is calculated (step S35).

It is determined whether or not the sum of the absolute value differences is equal to or smaller than a certain threshold (step S36). If the result of the determination in step S36 is that the sum of the absolute value differences is equal to or smaller than the threshold value, it is determined that the sample image frame acquired in step S33 is substantially the same as the search target image R, and the frame number of the sample image frame is changed. It is recorded as a search result (step S37).

The above series of processing is repeated until it is determined in step S32 that all the sample image frames have been obtained, and the processing ends.

Next, after the processing according to the procedure shown in the flowchart of FIG. 7 is completed, the search result is displayed as follows.

Based on the frame number of the sample image frame obtained as the search result in step S37, the searched sample image frame is displayed on the display unit 107 by the search engine / sample image display engine 105 in FIG.

Alternatively, based on the frame number of the sample image frame obtained as a search result in step S37,
When the original video data 101 is to be reproduced from the position of the frame number, the frame of the original video data 101 corresponding to the frame number of the sample image frame is determined using the correspondence table 103 (or the corresponding function table) shown in FIG. Look up the number. Then, by transmitting the information of the frame number and the display command to the controller 106, the image is reproduced from the corresponding frame of the original video data 101 using the video display engine 104 and displayed on the display unit 107.

(3) Fast-forward playback using spatio-temporal sample video metadata As shown in FIG. 2, in this embodiment, spatio-temporal sampled video metadata 102 includes additional information other than sample image information 201 In addition to the scene position information 202, screen change amount information 203 is described.

The screen change amount information 203 contains the original video data 1
01 is information indicating the amount of screen change between video frames that exist intermittently. For example, the sum of absolute value differences between frames is used. If the original video data 101 is MPEG-compressed video data, motion compensation between frames is used. Average of motion vector magnitude (average power) of the whole screen from the data of
Can be calculated and obtained. By attaching such screen change amount information 203 to the spatio-temporal sampling video metadata 102, it is possible to perform high-speed variable-speed playback.

JP-A-10-243351 (Japanese Patent Application No.
As described in “Video playback device”, the amount of screen change is kept constant by playing back the image slowly where the screen change is large and fast at the place where the screen change is small. A technique for realizing fast-forward playback that is easy to view is known. In this patent, it is assumed that there is a screen change for each frame, and that all frames are used. There is no mention of the case where the time is obtained discretely in time. In view of this, the present embodiment provides a method for realizing variable-speed reproduction in which a similar effect is obtained with respect to a temporally discrete sample image frame and a screen change amount.

First, with reference to a flowchart shown in FIG. 8, a basic processing procedure when performing variable speed reproduction using a sample image frame will be described.

First, a range for performing variable speed reproduction (in this case, fast forward reproduction) is specified (step S41). Let Fs be the start frame number and Fe be the end frame number of the variable speed playback range.

Next, the reproduction speed magnification m, that is, how many times the fast forward reproduction is to be performed is specified (step S42).

Next, the reproduction direction is specified, that is, whether fast-forward reproduction is to be performed by forward reproduction or reverse reproduction (step S43).

Next, the reproduction frame rate r [frame / second] of the sample image frame is designated (step S4).
4). The reproduction frame rate r differs depending on the television system. For example, in the case of NTSC, 30 [frame / frame]
Second], and 24 [frames / second] for PAL.

Here, assuming that the frame rate of the original video data 101 is R [frame / second], the number of frames to be skipped for the sample image frame group during variable speed reproduction is calculated based on this, as described later. (Step S
45).

Then, in order to reproduce the sample image frame at the reproduction frame rate r [frame / second], 1 / r
A sample image frame is acquired and displayed in a cycle of seconds (step S46).

In the case of forward reproduction, reproduction is started from the sample image frame number corresponding to the frame Fs, and is skipped in the direction of increasing the frame number. In the case of reverse reproduction, reproduction is started from the sample image frame corresponding to the frame Fe, and is skipped in the direction of decreasing the frame number.

Here, the processing in step S46 will be described in more detail. In the case of fast forward reproduction in the forward direction,
A sample image frame is obtained while increasing the frame number by (m × R / r) frames per cycle. That is, (m × R / r) is the number of frames to be skipped in the forward direction calculated in step S45, and step S46
In this case, the nearest sample image frame at the frame number of Fs + (m × R / r) × t is reproduced and displayed. Here, t is the number of cycles.

Similarly, in the case of fast forward reproduction in the reverse direction,
A sample image frame is acquired while decreasing by (m × R / r) frames per cycle. That is, (mxR
/ R) is the number of frames to be skipped in the reverse direction calculated in step S45, and in step S46, Fe− (m ×
The nearest sample image frame at the frame number of (R / r) × t is reproduced and displayed.

In this manner, variable speed reproduction at an arbitrary reproduction speed magnification can be performed using the sample image frame group. If there is no difference between the sample image frames taken out in each cycle, the same frame may be displayed continuously, thereby increasing the processing efficiency.

In the above description, the reproduction speed magnification m is constant unless changed by the user. Next, a method of performing smoother variable speed reproduction using the above-described screen change amount information 203 will be described. Is described. The basis of the variable speed reproduction is that the reproduction speed in the variable speed reproduction using the sample image frame is changed at every time according to the screen change amount information 203. For the sake of simplicity, consider a case where fast forward playback is performed on the entire original video data 101 without specifying a variable speed playback range.

First, the following parameters are defined.

Total number of frames of original video data 101: K
[Frame] Frame rate of original video data 101: R [frame /
Second] Reproduction frame rate of sample image frame: r [frame / second] Reproduction speed magnification: m Screen change amount information: Pi (i = 0,..., N) Weight given to reproduction speed corresponding to screen change amount: Wi Frame number of the original video data corresponding to the sample image frame: Fi (i = 0,..., N-1) Weight given to the reproduction speed corresponding to each frame of the original video data: Wj (j = 0,..., K) -1) Now, let L be the limit value of the amount of screen change of the sample image frame given to severe movement, and consider a value [Pi] that does not exceed the limit value L.

[Pi] = L, Pi> L, [Pi] = Pi, other cases (1) Also, the weight given to the reproduction speed in accordance with the screen change amount is Wi = [Pi]. I do.

Next, the weight for the reproduction speed of each frame will be considered. The weight Wi corresponding to the discrete reproduction speed is linearly interpolated to obtain Wj represented by the following equation.

Wj = Wi + (W (i + 1) −Wi) / (F (i + 1) −Fi) × t where t = 0,..., F (i + 1) −Fi, j = Fi,. i + 1) -1, i = 0,..., n-1 (2) When Wj is normalized such that the total sum of Wj becomes 1.0, W′j = Wj / ΣWj J = 0,..., K (3) Here, the number of display times N required for reproduction at a reproduction speed magnification m and a reproduction frame rate r [frame / second] is as follows.

N = K / (m × R / r) (4) When acquiring display image frames from a group of sample image frames in consideration of the weight given to the reproduction speed, assigning them to each sample image frame The weights W′j thus obtained are added, and the added value becomes Th = p / N (p = 0,..., N−1).
A sample image frame when the threshold value is exceeded is obtained.
That is, the nearest sample image frame corresponding to the frame number when the added value exceeds the threshold Th becomes the display image frame.

A display image frame is obtained in advance according to the above calculation, and a frame rate r [frame / second] is obtained.
If the screen change amount is large, the variable speed reproduction is performed later, and if the screen change amount is small, the variable speed reproduction is performed earlier. As a result, the image can be displayed at a desired reproduction speed magnification m. By using the above calculation, a video program of a certain time length can be reproduced within an arbitrary shorter time. By applying smoothing to the weight W'j given to the reproduction speed or performing special weighting in a scene change or a still image portion, a special effect can be further added to the variable speed reproduction.

Here, the case where the variable speed reproduction is performed on the entire original video data 101 has been described. However, the variable speed reproduction can be performed in the same manner in the case of the partial reproduction. That is, if W′j of the entire original video data 101 can be calculated, the problem of partial reproduction can be easily solved. Also, in the description of the case of variable-speed playback of the entire original video data 101, it is assumed that the screen change amount information exists in the start frame and the end frame of variable-speed playback. The screen change amount information may be used or a default value may be given for the calculation.

Hereinafter, with reference to the flowchart shown in FIG. 9, a specific processing procedure in the case where smoother variable speed reproduction is performed using the screen change amount information 203 as described above will be described. 9, the processing in steps S51 to S54 is basically the same as the processing in steps S41 to S44 in FIG.

That is, first, a range for performing variable speed reproduction (in this case, fast forward reproduction) with the screen change amount being fixed is specified (step S51). The start frame of the variable speed reproduction range is Fs, and the end frame is Fe. Next, the reproduction speed magnification m, that is, how many times the fast forward reproduction is to be performed is specified (step S52). Next, a reproduction direction is specified, that is, whether fast-forward reproduction is performed in forward reproduction or reverse reproduction (step S53). Next, the reproduction frame rate r [frame / second] of the sample image frame is specified (step S54).

Thereafter, the necessary number of display times N is obtained by the equation (4).
Is calculated (step S55). Further, the sum of the weights W′j shown in the equation (3) is Th = p / N (p = 0,
, N-1), the position of the sample image frame when it exceeds the threshold value, that is, the nearest sample image frame corresponding to the frame number when the added value exceeds the threshold value Th is calculated as the display image frame position. This is recorded in a table (step S56).

Then, in order to reproduce and display the sample image frame at the reproduction frame rate r [frame / second],
The display sample image frame is acquired and displayed using the table at a cycle of 1 / r second (step S5).
7).

In the case of performing the variable speed reproduction using the sample image frame group as described above, the reproduction speed is changed in accordance with the screen change amount, that is, the reproduction speed is reduced when the screen change amount is large, and the screen change amount is reduced. By lowering the reproduction speed at a small area, the above-described Japanese Patent Application Laid-Open No. 10-243351 is disclosed.
(Japanese Patent Application No. 09-042637), it is possible to realize easy-to-see fast-forward reproduction for a sample image frame while maintaining a constant screen change amount in the same manner as in the “video reproduction device”.

(4) Other Use Modes FIG. 10 shows sample image frames 501 and 50 near the scene change position (cut point) selected by the method described above.
This is an example in which 2,... Are displayed in a list. Since the image frame is not extracted from the original video data, such a list screen 500 can be created at high speed.

FIG. 11 shows an example in which the entire original image is displayed by a single bar 601 extending in the time axis direction, and a specified portion of the bar 601 is displayed by an enlarged bar 602.
In the enlarged bar 602, the image of the frame of the cut point included in the original video of this portion is displayed as a heading. Mouse cursor 6 over bar 602
03, the nearest sample image frame 6 most similar to the image frame at the designated point is considered in consideration of the position of the cut point.
04 can be selected and displayed as an icon. Since this processing can be performed at high speed, the icon image can be displayed in real time like a moving image by sliding the mouse icon left and right.

On the other hand, when considering the application of the monitoring system,
There is a need to efficiently find occasional events. For example, suppose that a background image is always shown on the monitoring screen, but an intruder is shown at some point. An intruder can easily find the difference image of the background image. At the same time as recording the video, the sample image frame is sampled coarsely in a place where there is no change in the screen,
Where the screen changes, the intruder can be reliably recorded by finely sampling in time. Information for managing a cut point or the like is stored as additional information on the screen on which the intruder appears, and a list can be displayed later. Also, only when there is an intruder,
By refining the spatial sampling of the specimen image frame, intruders can also be identified in the specimen image frame.

It is also effective to acquire a still image with higher definition than the original video when an intruder enters, and manage it as a sample image frame. When the resolution is insufficient for a normal video, it is possible to determine an intruder by using a still image with a higher resolution.

As described above, according to the present embodiment, in addition to the original video data, a sample image obtained by sampling the original video frame group at an arbitrary time interval and at an arbitrary spatial size. By recording sample image information consisting of frame group image data and attribute information and searching for sample image information instead of original video data, video search for a desired frame can be performed without burdening computer power or traffic. This can be easily performed. Further, by describing the scene change position information as supplementary information in the sample image information, it becomes possible to search for a sample image frame more similar to the desired frame. Furthermore, a desired frame can also be searched by obtaining the difference between the search target image and the image of each sample image frame, for example, the sum of absolute value differences, and searching for a sample image frame having a small value. Further, by reducing the reproduction speed when the screen change amount is large and by lowering the reproduction speed when the screen change amount is small, it is possible to realize easy-to-read variable speed reproduction with a constant screen change amount for the sample image frame. Is also possible.

Hereinafter, another embodiment of the present invention will be described.
In the description of the other embodiments, the same parts as those of the first embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted.

In the first embodiment, the spatiotemporal sample video metadata 102 has a plurality of sample image information 201 _{1 to} 201 _n , but a detailed description example thereof has not been described.
A second embodiment relating to a specific example of this description example will be described below.

FIG. 12 shows a description example of sample image information according to the second embodiment. Here, the sample image frame group is treated as one image (sample image), and the sample image information 701 is configured as a set thereof. Sample video is sample video information 7
01, the location may be described in the sample video information 701 by a URL or the like, or the sample video may be directly described as the sample video information 701.

The sample image information 702 is sample video information 701
Indicates the correspondence between the sample image frame in the sample video and the original video data frame, and is described in plurality according to the number of sample image frames included in the sample video. The sample image information 702 includes a media time 703 of the original video frame and a media time 704 of the sample video. The media time 703 of the original video frame indicates a frame of the original video corresponding to the sample image frame. As long as the frame of the original video can be uniquely determined, it may be a time such as a time stamp or a frame number. Also,
When the original video frame corresponding to the original video frame is obtained by the calculation, for example, when the original video frame is sampled at a constant interval, information necessary for the calculation (for example, the sampling interval) is described, and the media time 703 of the original video frame is described.
May be omitted. The sample video media time 704 indicates a specific sample image frame in the sample video indicated by the sample video information 701. The media time 704 of the sample video may be a frame number or the like as long as the sample image frame can be uniquely determined, or may be a time such as a time stamp when the sample video is treated as a normal video. Further, when the correspondence is sequentially performed with the sample video, it may be omitted.

FIG. 13 shows another description example of the sample image information. The sample image information 801 indicates the correspondence between each sample image frame and the original video data frame, and is described in plural according to the number of sample image frames. Specimen image information 801
Is composed of the media time 802 of the original video frame and the sample image data 803. The media time 802 is shown in FIG.
The frame position of the original video data corresponding to the sample image frame is shown similarly to the media time 703 in the description example shown by. Note that the main media time 802 may be omitted in the same manner as the media time 703. The sample images are individually prepared separately from the sample image data 801.
1 may be described by a URL or the like, or the sample image may be directly described in the sample image data 801 as sample image data. Further, instead of the sample image, another image such as an illustration showing the contents may be used as the sample image data.

FIG. 14 shows another description example of the sample image information. The description example shown in FIG. 14 includes both the description examples shown in FIG. 12 and FIG. Sample video information 90
Reference numeral 1 denotes sample video information 70 in the description example shown in FIG.
Similar to 1, the URL indicates the location of the sample image or the sample image itself. The sample image information 902 indicates the correspondence between each sample image frame and the original video data frame, and is described in plural according to the number of sample image frames. The sample image information 902 includes the media time 903 of the original video frame and the media time 904A of the sample video or the sample image data 9
04B. The media time 903 of the original video frame indicates a frame of the original video data corresponding to the sample image frame similarly to the media time 703 in the description example shown in FIG. Note that the main media time 903 may be omitted in the same manner as the media time 703. The media time 904A of the sample video is the same as the media time 704 in the description example shown in FIG. 12, and indicates a specific sample image frame in the sample video indicated by the sample video information 901. If the media time 904A is sequentially associated with the sample video, it may be omitted. The sample image data 904B is the same as the sample image data 803 in the description example shown in FIG. 13, and indicates the location of each sample image frame and the sample image frame itself.

According to the description example of FIG. 14, it is possible to replace a part of a sample image or add another sample image.

Next, a process for extracting sample image data at a desired media time will be described with reference to the description examples shown in FIGS. FIG. 15 shows the basic flow. In step S61, a media time of an original video frame corresponding to a desired sample image frame is input. The media time uniquely indicates a temporal position in the media, such as a time stamp or a frame number. In step S62, the first sample image information is extracted from the sample image information group described in the description examples of FIGS. Step S63
Then, the desired media time is compared with the media time of the original video frame included in the sample image information. If both are the same or the desired media time is later, the process proceeds to step S64, and the process proceeds to step S64. The sample image data shown is taken out. The method of extracting the sample image data differs depending on the description method. When the sample frame number is described, the corresponding sample image data of the sample video is extracted, and when the sample image data is described, it is used. If the media time included in the sample image information is later than the desired media time, the process proceeds to step S65, the next sample image information is extracted from the sample image information group, and the process returns to step S65.
Proceed to 63 to compare the media times.

FIG. 16 is a description example in which the attribute information of the sample image frame is added to the description examples shown in FIGS. Sample images use sample images of different sizes,
Since only a part of the original video data can be cut out and used as a sample image, an example for describing these parameters as attribute information is the description example shown in FIG.

The sample image frame group information 1001 is shown in FIG.
15 shows information according to the description example shown in FIG. The sample attribute information 1002 is attribute information of each sample image frame. According to the number of sample image frames included in the sample video,
Described multiple times. Sample attribute information 1002 is sample number 10
03, resolution information 1004, and area information 1005.

A sample number 1003 is a number corresponding to a specific sample image frame included in the sample image frame group indicated by the sample image frame group information 1001. If the sample frame number 1003 sequentially corresponds to the sample image frames in the sample image frame group, it may be omitted.

The resolution information 1004 indicates the resolution of the sample image frame indicated by the sample number 1003 with respect to the corresponding frame of the original video data. For example, it describes how many times the image has been reduced.

The area information 1005 indicates a corresponding slope in the corresponding frame of the original video data of the sample image frame indicated by the sample number 1003. If the sample image frame cuts out a part of the corresponding frame of the original video data, the area is described. If the sample image frame corresponds to the entire corresponding frame of the original video data, the region information may be omitted.

Although not shown, the attribute information may be described in each sample image information in the recording examples shown in FIGS.

FIG. 17 is an actual description example using the description method shown in FIG. It is assumed that an object exists only partially in the original video frame 1401. When a sample image frame of the original video frame 1401 is created, it is possible to create a sample image frame that reflects the contents of the image more by taking out and sampling only a part than sampling the entirety of both images. Therefore, the original video frame 140
A rectangular area 1402 within the area 1 is sampled, and sampling is performed so that the height and width are each halved to create a sample image frame 1403. A description example of the resolution information and the area information at this time is as shown in 1404.

FIG. 18 is a basic flow for displaying a list of sample image frames in response to a user request. Snub S
At 71, the user inputs a level to be displayed in a list. As an input method, a GUI such as a slider that changes continuously according to the display level may be used, or the input may be directly made by a number. Alternatively, an input device such as a wheel or a dial read by a computer or the like may be used.

In step S72, the number of sample images to be displayed in a list is calculated from the level value input in step S71. For example, assuming that the maximum display level is Lmax, the maximum display sample image frame number is Tmax, and the current display level is L, the display sample image frame number T = Tm
ax × L / Lmax.

In step S73, sample image frames to be displayed in a list are selected according to the number of display sample image frames. For example, sample image frames are selected at fixed time intervals or fixed frame intervals. Alternatively, when additional information such as cut point information is provided, a frame having higher importance such as a cut point or a top frame of a scene may be preferentially selected.

At step S74, a list of the selected specimen images is created and displayed.

FIG. 19 shows an interface for displaying a list of sample image frames using the basic flow shown in FIG. On a screen 1101, there are a slider 1102 for specifying a display level and a sample image list 1103. When the slider 1102 is moved to the position shown by the slider 1105 in the screen 1104 to increase the display level, the number of sample images displayed in the list increases as shown in the list display 1106. By using such an interface, the user can intuitively display as many sample images as necessary according to the content of the video.

FIG. 20 is an example of a screen display using the description example shown in FIG. By using the description example shown in FIG. 16, it is possible to handle sample image frames having different resolutions or sample image frames obtained by cutting out only a part of the screen. On the other hand, in the image, there are mixed regions such as a subtitle portion where it is desirable to perform sampling at a high resolution, and regions such as a background that are sufficient for sampling at a low resolution.
Therefore, a sample image frame group 1201 including sample image frames created from the same frame and having different resolutions and regions.
Is prepared, and these are superimposed and displayed as shown in a screen display example 1202, so that the subtitles can be displayed at a high resolution and the background can be displayed at a low resolution.

FIG. 21 is another example of a screen display using the description example shown in FIG. Image 1301 is a sample image frame sampled at low resolution. When the user points a region 1302 where a more detailed image is desired, such as a subtitle portion, with a mouse or the like, a sample image frame 1303 obtained by sampling only the region 1302 at a higher resolution is displayed by a pop-up or the like. Normally, the image 1301
Since a low resolution sample image frame is displayed, the size of the image can be reduced, and more images can be displayed in a list display or the like.

The present invention is not limited to the above-described embodiments, but can be implemented with various modifications.

[0137]

As described above, according to the image information description method of the present invention, search and display can be performed while confirming the contents of the video.

When a search is performed based on a sample image obtained by sampling original video data, a good video search can be performed even when a desired frame exists between scene changes.

Further, since the variable speed reproduction can be performed based on the sample image, the processing amount can be reduced, and the variable speed reproduction can be easily realized even on a device with a small computer power or on a network.

[Brief description of the drawings]

FIG. 1 is a diagram showing a system architecture according to an embodiment of the present invention.

FIG. 2 is a conceptual diagram showing the structure of original video data and spatio-temporal sample video metadata.

FIG. 3 is an explanatory diagram of sample image information included in spatiotemporal sample video metadata.

FIG. 4 is a diagram showing a management structure of specimen image information.

FIG. 5 is a time chart for explaining a description procedure of specimen image information.
9 is a flowchart illustrating a recording procedure of spatial sample video metadata.

FIG. 6 is a flowchart showing a procedure for searching for a sample image frame using scene change information included in spatiotemporal sample video metadata.

FIG. 7 is a flowchart showing a procedure for searching for a sample image frame for spatiotemporal sample video metadata;

FIG. 8 is a flowchart showing a procedure of variable speed reproduction using a sample image frame.

FIG. 9 is a flowchart showing a procedure of smooth variable-speed reproduction using a sample image frame and screen change amount information.

FIG. 10 is a view showing an example of a list display of sample image frames using scene change information included in spatiotemporal sample video metadata.

FIG. 11 is a diagram showing a display example of original video data and a sample image frame using spatiotemporal sample video metadata.

FIG. 12 is a diagram showing another description example of sample image information.

FIG. 13 is a diagram showing another description example of sample image information.

FIG. 14 is a diagram showing still another description example of sample image information.

FIG. 15 is a flowchart showing a search for sample image data using sample image information according to the description examples of FIGS. 12 to 14;

FIG. 16 is a diagram showing still another description example of sample image information.

FIG. 17 is a view showing a specific example of sample image information according to the description example of FIG. 16;

FIG. 18 is a flowchart illustrating an operation of displaying a sample image list in which the number of displayed images is changed according to a display level.

FIG. 19 is a diagram showing a change in a sample image list display when a display level is changed.

FIG. 20 is a diagram showing an example in which a plurality of sample images having different resolutions and regions are displayed in a superimposed manner based on the sample image information according to the description example of FIG. 16;

FIG. 21 is a diagram showing another example in which a plurality of sample images having different resolutions and regions are superimposed and displayed based on the sample image information according to the description example of FIG. 16;

 ──────────────────────────────────────────────────続 き Continuing from the front page (72) Inventor Yuji Mita 1st address, Toshiba-cho, Komukai-shi, Kawasaki-shi, Kanagawa Prefecture Inside the Toshiba R & D Center (72) Inventor Koji Yamamoto Toshiba, Komukai-shi, Kawasaki-shi, Kanagawa No. 1 town Toshiba Research & Development Center F-term (reference) GB36 GB37 GB38 HA21 HA29 HA33

Claims (26)

[Claims] 1. A sample image frame comprising: a plurality of sample image frames obtained by sampling video information composed of a plurality of video frames at an arbitrary time interval and an arbitrary size spatially; A description of attribute information for specifying a video frame of the video information corresponding to each of the image information. 2. The image information description method according to claim 1, wherein additional information relating to said video information is also described as said sample image information. 3. The image information description method according to claim 1, wherein the attribute information includes position information indicating a position on a time axis of a video frame corresponding to the sample image frame. . 4. The method according to claim 1, wherein the attribute information includes information on a size or a resolution of the sample image frame. 5. The image information description method according to claim 2, wherein the supplementary information includes scene change position information of the video information. 6. The image information description method according to claim 2, wherein the supplementary information includes information on a screen change amount of the video information. 7. The image information according to claim 1, wherein image data of the sample image frame or a pointer to the sample image frame is described as the sample image information. Description method. 8. A recording medium for storing the sample image information described by the image information description method according to claim 1. Description: 9. A desired video using sample image information on a plurality of sample image frames obtained by sampling video information consisting of a plurality of video frames at an arbitrary time interval and an arbitrary size spatially. In the video search method for searching information, attribute information including at least first position information indicating a position on a time axis to specify the video frame corresponding to the sample image frame is described as the sample image information, Based on the first position information and second position information indicating a position on the time axis of the desired image, a sample image frame having first position information closest to the second position information is searched for. A video search method characterized by the following. 10. A desired image using sample image information on a plurality of sample image frames obtained by sampling video information consisting of a plurality of video frames at an arbitrary time interval and an arbitrary size spatially. In the video search method for searching, the attribute information including at least first position information indicating a position on a time axis for specifying the video frame corresponding to the sample image frame is described as the sample image information, The scene change position information of the video information is described as incidental information in the sample image information, the first position information, the second position information indicating a position on the time axis of the desired video, and the scene change position information The sample image frame having the first position information closest to the second position information temporally before or after the scene change position information is determined based on A video search method characterized by searching. 11. A desired video using sample image information on a plurality of sample image frames obtained by sampling video information consisting of a plurality of video frames at an arbitrary time interval and an arbitrary size spatially. In the video search method for searching for, the attribute information including at least position information indicating a position on a time axis for specifying the video frame corresponding to the sample image frame is described as the sample image information, and the desired image is described. A video image search method for searching for a sample image frame whose difference from the sample image is equal to or less than a predetermined threshold value. 12. The video search method according to claim 11, wherein the position information described for a sample image whose difference from the desired image is equal to or less than a predetermined threshold is recorded as a search result. 13. A method for producing an image by using sample image information on a plurality of sample image frames obtained by sampling video information consisting of a plurality of video frames at an arbitrary time interval and an arbitrary size spatially. In a video reproduction method for performing variable speed reproduction, attribute information including at least position information indicating a position on a time axis for specifying the sample image frame and the video frame corresponding to the sample image frame is described as sample image information. The sample image information describes the information of the screen change amount of the video information as supplementary information, and the variable speed reproduction of the video is performed by changing the reproduction speed of the sample image frame according to the information of the screen change amount. A video reproducing method characterized by performing. 14. A desired video using sample image information on a plurality of sample image frames obtained by sampling video information consisting of a plurality of video frames at an arbitrary time interval and an arbitrary size spatially. A sample image information comprising attribute information including at least first position information indicating a position on a time axis for identifying the image frame corresponding to the sample image frame; Supplementary information indicating scene change position information of the video information added to image information, the first position information, second position information indicating a position on the time axis of the desired video, and the scene change position Based on the information, the sample image frame having the first position information closest to the second position information temporally before or after the scene change position information is detected. A video search device, comprising: a search engine for searching. 15. A desired video using sample image information on a plurality of sample image frames obtained by sampling video information consisting of a plurality of video frames at an arbitrary time interval and at an arbitrary spatial size. In the video search device for searching for, sample image information consisting of attribute information including position information indicating at least a position on the time axis to identify the video frame corresponding to the sample image frame, and the desired image A search engine for searching for a sample image frame whose difference is equal to or less than a predetermined threshold value. 16. A method for producing an image by using sample image information on a plurality of sample image frames obtained by sampling video information composed of a plurality of video frames at an arbitrary time interval and an arbitrary size spatially. In a video playback apparatus that performs variable-speed playback, a sample including image data of the sample image and attribute information including position information indicating at least a position on a time axis for specifying the video frame corresponding to the sample image frame. Image information, incidental information indicating a screen change amount of the video information added to the sample image information, and a variable speed of a video by changing a reproduction speed of the sample image frame according to the information of the screen change amount. A video playback device, comprising: a display engine that performs playback. 17. The method according to claim 17, wherein the sample image frame includes an image obtained by sampling only an arbitrary portion of one image of the video information at an arbitrary time interval and an arbitrary spatial size. The image information description method according to any one of claims 1 to 7, characterized in that: 18. The method according to claim 18, wherein the sample image frame includes an image obtained by sampling only an arbitrary portion of one image of the video information at an arbitrary time interval and an arbitrary spatial size. The video search method according to any one of claims 9 to 12, characterized in that: 19. The method according to claim 19, wherein the sample image frame includes an image obtained by sampling only an arbitrary portion of one image of the video information at an arbitrary time interval and an arbitrary spatial size. 14. The video reproducing method according to claim 13, wherein: 20. The sample image frame includes a sample obtained by sampling only an arbitrary portion of one image of the video information at an arbitrary time interval and an arbitrary spatial size. Claim 14 or Claim 15 characterized by the above-mentioned.
The video search device according to the above. 21. The sample image frame includes a sample obtained by sampling only an arbitrary portion of one image of the video information at an arbitrary time interval and an arbitrary spatial size. 17. The video playback device according to claim 16, wherein: 22. The image information description method according to claim 1, wherein the plurality of sample image frames are stored as one piece of video information. 23. The video search method according to claim 9, wherein the plurality of sample image frames are stored as one video information. 24. The video reproducing method according to claim 13, wherein said plurality of sample image frames are stored as one video information. 25. The method according to claim 14, wherein the plurality of sample image frames are stored as one piece of video information.
Or the video search device according to claim 15. 26. The video reproducing apparatus according to claim 16, wherein said plurality of sample image frames are stored as one video information.