JP3060919B2 - Compressed video decoding / display device and simplified compressed video editing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a compressed moving picture decoding / reproducing apparatus for retrieving and browsing the contents of stored compressed digital moving picture data at high speed, and to easily and easily store stored digital compressed moving picture data. The present invention relates to a simplified compressed moving image editing apparatus for providing a method of searching and browsing without waste.

[0002]

2. Description of the Related Art Along with improvements in image compression algorithms and hardware, large capacity and high speed storage devices, it has been desired to provide an environment in which moving image information can be easily handled. Moving image information has an enormous amount of data, and it takes a time equal to the recording time of the moving image information to browse and search all information, so when providing a moving image database to a user, At the same time, it is necessary to provide appropriate search / browsing means.

[0003] As one of the search methods for the content of a moving image, there is a moving image search method disclosed in Japanese Patent Laid-Open No. 2-113790.
In this method, in the method of searching for moving image information, a portion of the moving image information representing the feature of the search image for each search image unit is extracted as a scene, a collection of scenes is edited as a menu image, and the target image is extracted from the menu image. This is a moving image search method characterized by searching for a search image. As shown in this example, when the video is divided with the part where the content of the video changes as a boundary, and a range of video that is continuous in terms of content and meaning is set as a scene, a list of representative still images in each scene is displayed. The viewer / viewer can easily grasp the contents of the moving image. However, scene segmentation and editing work by visual inspection
Since it takes a great deal of time and effort,
No. 9, the moving image processing apparatus disclosed in Japanese Patent Application Laid-Open No. Hei 6-89545, the method for detecting a scene change part in a moving image editing process, and the moving image scene detecting apparatus disclosed in Japanese Patent Application Laid-Open No. Hei 6-236439.
Numerous scene change detection devices and methods have been proposed.

[0004] However, these scene change detection methods mainly use a correlation between adjacent frames to detect a portion where the video content greatly changes.
There is still insufficient aspect in the detection accuracy, and at present, automatic scene change detection can be said to be an auxiliary means for editing moving image information. In addition, in the edited video, even if adjacent frames greatly change, it is more meaningful for end users such as searchers and viewers to index them as a single scene in terms of meaning and content. It may be easy to use. In other words, instead of displaying the index images created from the results of automatic scene change detection as a list, the editor makes appropriate corrections such as deleting indexes, integrating multiple indexes, and adding indexes as necessary. It is more preferable to provide the user with the index information of the moving image stream. For that purpose, means for editing the result of automatic scene change detection is required.

It is desirable that not only a list of scene head images obtained by scene change detection be displayed as index images, but also that a moving image be reproduced immediately from an index image arbitrarily specified by the user. JP-A-6-1
The broadcast receiving / recording / reproducing apparatus disclosed in Japanese Patent Laid-Open No. 05280 regenerates program information immediately by reproducing a thinned-out image of recorded program information and selecting program information from the thinned-out image. In this apparatus, in order to search for an image in a compressed moving image file corresponding to a thinned image, an address where each thinned image is recorded is recorded, and decoding of the compressed moving image data is started by referring to this address. The position can be determined and the movie can be played immediately. However, in this method, only the immediate reproduction starting from the thinned image can be performed, and in order to perform the immediate reproduction of an arbitrary designated frame, address information on all the frames is required. Also, a compressed moving image of an inter-frame predictive coding method in which an intra-coded frame is included in a stream represented by MPEG which is an international standard of the moving image coding method and a code for random access is inserted. In the MPEG system stream which requires a decoding start frame number, a decoding start stream position, and a display start frame number information, and further, packetizes a video stream / audio stream and multiplexes them into one stream, Position information for starting system separation is also required.

[0006] As another immediate reproduction method of a compressed moving image,
There is a moving picture coding apparatus disclosed in Japanese Patent Application Laid-Open No. 6-326998. In this apparatus, in order to smoothly and easily play back encoded video data from the middle or at high speed, (MP
A method of inserting a value indicating a difference from the actual start position into the target start position of the GOP (defined by the EG) and reading the target start position of the GOP at the time of decoding is adopted. With this method, it is not necessary to analyze the MPEG video stream on a byte-by-byte basis, and it is possible to access the beginning of a GOP at high speed. However, a special encoder is required, and an MPEG stream created by another encoder or an MPEG system stream can be used. Cannot be applied.

In the image reproducing apparatus disclosed in Japanese Patent Laid-Open No. 6-54292, all or arbitrary I pictures (intra-frame codes) in a moving image sequence are obtained in order to obtain still image information from a moving image database. The position of the converted frame is stored as still image position information. This apparatus is characterized in that it is not necessary to prepare a still image file for displaying a list for browsing and searching for moving images, separately from moving image data.
A problem with this method is that it is possible to decode and display a specific frame included in a moving image at high speed only with the moving image compressed stream and the still image position information. The frame is not always suitable for grasping the moving image content. In addition, whether to select a method of creating a still image file for list display in advance or a method of decoding a desired still image from moving image data in response to a still image display request depends on the storage capacity of the database, This should be determined in consideration of the performance of the video decoding device, the response time to the user, and the like, and it cannot be said that either is superior.

Further, in the moving image database, a text-based search means using a keyword is also required. In the moving image display device disclosed in JP-A-6-162116, M
The present invention provides a simple moving image display device capable of searching for and displaying desired data from a large amount of moving image data accumulated by performing PEG-based predictive encoding.
Bibliographic information is embedded as user data in the EG stream, and a search can be performed using this bibliographic information. The bibliographic information is text-based information such as the title, contents, and playback time length of the moving image data added manually. In this method, a search using a keyword is possible, but it is desirable to add a function for visually searching and browsing moving image contents such as an index image list display for checking the search result.

[0009]

In a moving image database in which compressed moving images are stored, it is necessary to provide appropriate search / browsing means to end users who search / browse. As a search / browsing means, there is a method of extracting a frame image whose moving image content has largely changed due to automatic scene change detection and displaying the frame image in a list. This method allows information providers to create index information that automatically records scene change detection results without hassle,
Moreover, this is an effective means that allows the user to instantaneously grasp the contents of the moving image. However, the current automatic scene change detection is incomplete because it is not based on the video content. In order to compensate for this and provide index information without excess or deficiency, a means for easily re-editing the automatically created index information is required. In addition, not only means for visually searching and browsing the contents of one compressed moving image stream like index image list display,
A text-based search means for searching for a desired stream or a scene in the stream from a plurality of compressed moving image streams is also required.

In some cases, a plurality of compressed moving image streams in a compressed moving image database are re-edited according to the purpose and provided to an end user. However, compressed video data of the inter-frame prediction coding method represented by MPEG, which is an international standard of the video coding method, and compressed data of video and audio such as MPEG system streams are multiplexed into one stream. In the case of handling data, there is a limit to re-editing in units of frames if the compressed moving image stream remains unchanged. Therefore, in order to obtain the edited result as a new compressed moving image stream, a sophisticated moving image encoding device / decoding device is required. In addition, saving as a new stream every time re-editing requires a large amount of storage space. It is considered that it is desirable to be able to reproduce the video according to the edited result only with the edited file.

The functions of the compressed moving image database include the above-mentioned functions. In both cases of re-editing the index information and searching / browsing the database, a function of immediately reproducing the frame from an arbitrary designated frame is provided. Is required as one of the important user interfaces. If the time required to start playback does not matter, determine the decoding start position by analyzing the code from the beginning of the stream, or determine the seek amount appropriately based on the total number of frames and the specified frame number, and A method of detecting the decoding start position is conceivable. In order to minimize the time required to start playback, the correspondence between the frame number and the decoding start position of the stream must be determined in advance. In the compressed moving image stream of the fixed-length coding method, random access in units of frames is possible if the coding length of one frame is known, and in the compressed moving image of the intra-frame coding method of the variable-length coding, If the addresses are recorded, random access in frame units is possible. On the other hand, in the case of a compressed moving image stream of an inter-frame predictive encoding method such as MPEG or data in which a video / audio stream is multiplexed into one stream, compression over a plurality of frames is performed. -A special frame-based random access method that takes into account that audio is multiplexed is required.

SUMMARY OF THE INVENTION The present invention solves the above-mentioned problems, and provides a method for compressing a moving image or a compressed moving image of an inter-frame predictive encoding system represented by MPEG which is an international standard of the moving image encoding system.
Compressed video decoding with high-speed cueing function for multiplexing video / audio compressed data such as EG system stream into one stream
It is an object of the present invention to provide a display device and a simplified compressed moving image editing device that can easily edit a compressed moving image using the compressed moving image decoding / display device.

[0013]

According to the compressed moving picture decoding / display apparatus of the present invention, an MPEG video stream, M
It has a cue reproduction table creating means that can determine a decoding start position and a reproduction start position in the stream from the frame number corresponding to the PEG system stream. An index image decoding / displaying means for displaying a still image of a scene head frame as an index image based on a scene change detection result file described in a predetermined format; And an index image list display function for displaying a list of index images by the index image decoding / displaying means, and a cueing reproduction means for searching for an index image arbitrarily selected from the list of index images.

The simplified compressed moving picture editing apparatus of the present invention has the same function as the above-mentioned compressed moving picture decoding / displaying apparatus, and confirms the result of a scene change using an index picture list display function, a cue reproduction function and the like. Index image modifying means for changing / deleting / adding an index image; scene change result modifying means for modifying a scene change detection result file in accordance with the index image modification; and joining a plurality of scenes of a plurality of files in a desired order. Based on the simple editing information file created by the simple editing information file creating means for creating a file recording information for reproducing the video,
Compressed video decoding / display means capable of reproducing a plurality of scenes of a plurality of compressed video streams in a designated order; a variable length search keyword can be added to a header of a cue reproduction table; and a scene change detection result file It is characterized by comprising a search keyword registration means capable of adding a search keyword to each of the scene information therein.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS According to the first aspect of the present invention, there is provided a reproduction section storing means for storing a reproduction start / end frame number of a designated compressed moving image stream and outputting the number. The information necessary for the cue reproduction is read out by referring to the cue reproduction table that has been read out, and the decode information reading means for outputting the information and the output of the reproduction section storage means and the output of the decode information reading means are summarized. Decoding information storage means for storing the compressed moving image stream based on the decoding information, a stream transmitting means for starting the stream transmission from the position, and decoding the compressed moving image stream transmitted from the stream transmitting means. Decode according to, start video display when the playback start frame is reached, and decode when the playback end frame is reached. A video decoding means having a function of ending the video display, and a function of outputting a frame still image when the reproduction start / end is the same value, including intra-frame predictive encoded data in the stream, for random access. This is a compressed video decoding / display device corresponding to an inter-frame prediction coded compressed video stream in which a code is inserted, using a cue reproduction table created in advance, and irrespective of the stream position, This has an effect that high-speed cueing reproduction and a specified section reproduction from an arbitrary designated frame having a substantially constant cueing time are possible.

According to a sixth aspect of the present invention, there is provided a playback section storage means for storing playback start / end video frame numbers, and a playback section storage means for storing playback start / end audio frame numbers. Read the cue playback table corresponding to the MPEG system stream
Decode information reading means for reading out information necessary for cue reproduction in response to a cue reproduction command based on a video frame or an audio frame, and outputting the information, and output and decode information read means of a reproduction section storage means And decode information storage means for storing the same, and seeks the compressed moving image stream based on the decode information stored in the decode information storage means,
After outputting the pack header information read from the stream to the decode information storage means, a stream sending means for sequentially sending video / audio packets separated by the system, and a video stream received from the stream sending means are decoded, and the playback start frame number is set. A video-video decoding means having a function of starting video display when reaching, further ending decoding / video display when a playback end frame is reached, and a function of outputting a frame still image when the playback start / end is the same value, The audio stream received from the stream transmission means is analyzed, and when the decoding start frame is reached, audio playback is started, and further, when the playback end frame is reached, the audio decoding means for terminating decoding is stored in the decoding information storage means. Based on the information - having synchronous playback means to synchronize the audio playback, the international standard of video coding MP
Compressed video decoding /
It is a display device that uses a cue reproduction table created in advance and enables high-speed cue reproduction and a specified section reproduction from an arbitrary designated frame having a substantially constant cue time regardless of the stream position. It has the action of:

According to a fifteenth aspect of the present invention, a scene change detection result file in a predetermined format in which a frame number of a first frame of each scene in a compressed moving image stream is recorded, and an index image list display instruction is provided. Means for outputting a scene change information storage means, a cue-reproducible compressed moving image decoding / display means, and a cue image which is sequentially output from the cue-reproducible compressed moving image decoding / display means. Index image list display means to be displayed, playback section determination means for determining a playback section corresponding to an index image arbitrarily selected from the index image group displayed in the list by referring to the scene change information, and a moving image. Compression having display content control means for determining an image section reproduction instruction and an index list display instruction and controlling display contents An image decoding / display device that has a function of displaying a list of index images created from a compressed moving image stream without creating an index image file in advance, and playing a moving image from an arbitrarily selected index image. .

[0018] The invention according to claim 18 of the present invention is a moving image playback section storage means for storing a plurality of continuous index images selected from the list of index images as one moving image playback section. A playback order determining means for determining a playback order of a plurality of set video playback sections, and a video playback section stored in the video playback section storage means as one scene, and And a simple editing information file creating means for extracting the last frame number and arranging the simple editing information based on the playing order stored in the playing order determining means. Providing end-users, such as viewers and searchers, with a deep video with unnecessary content removed without creating a new compressed video stream as a result Has the effect of being able to create a simple editing information file that.

An embodiment of the present invention will be described below with reference to FIGS. (Embodiment 1) FIG. 1 shows an apparatus for creating a table for cueing and reproducing a compressed moving picture stream. This is for an inter-frame predictive coded compressed video stream that contains intra-frame predictive coded data in the stream and has a code for random access inserted, and is an international standard for video coding. MPEG video stream. Here, the compressed video stream is MPEG
Description will be made assuming that the video stream is a video stream.

The compressed moving picture stream cueing / reproducing table creating apparatus shown in FIG. 1 comprises code detecting means, frame number calculating means, header information creating means, and table creating means. Each part in FIG. 1 will be described in order. 101
Reads a compressed video stream, sequentially detects a GOP code serving as a reference for random access included in the stream, a picture code added to the head of each encoded frame, and the like. Is a code detecting means for storing the position of. 10
2 counts the number of picture codes detected by the code detection means 101, and every time a GOP code is detected,
Frame number calculating means for outputting the cumulative frame number immediately before each GOP code. 103 is a code detecting means 1
The header information creating means creates a header describing the total number of frames, the reproduction time, the parameters at the time of decoding, and the like, based on the output of 01 and the number of frames calculating means. 104
Is a code detecting means 101, a frame number calculating means 10
2. A table file creating unit that creates a cue reproduction table using the output of the header information creating unit 103 and outputs it as a file.

The operation of the compressed moving picture stream cueing / playback table creating apparatus having the above configuration will be described. First, the GOP code and the picture code are detected by the code detecting means 101. The code length of the GOP code and the picture code is 4 bytes. The code detecting means 101 includes a buffer having an appropriate length. The compressed moving image stream is read into the buffer, and the stream is analyzed in units of 4 bytes while shifting the pointer in units of bytes. In addition, decoding parameters and the like included in the sequence layer of the MPEG video stream are stored as necessary.

When the GOP code and the picture code are detected by the code detecting means 101, the pointer indicating the analysis position is shifted by 4 bytes. When a picture code is detected, a picture code detection signal is sent to the frame number calculation means. Since there are I, P, and B pictures in the picture, a signal that distinguishes these may be sent as needed. On the other hand, the frame number calculation means 102 counts up the number of frames each time it receives a picture code detection signal from the code detection means 101.

When a GOP code is detected by the code detecting means 101, a GOP code detection signal is sent from the code detecting means 101 to the frame number calculating means 102, and at the same time, a table creation means 104 is sent from the head of the stream. Outputs the number of offset bytes to the GOP code position. Accordingly, the frame number calculating means 102
Outputs the cumulative number of frames from the head of the stream to the GOP code to the table creation means. In the table creation means 104, GO
A record is created with a set of the number of offset bytes up to the P code position, and is sequentially added to the table file. When the end of the stream is confirmed by the code detecting means 101, the frame number calculating means 102
A stream analysis end signal is sent to the header information creating means 103 and the table creating means 104. In the header information creating means, the decoding parameters stored in the code detecting means 101 and the frame number calculating means 10
2, the total number of frames and the like stored in the storage unit 2 are collected in accordance with a predetermined format, and are output as header information. The table creation means adds the header information to the table file and ends the processing.

However, in the above description, each time the frame number calculating means 102 receives the picture code detection signal from the code detecting means 101, it counts up the number of frames and calculates the cumulative number of frames from the head of the stream to the GOP code. However, TC (Time) in the GOP header
Code) and TR (Temporal Refere) in the picture header
nce) may be used to calculate the cumulative number of frames.

Table 1 is an example of an MPEG video stream cue reproduction table file. In this example, three bytes are prepared in the cumulative number-of-frames recording area, and four bytes are prepared in the offset byte number (hereinafter, abbreviated as the offset byte number) from the head of the stream to the GOP code. The number of allocated bytes may be changed according to the length of the compressed moving image stream to be handled.

[0026]

[Table 1]

As an example, a method of cueing and reproducing an MPEG video stream from the 100th frame using the table file of Table 1 will be described. First, the numerical value of the accumulated frame number recording area of the table file is checked in order from the top, and a record having a value of 100 or more is searched. The record larger than 100 is the eighth 105, and the offset byte number of that record is 190038 bytes. This is because when a stream is sought by 190038 bytes, a GOP code exists at that position,
When decoding is started from the OP code and displayed, 106
This means that reproduction is performed from the frame (because 105 frames exist immediately before the GOP code). That is, in order to reproduce from the 100th frame, it is necessary to refer to the record immediately before the table file. The previous record has a cumulative frame number of 90 and an offset byte number of 142,350. Then, the stream is sought for 142350 bytes, and the GOP
Start decoding from code. However, the decoded image is not displayed immediately. The frame number of the frame that has started decoding is 91. From here, the number of frames is counted, and when the 100th frame is reached, display is started.

As described above, the cue reproduction table file shown in Table 1 contains the intra-frame prediction coded data in the stream and the inter-frame prediction coded compressed moving image stream in which the code for random access is inserted. On the way
When playing from, the code for random access (M
It utilizes the fact that decoding must be started from a GOP code in a PEG video stream, and is for performing high-speed cue playback from a designated playback start frame. A variable length area in which the title of the stream and the search keyword are described may be prepared in the header of the table file so that the table file can be used for searching the moving image stream in the database.

FIG. 2 shows an MPEG system stream cue reproduction table creating apparatus. This is an extension of the compressed moving picture stream cueing / playback table creating apparatus of FIG. 1 to correspond to an MPEG system stream in which a video stream and an audio stream are multiplexed. Here, the target compressed moving image stream is an MPEG system stream, but it is also possible to adapt to a multiplexed stream similar to this.

The MPEG system stream cue reproduction table creating apparatus shown in FIG. 2 comprises a system code detecting means,
It comprises a video stream analyzing means, an audio stream analyzing means, and a table creating means. Each part in FIG. 2 will be described in order. Reference numeral 201 denotes a system code for reading an MPEG system stream, detecting a pack start code and a packet start code included in the stream, storing a stream position of the code, and transmitting a video packet and an audio packet separated from the system. It is a detecting means. Reference numeral 202 denotes a video packet analysis unit that analyzes a plurality of video packets transmitted from the system code detection unit 201 as one video stream. An audio packet analysis unit 203 analyzes a plurality of audio packets transmitted from the system code detection unit 201 as one audio stream. Reference numeral 204 denotes a table creation unit that creates a cue reproduction table based on outputs from the video packet analysis unit 202 and the audio packet analysis unit 203. The video packet analyzing means 202 is comprised of a video packet code detecting means, a video frame number calculating means, a video stream information storing means, and a video cueing / playback record creating means. Reference numeral 205 denotes a video packet code for analyzing a plurality of video packets transmitted from the system code detection means as one video stream, and detecting a picture code included in the stream and a video packet including a GOP code. It is a detecting means.
206 counts the number of picture codes detected by the code detection unit 205 in the video packet, and every time a GOP code is detected, the cumulative number of frames up to immediately before each GOP code and the number of frames after the GOP code in the packet It is a video frame number calculation unit that outputs the number of picture codes. 207 is a code detecting means 20 in the video packet.
5 is a video stream information storage unit for storing information such as a parameter at the time of decoding the video stream and a total number of frames, which are stored in 5. Each time a video packet including a GOP code is detected by the video packet code detecting means 208, the number of frames in the packet, the number of accumulated frames, the absolute number of offset bytes from the head of the stream to the pack header, and the number of bytes from the pack header to the packet header The relative offset bytes of
This is a video cueing / playback record creating means for creating a video cueing / playback record.

The audio packet analysis means 203 includes an audio packet code detection means, an audio frame counting means, an audio stream information storage means,
It is composed of audio cue reproduction record creation means. 209 analyzes a plurality of audio packets transmitted from the system code detection means 201 as one audio stream, and AAU (Au)
(dio Access Unit) is a code analysis means in an audio packet for detecting a header. Reference numeral 210 denotes an audio frame number calculation unit that counts the number of AAU header codes detected by the audio packet code detection unit 209 and outputs the accumulated AAU number immediately before each audio packet code and the number of AAUs in the packet. . 21
Reference numeral 1 denotes an audio stream information storage unit that stores information such as a parameter at the time of audio stream decoding stored in the audio packet code detection unit 209 and a total number of frames. Reference numeral 212 denotes a frame in an packet (AAU) each time one packet analysis is completed by the code in the audio packet detecting means.
An audio cueing / playback record for compiling an audio cue / playback record by summarizing the number, the accumulated frame (AAU) number, the absolute offset byte number from the stream head to the pack header, and the relative offset byte number from the pack header to the packet header It is a creation means.

The operation of the MPEG system stream cueing / playback table creating apparatus having the above configuration will be described. First, the system code detecting means 201 detects a pack header code and a packet header code. The code length of the pack header code and the packet header code is 4 bytes. The system code detection means has a buffer of an appropriate length inside,
The EG system stream is read, and the stream is analyzed in units of 4 bytes while shifting the pointer in units of bytes. When the GOP code and the picture code are detected, the pointer indicating the analysis position is shifted by 4 bytes. Also, the stream position of the pack start code and the stream position of the packet start code are stored, and the information in the system header is stored in the table creating means 20.
4 is output. When the system code detection means 201 detects a video packet header, it sends a video packet to the video packet analysis means 202, and when an audio packet header is detected, sends an audio packet to the audio packet analysis means 203. In addition to the packet transmission, the absolute offset byte number from the head of the stream to the pack start code of the pack to which each packet belongs and the relative offset byte number from the pack start code of the pack to which each packet belongs to each packet start code are output. These two offset byte numbers are stored in the video packet analyzing means 202 or the audio packet analyzing means 203.

Here, the video packet analyzing means 202
Will be described. The video packet analysis means 202
The video packets sequentially transmitted from the system code analyzing means 201 are analyzed as one video stream. However, the code detection means 205 in the video packet performs an analysis conscious of the packet boundary, and
If the P code and the picture code are divided into two packets, it is determined that they belong to the previous packet.
Also, the absolute offset byte number from the head of the stream to the pack start code of the pack to which each packet belongs and the relative offset byte number from the pack start code to the packet start code of the pack to which the packet belongs are stored.

When a picture code is detected by the video packet code detecting means 205, a picture code detection signal is transmitted to the video frame number calculating means 206. Since there are I, P, and B pictures in the picture, a signal that distinguishes these may be sent as needed. On the other hand, in the video frame number calculation means 206,
Each time a picture code detection signal is received, the number of frames is counted up.

When the GOP code is detected in the video packet code detecting means 205, a GOP code detection signal is transmitted to the video frame number calculating means 206, and at the same time, a video cueing / playback record generating means 208 The absolute offset byte number from the head of the stream to the pack start code of the pack to which the packet belongs and the relative offset byte number from the pack start code of the pack to which the packet belongs to each packet start code are output. Accordingly, the video frame number calculation means 206 outputs the accumulated frame number immediately before the GOP code and the number of picture codes existing after the GOP code in the packet including the GOP to the video cueing / playback record creation means 208. I do. The video cueing / playback record creating means 208 creates a video cueing / playback record using the transmitted offset byte number, frame number, and the like as a set, and sequentially creates the table file creation means 204.
Output to

The video cueing reproduction record includes the number of picture codes existing after the GOP code in the packet including the GOP (hereinafter abbreviated as the number of frames in the packet), the cumulative number of frames from the head of the stream to immediately before the GOP code, It is a set of the absolute offset byte number from the head to the pack start code and the relative offset byte number from the pack start code to the packet start code, and is created only for the packet including the GOP code.

The video stream information storage means 20
Numeral 7 is for storing information such as a parameter at the time of decoding the video stream received from the video packet code detecting means 205 and the total number of frames received from the video frame number calculating means 206. These are stored in the header of the table file. It is described.

Here, the audio packet analyzing means 2
03 will be described. Audio packet analysis means 2
In step 03, audio packets sequentially transmitted from the system code analysis means 201 are analyzed as one audio stream. However, the code detection means 209 in the audio packet performs an analysis in consideration of the packet boundary, and when the AAU header code is divided into two packets, determines that the AAU header code belongs to the previous packet. Also, the absolute offset byte number from the head of the stream to the pack start code of the pack to which each packet belongs and the relative offset byte number from the pack start code to the packet start code of the pack to which the packet belongs are stored.

Audio packet code detecting means 20
In 9, when an AAU header code is detected, an AAU header detection signal is sent to the audio frame number calculation means 210. On the other hand, the audio frame number calculation means 210 counts up the number of frames each time it receives the AAU header detection signal.

Audio packet code detecting means 20
In step 9, when the analysis of one packet is completed, the code detection unit 209 in the audio packet sends a packet analysis end signal to the number-of-audio-frames calculation unit, and sends each packet from the head of the stream to the audio cueing / playback record creation unit. It outputs the absolute offset byte number from the pack start code of the pack to which the pack belongs to and the relative offset byte number from the pack start code of the pack to which each packet belongs to each packet start code.
Accordingly, the audio frame number calculation means 210
Cumulative A from the beginning of the stream to the end of the target packet
The number of AUs and the number of AAUs in the packet are output to the audio cueing / playback record creation unit 212. (However, the cumulative number of AAUs may be defined as the number of AAUs from the head of the stream to immediately before the packet code.) The audio cueing / playback record creation unit 212 calculates the transmitted offset byte number, AAU number, and the like as one. Records are created as a set and are sequentially output to the table file creation means 204.

The audio cueing record is a set of the number of AAUs in the packet, the number of accumulated AAUs, the number of absolute offset bytes from the head of the stream to the pack start code, and the relative number of offset bytes from the pack start code to the packet start code. This is created for every audio packet.

The audio stream information storage means 211 includes a code detection means 209 in the audio packet.
This is for storing information such as a parameter for decoding an audio stream received from the device and a total number of frames received from the number-of-audio-frames calculating means 210, and these are described in a header of a table file.

The table file creating means 204 includes identifiers for distinguishing between video and audio in the cue reproduction records sent from the video cue reproduction record creation means 204 and the audio cue reproduction record creation means 212. Is added to the table file sequentially.

When the end of the stream is confirmed by the system code detecting means 201, the video packet analyzing means 202, the audio packet analyzing means 20
3, and a stream analysis end signal is sent to the table creation means 204. Upon receiving this signal, the table creation unit 204 receives the information stored in the system code detection unit 201, the video stream information storage unit 207, and the audio stream information storage unit 211, creates a header, and writes this into a table. Append to file.

Table 2 shows an example of an MPEG system stream start reproduction table file. In this example,
1 bit for video / audio identifier, 7 bits for the number of frames in the packet, 3 bytes for the cumulative frame number, 4 bytes for the absolute offset byte number from the head of the stream to the pack header, and the relative offset byte number from the pack header to the packet header Two bytes are prepared. The number of allocated bytes may be changed according to the length of the MPEG system stream to be handled. The number of frames in a packet in Table 2 means the number of picture codes existing after the GOP code in a packet including a GOP in the case of video, and the number of AAUs in the packet in the case of audio.

[0046]

[Table 2]

In the MPEG system stream, a video stream and an audio stream are packetized, and the packets are multiplexed into one stream. Therefore, when performing cue playback from the middle,
The starting position of the system separation of video and audio must be determined. In the case of cueing reproduction on the basis of a video frame, the GOP code serves as a guide of the system separation start position. Therefore, the packet code position of the packet including the GOP is provided as a video cue reproduction record. Further, the MPEG system stream is
Several packets are grouped together, and a header is added thereto to form a pack. The main purpose of the pack is to enable synchronous decoding of video and audio from the middle of a stream. The SCR (System C
lock Reference, system time reference value). Therefore, in consideration of referring to the information in the pack header, the offset byte number is recorded separately for the absolute offset byte number from the stream head to the pack header and the relative offset byte number from the pack header to the packet header. It is. However, when audio playback is not performed simultaneously with video playback, or when strict video / audio synchronized playback is not required, information in the pack header may not be used. In this case, only the absolute offset byte number from the stream head to the packet including the GOP code is recorded.

If such a cue reproduction table file is created for the MPEG system stream, cue reproduction based on video frames and cue reproduction based on audio frames can be performed at high speed. Can be. Further, in Table 2, the number of frames in the packet for video heading reproduction is not necessarily required, and thus may not be recorded. However, it is preferable that the video and audio records have the same record length.

Table 3 shows an example of the header of the cue reproduction table file. AAU in table file header
Header, system header and sequence header are MPE
This is a copy from the G system stream. These are recorded in the table header because they include decoding parameters necessary for cueing reproduction in the middle of the stream. Also, by looking at this information, the features of the MPEG system stream can be confirmed.

[0050]

[Table 3]

A cue reproduction table must be prepared in advance for cue reproduction. However, it is not necessary to decode the entire stream, and the cue can be generated only by detecting the start code. It is possible to create a table in a very short time, and it becomes a file having an extremely small capacity as compared with a compressed moving image stream. Also, if it is incorporated into an encoder, it can be created at the time of encoding. Furthermore, a table file may be added as a user code to the beginning of the compressed moving image stream, and a decoder may be provided with a cue reproduction table reading function.

FIG. 3 shows an apparatus for reading decoded information of an MPEG system stream. This is the MPE of FIG.
This is for obtaining decoding information required for cueing using the cue reproduction table created by the G system stream cue reproduction table creation device. Be incorporated. Prior to describing the cue-reproducible compressed moving image decoding / display device, a decoding information reading device will be described. (Also, the decoding information reading device for the MPEG video stream has almost the same configuration as the decoding information reading device for the MPEG system stream, so the description is omitted.) The decoding information reading device of FIG. It comprises a reproduction start number storage means, a table reference means, and a record storage means. Each part in FIG. 3 will be described in order. Reference numeral 301 denotes a video reproduction start number storage unit for storing an externally input video reproduction start frame number, 302 denotes an audio reproduction start number storage unit for storing an externally input audio reproduction start frame number, and 303 denotes a head. The read-out / playback table file is read, and a stream seek for playing back from the frame of the playback start number stored in the video playback start number storage means 301 or the audio playback start number storage means 302 with reference to this. Table reference means 304 for determining a cue reproduction record in which the number of bytes and the decoding start frame number are recorded, 304
Is a record storage unit for storing the record determined by the table reference unit 303 and outputting the record. In addition,
The video reproduction start number storage means 301 and the audio reproduction start number storage means 302 are not used at the same time.

The operation of the decoding information reading apparatus having the above configuration will be described. First, the decoding information reading process based on a video frame will be described with reference to the flowchart of FIG. The outline of each step of this flowchart will be described. Step 101 (S10 in the drawing)
1 (hereinafter the same), the reproduction start frame number stored in the video reproduction start number storage means 301 is read. In step 102, a record storage buffer for storing the referenced record in the table file is initialized. Two record storage buffers are provided in the table reference means 303, and are used to store the currently referred record and the record referred to immediately before. Two buffers are the current record storage buffer,
Let's call it the previous record storage buffer. The records to be stored are the video / audio identifiers shown in Table 2, the number of frames in a packet, the number of accumulated frames, the absolute offset bytes from the stream head to the pack header, and the relative offset bytes from the pack header to the packet header. Consists of a number. In step 103, one record of the cueing reproduction table file that has already been read is read into the current record storage buffer. In step 104, the video / audio identifier in the current record storage buffer is checked to determine whether or not it is a video cue record. If it is not a video cue record, the process returns to step 103, and the next record in the table is read into the current record storage buffer. If it is determined in step 104 that there is a video cueing record, the process proceeds to step 105 where the reproduction start frame number is compared with a value obtained by adding 1 to the number of accumulated frames in the current record storage buffer. If this matches, it means that decoding and display should be performed in accordance with the record , so the process proceeds to step 108, where the record in the current record storage buffer is determined as a reference record,
In step 108, the record is output to the table storage means 304 in FIG. In step 105, if the value obtained by adding one to the cumulative number of frames in the current record storage buffer is smaller than the reproduction start frame number, it means that the record to be referred to has not yet been reached.
In step 109, the contents of the current record storage buffer are copied to the previous record storage buffer, and
Returning to 3, the next record in the table is read, and the processing is continued. In step 105, if the value obtained by adding 1 to the number of accumulated frames in the current record storage buffer is larger than the reproduction start frame number, it means that the record to be referred to has been passed.
Then, the previous record storage buffer is determined as a reference record, and the record is output to the table storage means 304 in FIG. In such a flow, the reference record in the table file is determined.

Using the example of the table file for cueing reproduction of the MPEG system stream shown in Table 2, the process of determining the reference record in the table file when cueing and reproducing the video from the 50th frame will be described. The value of the video / audio identifier in the table file of Table 2 is 0, that is, only the video cueing and playback record is focused on, and the audio cueing record is skipped. The video cueing record is created only for the packet including the GOP code. Looking at the table file in order from the top, search for a record in which the accumulated frame number becomes 50 or more.
In the case of Table 2, this is the fourth record from the bottom. The cumulative frame number of this record is 62, and the absolute offset byte number from the stream head to the pack header is 27703.
8 bytes, the relative offset byte number from the pack header to the packet header is 4108 bytes. this is,
When seeking 277038 bytes of the system stream, there is a pack header code, and from there a 410
Since the packet header of the packet including the GOP is located at the position where the 8-byte seek has been performed, if the system separation is started from that and the decoding is started from the GOP present in the packet, it means that the data is reproduced from the 63rd frame. That is, in order to perform reproduction from the 50th frame, decoding must be started from the immediately preceding GOP. Therefore, referring to the previous video cueing / playback record, the cumulative number of frames is 32, and the absolute offset byte number from the head of the stream to the pack header is 135.
450 bytes, and the relative offset byte number from the pack header to the packet header is 12 bytes. Since this record is a record necessary for cue reproduction from the 50th frame, the stream is referred to and cue reproduction is performed with reference to this record.

Next, the process of calculating the stream seek amount based on the audio frame will be described. The amount of stream seek is determined by the table reference means 302. The process of calculating the amount of stream seek based on an audio frame (a frame means AAU) is shown in the flowchart of FIG. The outline of each step is described according to this flow. In step 201, the reproduction start frame number stored in the audio reproduction start number storage means 302 is read. In step 202, a record storage buffer for storing the referenced record in the table file is initialized. This record storage buffer is prepared in the table reference means 303, and records to be stored include the video / audio identifier, the number of frames in a packet, the number of accumulated frames, and the number of frames from the head of the stream to the pack header shown in Table 2. It consists of the absolute offset byte number and the relative offset byte number from the pack header to the packet header. Among these, the cumulative frame number means the cumulative frame number (AAU number) from the stream head to the target packet. In step 203, one record of the cueing reproduction table file that has already been read is read into the record storage buffer. In step 204, the video / audio identifier in the current record storage buffer is checked, and it is determined whether or not the record is an audio cueing reproduction record. If the record is not for audio cueing playback,
Returning to step 203, the next record in the table is
Read into record storage buffer. If it is determined in step 204 that the record is an audio cue record, the process proceeds to step 205. In step 205,
A range of frames existing in the packet is obtained from the accumulated number of frames in the record storage buffer and the number of frames in the packet, and this range is defined as a frame range in the packet. In step 206, the reproduction start frame number is compared with the frame range in the packet, and it is determined whether or not the specified reproduction start frame is in the packet. If it is determined that the packet is not included in the packet, the process returns to step 203 to read the next record in the table file. If it is determined in step 206 that the designated reproduction start frame is within the packet, the record in the record storage buffer is set as a reference record in step 207, and this record is output in step 208. In such a flow, the reference record in the table file is determined.

Using the example of the table file for cueing playback of the MPEG system stream shown in Table 2, the process of determining a reference record in the table file when cueing and playing back audio from the 50th frame will be described. Note that only the value of the video / audio identifier in the table file in Table 2 is 1, that is, only the record for audio cue reproduction.
Skip the cue record for video. Looking at the table file in order from the top, search for a record in which the accumulated frame number becomes 50 or more. In the case of Table 2, this is the fifth record from the top. The number of frames in the packet of this record is 13, the cumulative number of frames is 53, and the absolute offset byte number from the stream head to the pack header is 123138.
The relative offset byte number from the byte and pack header to the packet header is 4108 bytes. This means that when the system stream seeks 123138 bytes,
There is a pack header code, and further, there is a packet header at a position where it is sought by 4108 bytes, which means that the 41st to 53rd frames exist in the packet. Therefore, 5 existing in the packet
Decoding and reproduction may be performed from the 0th frame.

Using the cueing reproduction table created by the cueing table creation device corresponding to the MPEG system stream shown in FIG. 2 and the decoding information reading device shown in FIG. 3, decoding is performed to reproduce a moving image from a designated frame. Can be determined at high speed. And
By incorporating the decoding information reading device of FIG. 3 in the compressed video decoding / display device, a compressed video decoding / display device capable of high-speed cue reproduction from an arbitrary frame can be realized.

FIG. 6 is a block diagram of a compressed moving picture decoding / display apparatus compatible with an MPEG video stream. It comprises a playback section storage unit, a decode information reading unit, a decode information storage unit, a stream transmission unit, and a video decoding unit. However, these units, an overall control unit that controls each unit, a unit that converts a decoded frame image into a video signal, and the like are omitted. (The same applies to the following figures.) Each part in FIG. 6 will be described in order. 401 is
This is a playback section storage unit that stores a playback start frame number and a playback end frame number input from the outside. Reference numeral 402 denotes a decode information reading unit that reads a cue reproduction table file created in advance and uses the table to output decode information necessary for reproducing a moving image from the frame of the reproduction start frame number. . The decoding information reading unit 402 has substantially the same configuration as the decoding information reading device of FIG.
It comprises a record storage means 405. 406
Is decoding information storage means for storing decoding information output from the decoding information reading unit and the reproduction start / end frame numbers stored in the reproduction section storage information as decoding information, and controlling decoding processing. 407 is based on the decode information in the decode information storage means 406
Stream sending means for sending the PEG video stream after seeking. 408 decodes the stream sent from the stream sending means 407 based on the decode information received from the decode information storage means, starts displaying when the playback start frame is reached, and decodes and decodes when the playback end frame is reached. This is video decoding means for ending display.

The operation of the compressed video decoding / display apparatus for the MPEG video stream having the above configuration will be described. First, the reproduction start frame number and the reproduction end frame number input from the outside are stored in the reproduction section storage means 40.
1 is stored. Then, the reproduction start frame number is output to the decode information reading unit 402, and the reproduction start / end frame numbers are output to the decode information storage means. Upon receiving the reproduction start frame number, the decode information reading unit 402 refers to the cueing reproduction table and refers to the cumulative number of frames from the designated frame to the GOP code required for cueing reproduction, the offset from the stream head to the GOP code. The two kinds of numerical values of the number of bytes are read and output as decode information. The operation of the decode information reading unit has been described in detail with reference to Table 1. The decode information storage unit 406 stores the playback start / end frame numbers sent from the playback section storage unit 401 and the G number sent from the decode information reading unit 402.
The decoding start frame number calculated from the cumulative number of frames up to the OP code and the number of offset bytes from the head of the stream sent from the decode information reading unit 402 to the GOP code are stored as decode information. The decoding information storage means 406 determines that the reproduction start / end frame number is the same value when the reproduction start / end frame number is the same value, and judges the reproduction start / end frame number is different when the reproduction start / end frame number is different, the section designation moving image reproduction instruction. When the start frame number is set and the reproduction end frame number is undefined, it is determined that the reproduction is a cue reproduction instruction, and then the stream transmission means 407 and the video decoding means 408 are controlled. The stream sending means 407
After seeking the MPEG video stream by the number of offset bytes up to the GOP code stored in 6, the MPEG video stream is sent to the video decoding means. Since this stream always starts from a GOP, the video decoding means 408
Starts decoding from the beginning of the received stream.
Then, counting is started from the decoding start frame number, and when the reproduction start frame number is reached, the display is started,
When the playback end frame number is reached or the end of the stream is reached, decoding / display is terminated. However, start playback
If the end frame numbers have the same value, frame still image data of that frame number is output. The video decoding means outputs to the decode information storage means the frame number of the frame being decoded, the frame number of the frame being displayed, and a signal indicating the decoding / display status (indicating the start / end of decoding). Further, the decoding information stored in the decoding information storage means 406 can be output to the outside as needed.

As described above, the MPEG having the configuration shown in FIG.
In a compressed moving picture decoding / display apparatus for video streams, high-speed cue reproduction from an arbitrary designated frame and high-speed decoding of a frame still image of an arbitrary designated frame are performed by using a cue reproduction table file created in advance. It is possible. Also, instead of separately preparing the cue reproduction table creating device of FIG. 1, a cue reproduction table creator having the same function as this device is incorporated in the compressed moving picture decoding / display device of FIG. But it is good.

(Embodiment 2) FIG. 7 is a configuration diagram of a compressed moving picture decoding / display apparatus compatible with an MPEG system stream. FIG. 7 is a block diagram of M of FIG. 6 described in (Embodiment 1).
The configuration is almost the same as that of the compressed video decoding / display device compatible with the PEG video stream, and only an audio decoding unit is added according to the difference between the video stream and the system stream. Therefore, only the points different from FIG. 6 in the configuration and operation will be described.

The compressed moving picture decoding / display apparatus corresponding to the MPEG system stream shown in FIG.
1. Decoding information reading section 502, decoding information storage means 506, stream transmission means 507, and video / audio synchronous reproduction section 508. Further, the video / audio synchronous playback unit 508 includes a synchronization unit 509,
It comprises an audio decoding means 510 and a video decoding means 511. The audio decoding means 510 reads the audio stream sent from the stream sending means 507, seeks to an appropriate position, and then decodes and reproduces the audio stream. The video decoding unit 511 reads the video stream sent from the stream sending unit 507, starts decoding after detecting the GOP code, and starts displaying when the playback start frame is reached. The synchronization unit 509 synchronizes the audio decoding unit 510 with the video decoding unit 511.

This compressed moving picture decoding / display apparatus uses the MPE
Since it is compatible with the G system stream, the cue reproduction table file created by the cue reproduction table creator of FIG. 3 is used. With this table,
Both cue playback based on video frames and cue playback based on audio frames are possible,
Here, cueing reproduction based on a video frame will be described. The information stored in the decode information storage means 506 includes the reproduction start information output from the reproduction section storage means 501.
The end frame number, the decoding start frame number calculated from the number of frames in the packet output from the decode information reading unit 502 and the cumulative number of frames, and the absolute offset byte number from the head of the stream output from the decode information reading unit 502 to the pack header , Relative offset bytes from the pack header to the packet header. The decoding information storage unit 506 determines that the reproduction start / end frame number is the same value when the reproduction start / end frame number is the same value, and determines that the reproduction start / end frame number is different when the reproduction start / end frame number is different. When the start frame number is set and the reproduction end frame number is undefined, it is determined to be a cue reproduction instruction, and the stream transmission means and the video decoding means are controlled. From the decode information storage means 506 to the stream transmission means 507, the absolute offset byte number from the stream head to the pack header,
Also, the relative offset byte number from the pack header to the packet header is sent. Stream sending means 50
7 seeks the stream up to the pack header, reads the information described in the pack header, outputs it to the synchronization means 509, and outputs the information to the video / audio system.
Start separation. Regarding the video packet, the transmission to the video decoding means 511 from the video packet (including the GOP code) which can be determined from the relative offset byte number from the pack header to the packet header is started. As for the audio packets, all audio packets detected after the start of system separation are output from the audio decoding means 5.
Send to 10. Since the transmitted video stream has only been subjected to system separation, the head of the stream is
It is not always a GOP code. Therefore, the video decoding unit 511 analyzes the transmitted video stream in byte units, detects a GOP code, and starts decoding processing from the detected GOP code. Since the frame number of the frame immediately after the GOP is known from the decode information storage means as the decoding start frame number, counting is started from this frame number, and when the playback start frame number is reached, display is started. When decoding is reached, the decoding / display is terminated. The decoding start frame number and the reproduction start / end frame numbers used at this time are obtained from the decoding information storage 506 via the synchronization means 507. During video decoding / display, the audio decoding means detects an audio frame (AAU) corresponding to the video display start frame number, starts decoding / playback from that frame, and synchronizes with the end of video playback. The audio decoding / playback ends. Synchronization means 507 controls the synchronous reproduction of video and audio. If the reproduction start / end frame numbers have the same value, the video decoding means outputs the frame still image data of that frame number. At this time, the stream transmitting means 407 does not transmit the audio packet, and the audio decoding means does not operate. Also, since the information of the pack header is not required, the absolute offset bytes from the head of the stream to the pack header and the relative offset bytes from the pack header to the packet header are added, and the stream is sought by that number of bytes. System separation may be initiated from the video packet containing the GOP code at the location. Such a stream seek method may be employed when playing video without audio playback or when strict synchronized playback of video and audio is not required. Further, it is also possible to output the decode information stored in the decode information storage means 506 to the outside as needed. Although not shown in the figure,
The audio synchronous playback unit outputs the frame number of the frame being decoded, the frame number of the frame being displayed, and a signal indicating the decoding / display status (indicating decoding start / end, etc.) to the decoding information 506, and stores the decoding information. Means 506 is
This information is also stored as decode information.

As described above, the MPEG having the configuration shown in FIG.
By using a cue reproduction table file created in advance, high-speed cue reproduction from an arbitrary designated frame and high-speed decoding of a frame still image of an arbitrary designated frame can be achieved by using a cue reproduction table file created in advance. It is possible. Also, instead of separately preparing the cue reproduction table creation device of FIG. 2, a cue reproduction table creation unit having the same function as this device is incorporated in the compressed moving picture decoding / display device of FIG. But it is good. In addition, when the cue reproduction is performed with reference to the audio frame, a decoding information reading unit having the same function as that of the decoding information reading apparatus of FIG. 3 is prepared, and the configuration is such that an audio reproduction section can be input.

(Embodiment 3) FIG. 8 is a block diagram of a compressed video decoding / display apparatus capable of creating an index image file. This device has a cue-reproducible compressed moving image decoding / display unit having the same function as that of the compressed moving image decoding / display device of FIG. 6 or 7 therein. This apparatus uses a scene change detection result file in which the result of detecting a break in a scene in a moving image stream is recorded by any method, and a cueing reproduction table created by the cueing reproduction table creation apparatus of FIG. Then, the head still image of each scene is decoded from the compressed moving image stream, and the still image group is output as one file. It is assumed that the still image at the head of the scene is used as an index image to be displayed in a list when searching and browsing the compressed moving image stream.

FIG. 9 shows an example of a scene change detection result file, which has a format in which scene information is enumerated with a set of a start frame number and an end frame number of each scene. However, the method of obtaining the scene change detection result does not matter. The format for recording the scene change detection result may be any format other than that shown in FIG. 9 as long as it is a predetermined format.

The index image file creating apparatus shown in FIG. 8 comprises a scene change information storing means, a cue / reproducible compressed moving image decoding / display unit, a frame still image storing means, a header creating means, and a file creating means. In FIG.
Each part will be described in order. Reference numeral 601 denotes a scene change detection result file in which a result of detecting a scene break in a moving image stream is recorded, and scene change information described in the file is stored. Start playing
This is a scene change information storage unit that outputs as an end frame number. Reference numeral 602 denotes a cue reproduction which decodes a stream in accordance with a frame reproduction section input from the scene change information storage means using an MPEG video stream or an MPEG system stream and a cue reproduction table file corresponding to the stream. A possible compressed video decoding / display unit. Reference numeral 603 denotes a frame still image storage unit which reduces or compresses a frame still image output from the cue reproduction / reproducible compressed moving image decoding / display unit 602, temporarily stores the reduced still image, and sequentially outputs it to the file creating unit. . Reference numeral 604 denotes decoding information output from the cue reproduction-capable compressed moving image decoding / display unit 602,
This is header information creating means for creating header information to be added to the index image file and storing the created header information. 6
Reference numeral 05 denotes a file creation unit that combines the header information recorded in the header creation unit 604 and the frame still image data recorded in the frame still image storage unit 603 to create an index image file.

The operation of the index image creating apparatus having the above configuration will be described. First, a scene change detection result file is read into the scene change information storage unit 601. However, only the frame number at the head of each scene recorded in the scene change detection result file is sufficient.
The frame number at the head of each scene is stored in the decoding / display unit 602 for decoding and displaying the compressed moving image that can be searched for.
03 is output. However, the playback start frame number and the playback end frame number input to the cue-playable compressed moving image decoding / display unit 602 are both set to the scene start frame number. Since the playback start / end frame numbers have the same value, the cueing / reproducible compressed moving image decoding / display unit 602 decodes and outputs the frame still image of the frame number, and at the same time, decodes the header information to the header creation unit 6.
04. This cue reproduction / removable compressed video decoding /
The display unit 602 decodes the compressed moving image shown in FIG.
Equivalent to a display device, depending on the type of target stream
Use them properly. The frame still image storage unit 603 performs reduction processing of the still image or compression processing within the frame, and temporarily stores the image. However, the decoded image may be stored as it is. The header creating unit 604 creates a header of the index image file by using the decoding information output from the cue / reproducible compressed moving image decoding / display unit 602. File creation means 605
Then, the frame still image data stored in the frame still image storage unit 603 is sequentially written into the same file. The above processing is performed on the frame numbers at the beginning of all scenes stored in the scene change information storage means.
5 adds the header information created by the header creating unit 410 to the index image file, and ends the processing.

FIG. 10 is an example of an index image file. In this example, decoding from a compressed video stream
Each index image file has been reduced to a suitable size,
This is put together in one file. Each index image has the same size, its size can be calculated from the width / height and format of the image, and random access is possible. In the index image format description field,
A code for determining a predetermined format type is described. Further, an information description column such as the number of frames in the compressed moving image stream is provided. Further, if necessary, an area for writing the beginning and end frame numbers of the scene and a search keyword is provided in correspondence with each index image.

The apparatus shown in FIG . 8 can be applied to any of the MPEG video stream and the MPEG system stream by changing the cue / reproducible compressed moving image decoding / display unit 602. This apparatus is an example to which the compressed moving picture decoding / display apparatus shown in FIG. 6 or 7 is applied, and is characterized in that it can decode any frame still image included in a stream at high speed. The reason for creating the index images in advance is to reduce the time required for displaying the index image list at the time of browsing / searching, and
This is to add a search keyword to each scene in the index image file and provide a visual search means.If there is a high-speed compressed video decoding / display device, frames can be extracted from the compressed video stream as necessary. A method of decoding a still image may be adopted.

(Embodiment 4) FIG. 11 is a block diagram of a compressed moving picture decoding / displaying apparatus having an index picture list display function. It consists of a decoding / display unit.

Each part in FIG. 11 will be described in order. 7
An index image list display unit 01 receives an instruction from a user, reads an index image in an index image file according to an index image display instruction, and displays a list of the index images. Reference numeral 702 denotes a playback section determination unit that determines a playback section according to the selected index image. Reference numeral 703 denotes a cue-playable compressed moving image decoding / display unit.

The operation of the compressed moving picture decoding / display apparatus having the above-structured index picture list display function will be described. First, the index image list display unit 701
In accordance with a command from the user, an index image file is read, and a list of index images is displayed. The user selects one index image or a plurality of continuous index images from the list of index images. Here, there are two types of moving image reproduction methods that can be specified by the user: cueing reproduction and section reproduction. Cueing playback is a playback method in which playback from a specified index is performed and playback is continued until a playback stop command is issued or the end of the stream is reached. This is a playback method for performing playback from the start frame number to the end frame number of the scene. Therefore, when one index is selected, cueing reproduction or section reproduction is possible, and when a plurality of consecutive indexes are selected, section reproduction is automatically performed. When the user selects an index image and designates a reproduction method, the index image list display means 701 determines a reproduction start / end frame according to the specification and outputs it to the reproduction section storage means. However, it is assumed that the index image file records the start / end frame number information of the scene corresponding to each index. The playback section storage unit 702 finds the stored playback start / end frame numbers and searches for a reproducible compressed moving image decoding / display unit 703.
, And the cue / reproducible compressed moving image decoding / display unit 70
No. 3 reproduces the compressed moving image stream using the cue reproduction table file.

This device is an index image creation device shown in FIG.
A still image group included in the stream is displayed as a list of index images by using an index image file created by the apparatus , and a compressed moving image can be searched for from the index image selected by the user and reproduced. The compressed video stream is an MPEG video stream, MPEG
It can be any of the system streams.

(Embodiment 5) FIG. 12 is a block diagram of a compressed moving picture decoding / display apparatus having an index picture decoding / list display function. It comprises an image decoding / display unit, display content control means, and index list display means.
Since the configuration is the same as that of the compressed video decoding / display device with the index image list display function of (Embodiment 4) of FIG.

Each part in FIG. 12 will be described in order. 8
01 reads the scene change detection result file,
This is a scene change information storage unit that stores this and generates a frame still image decoding instruction or a moving image reproduction instruction in accordance with an instruction from the user . A decoding control unit 802 discriminates between a frame still image decoding instruction and a moving image reproduction instruction and controls decoding processing. Reference numeral 803 denotes a cue-playable compressed moving image decoding / display unit. Reference numeral 804 denotes display content control means for distinguishing between a frame still image decoding instruction and a moving image reproduction instruction and displaying a frame still image, an index image list, and a moving image. Reference numeral 805 denotes an index image display unit that arranges and displays the decoded frame still images.

The operation of the compressed moving picture decoding / display apparatus having the above-structured index picture list display function will be described. First, the scene change information storage means 801
Read the scene change detection result file. This file is obtained by dividing the compressed moving image stream into a plurality of scenes and recording the first frame number and the last frame number of each scene. When the user issues an index image display command, the scene change information storage unit 801 outputs the first frame number of the scene to the decoding control unit 802 together with the frame still image decoding command. The index images to be displayed in a list may be the first frame of all scenes or the first frame of a plurality of continuous scenes. The completion of decoding of the still image of the first frame of one scene can be confirmed by the decode information output from the cue / reproducible compressed moving image decoding / display unit 803. Therefore, the scene change information storage unit 801 decodes the one frame. Each time is finished, the head frame number of the next scene is output. When the frame number is output from the scene change information storage means, the decoding control means 802 decodes the compressed moving image that can be searched for
The playback start / end frame numbers are output to the display unit 803,
It controls the cue / reproducible compressed moving image decoding / display unit 803 and the display content control unit 804. The playback start / end frame numbers at this time are the frame numbers at the beginning of the scene. Search / Reproduce Compressed Moving Picture Decoding / Display Unit 8
03 decodes the frame still image in accordance with the output from the decoding control means 802 and outputs it to the display content control means 804. The display content control unit 804 outputs the frame still image data to the index image list display unit 805. The index image list display means reduces the sequentially transmitted frame still images and arranges and displays them.
When the still image at the head of the scene is decoded according to the scene change information and is displayed as a list, the index list display unit 805 enters a state in which a moving image reproduction command can be accepted. When the user selects an index image, the index image list display unit 805 sends the scene change information storage unit 80
Information about the selected index image is output. This information is for linking the selected index image with the scene in the scene change information. The scene change information storage unit 801 determines the playback start / end frame numbers with reference to the scene change information and the selected playback method (start playback or section playback), and outputs a moving image playback command to the decoding control unit 802. I do.
The decoding control means 802 outputs the playback start / end frame numbers to the cue-reproducible compressed moving image decoding / display unit 803, and outputs the cue-reproducible compressed moving image decoding / display unit 803 and the display content control means 804. Perform control. The cueing-reproducible compressed moving image decoding / display unit 803 decodes the stream in accordance with the output from the decoding control unit 802, and outputs data from the reproduction start frame to the reproduction end frame to the display content control unit 804. The display content control unit 804 displays the received decrypted data as a moving image.

This apparatus does not need to create an index image file in advance, and decodes a scene leading image in a stream using a scene change detection result file in which scene information in the compressed moving image stream is recorded. Has a function of displaying this as a list of index images, and is further characterized in that cue reproduction can be performed from any index image. By changing the cue reproduction capable compressed moving image decoding / display unit 803, Both the MPEG video stream and the MPEG system stream can be supported. Although omitted in the description, when an MPEG system stream is targeted, audio playback can be performed simultaneously in synchronization with video playback. Further, a function of decoding / reproducing only audio may be added corresponding to the index image. Further, the cue reproduction-capable compressed moving image decoding / display unit 803 may be replaced with a unit capable of supporting both the MPEG video stream and the MPEG system stream. In such a case, the cue reproduction-capable compressed moving image decoding / combining the functions of the compressed moving image decoding / display device of FIGS. 6 and 7 is combined.
It becomes a display unit.

(Embodiment 6) FIG. 13 shows a simplified compressed moving image editing apparatus for editing an index image.
(Editing an index image is also defined as simple editing.) The compressed moving image decoding / display apparatus described in (Embodiment 5) uses information recorded in a scene change detection result file. Thus, it is possible to decode the still image of the first frame of each scene from the compressed moving image stream and display it as a list. The scene change detection result file used at that time is a file in which a portion where the content of the video changes by some method is detected, and the result of division into a plurality of scenes is recorded. Since performing this operation manually requires a great deal of labor, various automatic scene change detection methods have been developed. However, since these are not scene divisions after understanding the contents of the video, there are aspects in which accuracy is insufficient. FIG.
The simplified compressed moving image editing device 3 is for compensating for the deficiency of automatic scene change detection, editing the scene change result file, and providing the user with more appropriate index information.

The simplified compressed moving picture editing apparatus shown in FIG. 13 has many parts which overlap with the structure of the compressed moving picture decoding / display apparatus described in (Embodiment 5). The simplified compressed moving image editing apparatus of FIG. 13 includes a frame number calculating unit 901, a scene change information storing unit 9
02, scene change information editing means 903, decoding control means 904, cueing / reproducible compressed moving picture decoding / displaying section 90
5, display content control means 906, playback frame information storage means 907, index image list display means 908, and playback section determination means 909. Also, from the figure,
The moving image reproduction control means for controlling the moving image reproduction is omitted. This moving image reproduction control means is for receiving commands from the user, such as reproduction / stop / pause of moving images, and for controlling decoding / reproduction of compressed moving images.

The frame number calculating means 901 receives a detailed frame image display command from the user, calculates a frame number corresponding to the command, and generates a frame still image decoding command. Here, the detailed frame image is a list of continuous frame images starting from the specified frame number or frame images starting from the specified frame number and thinned out at equal intervals. This is a function for making it easier to select a new index image when changing or adding an index image. For the same reason, a forward / backward frame forward playback function or the like may be added to the cue / reproducible compressed moving image decoding / display unit 905. The decoding control means 904 determines the means for inputting the reproduction start / end frame numbers, and generates a frame still image decoding instruction or a moving image reproduction instruction accordingly. Reproduction frame information storage means 9
Reference numeral 07 denotes a cue-reproducible compressed moving image decoding / display unit 905.
The number of the frame being reproduced, which is included in the decoding information output from the storage device, is stored. The index image list display means 908 performs not only a list display of index images but also a list display of detailed frame images. The scene change information editing means 903 edits the scene change information in accordance with the index image editing command from the user, and reflects the result in the scene change information storage means 902 and the index images displayed in the list. At this time, the frame numbers of the new index image and the deleted index image are obtained from the reproduction frame information storage unit 907 or the reproduction section determination unit 909. The scene change information editing means also has a function of registering a search keyword for each scene.

There are three types of editing commands from the user: index image deletion, change, and addition. If the user wants to delete the index image, after selecting one index image from the index images displayed in the list,
Send an index delete command. At this time, the scene change information editing unit 903 integrates the scene information corresponding to the deleted index image with the previous scene information. If you want to add an index image, select one frame image from the list of detailed frame images, or pause the reproduced moving image at an appropriate After the decision is made, an index addition instruction is sent. At this time, the scene change information editing unit 903 divides the scene information including the added frame image into two with the added frame as a boundary. An index image change means a change in a scene boundary , and is a combination of deletion and addition. However, it is assumed that an appropriate GUI for performing these operations is provided to the user.

FIG. 14 shows that the index image is edited.
FIG. 9 shows a situation where a scene change detection result file is modified. In the original scene change detection result file, there are ten scenes from scene A to scene J.
This is edited, scenes B and E are deleted, the boundaries between scenes D and F are changed, scene H is divided, and scenes x and y are added. FIG. 15 is an example of an edited scene change detection result file, in which a registered search keyword is recorded following a start frame and an end frame of each scene. This keyword is used for a scene search.

(Embodiment 7) FIG. 16 shows a simplified compressed video editing apparatus. Simple editing refers to an editing method of creating a file in which only editing information is recorded, instead of saving the editing result as a new compressed moving image stream. Although it is not suitable for fine-grained editing, it is considered to be effective when the order of relatively long scenes scattered in a plurality of compressed moving image streams is changed and played back.
At the time of playback, the simplified editing information file in which the result of the simplified editing is recorded is read, and the stream file name and the scene start / end frame numbers (that is, the playback section) recorded in the file are sequentially read to read the compressed moving image stream. The section reproduction is repeated.

The simplified compressed moving picture editing apparatus shown in FIG.
Since the configuration is almost the same as that of the compressed moving image simple editing apparatus for editing the index image of No. 3, the different points will be mainly described. The apparatus shown in FIG.
001, scene change information storage means 1002, scene change information editing means 1003, decoding control means 100
4. Cueing-reproducible compressed moving image decoding / display unit 1005,
It comprises a display content control unit 1006, a reproduction frame information storage unit 1007, an index list display unit 1008, a reproduction section determination unit 1009, and a simple editing unit 1010. Among them, the simple editing unit includes an editing result display unit 101
1. It comprises simple editing information file creating means 1012. 13 in that a simple editing unit 1010 is added.

At the time of the simple editing work, mainly, the simple editing unit 1010 and the index image list display means 100 in FIG.
8. The playback section determination means 1009 operates. The editing command input from the outside is transmitted to the index image list display unit 10.
08, transmitted to the simple editing information file creating means 1012. The editing commands include a playback section index image selection command and a playback section index image alignment command. The playback section index image selection command is a command for selecting an index image corresponding to a scene to be played. Here, the selected index image is referred to as a reproduction section index image. The playback section index image alignment command is a command for determining the playback order of a plurality of playback section index images.

Next, the operation of the apparatus shown in FIG. 16 at the time of simple editing will be described. The editing result display means 1011 duplicates the index image selected from the index image group displayed in the list by the index image list display means 1008, and displays this in a new display area (window) as a reproduction section index image. When the playback section index image is selected, the playback section determination unit 1009 obtains the scene playback start / end frame numbers corresponding to the index image, and uses this to obtain the simple edit information file creation unit 1.
012. A plurality of playback section index images can be selected, and the selected plurality of playback section index images are arranged and displayed in a list according to the playback order.
In addition, the arrangement order of the playback section index images can be changed, and information on the arrangement order of the playback section index images (that is, the playback order) is output to the simple editing information file creating unit 1012. An appropriate GUI is provided for changing the arrangement order and adding / deleting a playback section index image. The simple edit information creating unit 1012 creates a simple edit information file based on the playback start / end frame numbers output from the playback section determination unit 1009 and the information on the playback order output from the edit result display unit 1011. I do. At this time, if a search keyword or a scene title is registered, these are added to the simple edit information file. In this way, the simplified compressed moving image editing apparatus of FIG. 16 can create a simplified editing information file for reproducing a plurality of scenes to be reproduced in a desired order in addition to correcting the index image itself.

FIG. 17 is a conceptual diagram of the simple editing. The user (editor) is presented with an index image list display window and a simple edit window. The user
When the index image corresponding to the scene to be reproduced is clicked using a mouse or the like, the index image is duplicated, and the duplicated index image is dropped on the simple edit window. This operation is repeated, and the reproduction section index images in the simple edit window are rearranged in a desired order. The sorting operation can be performed with a mouse.
The simplified compressed moving image editing apparatus of FIG. 16 also has a function of modifying the index image itself, and utilizes this function as needed. In addition, if a moving image playback window is provided, the user can work while checking the result of the simple editing.

FIG. 18 shows an example of the simplified edit result file, which corresponds to FIG. The left side of FIG. 18 is a scene change detection result file. According to this file,
When the user selects the index images of scenes A, M, J, K, and D from the index images from scene A to scene O displayed in the list and desires to reproduce the images in this order, FIG. A simple edit information file as shown on the right side of 18 is created. In this example, a search keyword and a scene title are added to each scene.

By expanding the simplified compressed moving picture editing apparatus shown in FIG. 16, it is also possible to create a simplified editing information file including scenes of a plurality of compressed moving picture streams. FIG. 19 is a conceptual diagram of the simple editing when a plurality of files are edited at one time, and FIG. 20 is an example of a simple editing result file corresponding thereto. In this case, the user
An index image list display window for the number of compressed moving image streams to be edited and one simple editing window are presented. The user freely selects a plurality of index images in the plurality of compressed moving image streams and determines a reproduction order. In the created simple edit information file,
A compressed moving image stream file name is recorded together with the start / end frame numbers of each scene and a search keyword.

Further, an index image file with simple editing information may be created by reducing (or compressing) the index image of the reproduction section into one, and adding simple editing information as a header thereto. If this index image file with simplified editing information is used, the head image of each scene can be decoded and displayed at high speed, and the response time to the user can be reduced.

(Eighth Embodiment) FIG. 21 shows a compressed moving picture decoding / display apparatus based on a simple editing information file. The configuration of this device is almost the same as that of the compressed video decoding / display device with index image decoding / list display function shown in FIG. The apparatus shown in FIG. 21 includes a simplified editing information storage unit 1101, a file management unit 1102, a decoding control unit 1103, and a cue-playable compressed moving image decoding / display unit 1.
104, a display content control unit, 1105, and an index image list display unit 1106. The difference from the apparatus of FIG. 12 is that the file to be read is not a scene change detection result file but a simple edit information file. Since the simplified editing information file includes scenes of a plurality of compressed moving image streams, a file management unit for managing the compressed moving image stream and its attached files is added.

First, the simple edit information storage means 1101
The simple editing information file is read, and the moving image file name corresponding to the scene is output to the file management information. In response to this, the file management means becomes accessible to the requested compressed moving image stream and the cue reproduction table file. However, the cue reproduction table file name corresponding to the compressed moving image stream file name can be specified. Also,
The simple editing information storage unit 1101 outputs a frame still image decoding instruction and a reproduction start / end frame number of each scene to the decoding control unit 1103. Completion of decoding of one frame means that the cue reproduction-capable compressed moving image decoding / display unit 1
This can be confirmed by the decode information output by the reference numeral 104. If this process is sequentially repeated for the number of scenes in the simple edit information,
Still images of the first frame of each scene are displayed as a list as index images. Then, when one index image is selected from the index images displayed in the list, information specifying the selected index image is output to the simplified editing information storage means. Then, the simple editing information storage unit outputs to the decoding control unit 1103 the moving image reproduction instruction and the reproduction start / end frame numbers of the scene corresponding to the selected index image. However, the moving image playback command includes a cue playback command and a section playback command.
Selected by the user. Thereafter, the moving image is reproduced from the index image through the same process as that of the apparatus of FIG.

As described above, the compressed video decoding / display apparatus of FIG. 21 continuously decodes and decodes a plurality of scenes scattered in a plurality of compressed video streams based on the simplified editing information file.
Can be displayed, and can be decoded and displayed from an index image that lists the first frame of each scene.
It is.

(Embodiment 9) FIG. 22 shows a compressed moving picture decoding / display apparatus with a keyword search function. The configuration of this device is almost the same as the compressed video decoding / display device based on the simplified editing information file in FIG.
Since the configuration is the same , the following description will focus on the differences. The apparatus shown in FIG. 22 includes a keyword search unit 1201, a file management unit 1202, a scene change information storage unit 1203, a decoding control unit 1204, a cue / reproducible compressed moving image decoding / display unit 1205, and a display content control unit 120.
6. Index image list display means 1207. The file management unit 1202 manages a compressed moving image stream and its attached files. The keyword search unit 1201 performs a keyword search based on stream search keywords and scene search keywords input from the outside, and displays search results. This search result,
In addition to text-based information such as a stream name and the number of scene frames, a still image of a scene top frame and the like are also included. This keyword search may be a method of directly searching a file having a keyword registration area, or a method of searching a keyword management file which collectively manages all keywords of all files. The search result is presented to the user and stored in the scene change information storage means. Then, based on the scene change information, the scene head frame in the stream is displayed as a list of index images. If the user selects an index image desired to be reproduced from the index images displayed in the list, the search result can be immediately confirmed as a moving image. The apparatus shown in FIG. 22 can perform a search not only with a keyword in a unit of a compressed moving image stream but also with a keyword in a scene unit.
It is characterized in that search results can be presented visually.
Although a simple editing information file is not shown in the figure, keyword search, index image list display, and moving image reproduction including this file may be performed.

(Embodiment 10) FIGS. 23 and 24
Is a client-server type compressed video decoding / display device. FIG. 23 shows a server device, and FIG. 24 shows a client device. This device has basically the same configuration as the compressed video decoding / display device with a keyword search function shown in FIG. The major difference is that, because of the client-server configuration, the cue-playable compressed moving image decoding / display unit is divided into two. The server apparatus seeks the stream in response to the cue reproduction request, and then sends the stream to the client together with the decode information. The client device decodes and displays the received stream based on the decode information.

The server shown in FIG. 23 includes a keyword search unit 1301, a file management unit 1302, a stream transmission unit 1303, a scene change information storage unit 1308,
It comprises a command discriminating means 1309. Among them, the stream transmission unit 1303 includes a playback section storage unit 1304,
It comprises a decode information reading unit 1305, a decode information storage unit 1306, and a stream sending unit 1307, and is the same as the compressed video decoding / display device of FIG. 7 except for the video / audio synchronous playback unit. Also, 13
09, a command from the client is determined, and a keyword search unit 1301 and a scene change information storage unit 130
8 and responds to commands from the client. The server-side communication control unit 1310 controls communication with the client, receives a command from the client, and transmits a compressed moving image stream or the like in response to the command.

On the other hand, the client device shown in FIG. 24 includes a command transmission control means 1401, a decoding control means 1402, a video / audio synchronous reproduction section 1403, and a display content control means 1.
404, an index image list display unit 1405, and a client side communication control unit 1406. Among them, the video / audio synchronous playback unit 1403 is the same as the video / audio synchronous playback unit of the compressed moving picture decoding / display device of FIG. Also, the command transmission control means 140
Reference numeral 1 is for determining a search command, an index image display command, a moving image reproduction command, and the like from a user, and transmitting the command to the server in response to the command. The client-side communication control unit 1406 controls communication with the server, transmits instructions to the server, and receives a compressed moving image stream and the like.

By combining the server device shown in FIG. 23 and the client device shown in FIG. 24, it is possible to achieve the same function as the compressed video decoding / display device with a keyword search function shown in FIG. Feature of the client-server type compressed video decode and display device, when a frame is designated cue playback from the middle stream and, during decoding of a frame still image included in the compressed video stream, it should start decoding strike
It is to transmit a stream after detecting a stream position . Although the above description is directed to the MPEG system, it is possible to support an MPEG video stream by only partially changing the device configuration. Further, the device configuration may be changed so as to support both the MPEG video stream and the MPEG system stream.

[0100]

As described above, the compressed moving picture decoding and decoding of the present invention
Since the display device uses a cue reproduction table file created in advance, it is difficult to perform cue reproduction from a designated frame at high speed, such as MPEG, which is an international standard for moving picture coding. This has the effect that high-speed cue reproduction of the image stream is possible. Further, the compressed moving image decoding / display device and the simplified compressed moving image editing device of the present invention provide a function of displaying a list of scene leading images in a stream as index images, so that only the scene desired by the user is immediately displayed. The effect of being able to play back, and creating a file that records only the edited result without creating a new compressed moving image stream as the edited result,
It has an effect that a moving image can be reproduced based on this file.

[Brief description of the drawings]

FIG. 1 is a diagram showing an MPEG video stream cue reproduction table file creation device according to a first embodiment of the present invention;

FIG. 2 is a diagram showing an MPEG system stream cue reproduction table file creating apparatus according to the first embodiment;

FIG. 3 is a diagram showing a decoded information reading apparatus according to the first embodiment;

FIG. 4 is a flowchart of a process of determining a record for playing back a video of an MPEG system stream according to the first embodiment;

FIG. 5 is a flowchart of a process of determining a record for audio cueing reproduction of an MPEG system stream according to the first embodiment;

FIG. 6 is a diagram showing a compressed moving image decoding / display apparatus capable of creating an index image file according to the first embodiment;

FIG. 7 is a view showing an example of a scene change detection result file according to the second embodiment;

FIG. 8 is a view showing an example of an index image file format according to the third embodiment;

FIG. 9 is a diagram showing a compressed video decoding / display device (MPEG video stream) according to the third embodiment;

FIG. 10 is a diagram showing a compressed video decoding / display device (MPEG system stream) according to the third embodiment;

FIG. 11 is a diagram showing a compressed moving image decoding / display device with an index image list display function according to the fourth embodiment;

FIG. 12 shows an index image decoding and decoding method according to the fifth embodiment.
Diagram showing compressed video decoding / display device with list display function

FIG. 13 is a diagram showing a simplified compressed moving image editing apparatus capable of editing an index image according to the sixth embodiment;

FIG. 14 is a conceptual diagram of index image editing according to the sixth embodiment.

FIG. 15 is a diagram showing a scene change information file with a search keyword according to the sixth embodiment;

FIG. 16 is a diagram showing a simplified compressed video editing apparatus according to the seventh embodiment;

FIG. 17 is a view showing a conceptual diagram (single file) of simple editing according to the seventh embodiment;

FIG. 18 is a diagram showing a simplified edit result file (single file) according to the seventh embodiment;

FIG. 19 is a conceptual diagram of simple editing according to the seventh embodiment (multiple files).

FIG. 20 is a diagram showing a simplified editing result file (a plurality of files) according to the seventh embodiment;

FIG. 21 is a diagram showing a compressed moving picture decoding / display apparatus based on a simplified edit file according to the eighth embodiment.

FIG. 22 is a diagram showing a compressed video decoding / display device with a keyword search function according to the ninth embodiment.

FIG. 23 is a diagram showing a server unit of the client-server type compressed moving image decoding / display apparatus according to the tenth embodiment.

FIG. 24 is a diagram showing a client unit of a client / server type compressed moving picture decoding / display apparatus according to the tenth embodiment.

[Explanation of symbols]

 401 playback section storage means 402 decode information reading unit 403 video heading playback start number storage means 404 table reference means 405 record storage means 406 decode information storage means 407 stream transmission means 408 video decoding means

──────────────────────────────────────────────────続 き Continuation of the front page (72) Katsuhiro Kanamori 3-10-1, Higashi-Mita, Tama-ku, Kawasaki City, Kanagawa Prefecture Matsushita Giken Co., Ltd. (56) References JP-A-6-54292 (JP, A) JP-A-6-162116 (JP, A) JP-A-6-326998 (JP, A) JP-A-6-105280 (JP, A) JP-A-6-236439 (JP, A) JP-A-5-236449 ( JP, A) (58) Fields surveyed (Int. Cl. ⁷ , DB name) H04N 5/91-5/956 H04N 5/85 H04N ^7/ 24-7/68 G11B 20/10-20/12

Claims (27)

(57) [Claims] 1. A reproduction section storing means for storing and outputting reproduction start and end frame numbers of a designated compressed moving image stream, and a cueing reproduction table which has been created in advance. Read the information required for playback,
Decoding information reading means for outputting the same, decoding information storage means for storing the output of the reproduction section storage means and the output of the decoding information reading means as decoding information, and seeking the compressed moving image stream based on the decoding information. Stream transmitting means for starting to transmit the stream from the position, decoding the compressed moving image stream transmitted from the stream transmitting means in accordance with the decoding information, and starting a video display when a reproduction start frame is reached; A compressed moving picture decoding / display apparatus including a video decoding means having a function of terminating decoding / video display when a playback end frame is reached and a function of outputting a frame still image when the playback start and end frames have the same value. . 2. A cue reproduction table, for all random access reference codes included in a compressed video stream, the number of offset bytes from the stream head to the code and the cumulative number of frames from immediately before the code The compressed moving picture decoding / display apparatus according to claim 1, wherein the compressed moving picture decoding / display apparatus is recorded. 3. The decoding information storage means stores the decoding start stream position and the decoding start frame number calculated from the information received from the decoding information reading means and the reproduction start and end frame numbers received from the reproduction section storage means as decoding information. 2. The compressed video decoding / display apparatus according to claim 1, wherein: 4. An inter-frame predictive coded compressed moving image stream containing intra-frame predictive coded data in which a code for random access is inserted in the stream is analyzed from the head of the stream, and is used as a reference for random access. Code detection means for detecting a code and a frame included in a stream, frame number calculation means for calculating the total number of frames from the head of the stream to a reference code for random access, the code detection means and the frame number calculation means And a header information creating unit for creating a header describing the stream information by using the output of the code detecting unit and the output of the frame number calculating unit. The number of offset bytes up to and the cumulative 2. The compressed moving picture decoding / display apparatus according to claim 1, further comprising a table creating means for creating a table summarizing the number of frames, adding a header created by said header information creating means, and outputting the file as a file. 5. An MP which is an international standard for a moving picture coding system.
A code detecting means for analyzing a code of the MPEG video stream compressed by the EG from the head of the stream, detecting a desired start code, and storing a position of the code in the stream;
Frame number calculating means for calculating the number of frames up to immediately before the code; and when the code detecting section detects a GOP code serving as a reference for random access, the offset byte number from the head of the stream to the GOP code and GO
The summary of the number of accumulated frames to P as records, upon reaching the end of the stream, to create a table that summarizes the records created for each GOP, and a table creation means for outputting it as a file
The compressed video decoding / display device according to claim 1 . 6. A video playback section storage means for storing playback start and end video frame numbers, an audio playback section storage means for storing playback start and end audio frame numbers, and a cue corresponding to an MPEG system stream created in advance. Decoding information reading means for reading a reproduction table, reading information necessary for cue reproduction in response to a cue reproduction command based on a video frame or an audio frame, and outputting the information; A decoding information storage unit for storing the output of the information reading unit and storing the same; seeking a compressed moving image stream based on the decoding information stored in the decoding information storage unit; After output to the synchronization means, Stream transmitting means for sequentially transmitting the video / audio packets, and decoding the video stream received from the stream transmitting means, and when the reproduction start frame number is reached, video display is started.
A video decoding unit having a function of terminating decoding / video display when a reproduction end frame is reached, and a function of outputting a frame still image when reproduction start / end is the same value, and an audio stream received from the stream transmission unit. Analyzing, when the decoding start frame is reached, audio playback is started, and further, when the playback ending frame is reached, decoding is ended. Based on the information stored in the decoding information storage means, a video is reproduced. A compressed moving picture decoding / display apparatus provided with a synchronous reproduction means for synchronizing audio reproduction. 7. A cueing / playback table includes, for a video packet including a GOP code in an MPEG system stream, a cumulative number of frames from the head of the stream to the GOP code, and a number of frames from the head of the stream to a pack header code including the packet. 7. The compressed moving image decoding apparatus according to claim 6, wherein a record is recorded in which three types of data, that is, an absolute offset byte number and a relative offset byte number from the pack header code to the packet header code, are recorded. Display device. 8. The cue reproduction table according to claim 1, wherein
For a video packet containing a GOP code in the system stream, the packet type identifier, the number of frames in the packet, the cumulative number of frames from the head of the stream to the GOP code, the absolute offset byte number from the head of the stream to the pack header code containing the packet, and A record that summarizes five types of data of the relative offset bytes from the pack header code to the packet header code, and the packet type identifier, the number of frames in the packet, and the cumulative number of frames from the beginning of the stream to the end of the packet for all audio packets 5 types of data, the absolute offset byte count from the head of the stream to the pack header code including the packet and the relative offset byte count from the pack header code to the packet header code Summary record is one that was recorded claim 6 compressed video decode and display device as claimed. 9. The decoding information storage means includes: a system separation start stream position, a video decoding start position, and an audio decoding start position calculated from information received from the decoding information reading means; 9. The decoding / display of a compressed moving picture according to claim 6, wherein the end frame number and information recorded in a pack header read from the stream by the stream sending means are stored as decoding information. apparatus. 10. M which is an international standard for video coding
Code analysis is performed on the MPEG system stream compressed by PEG from the beginning of the stream, the pack start code and the packet start code are detected, the positions of the pack start code and the packet start code in the stream are stored, and the system separated video・
A system code detecting means for transmitting an audio packet, and when a video packet is detected by the system code detecting means, stream analysis is performed by regarding video data divided into a plurality of packets as one video stream, and performing picture analysis. A video packet analyzing means for detecting a code including a code and a packet including a GOP code; a video frame number calculating means for counting the number of picture codes detected by the video packet analyzing means; and a video packet analyzing means including a GOP code. If a packet is detected, the number of frames contained in the packet, the absolute offset bytes from the beginning of the stream to the pack header code containing the packet, the relative offset bytes from the pack header code to the packet header code, Video cue reproducing record storage means for storing in one record a summary of the number of accumulated frames up to the frame number and the GOP coding DOO,
7. The compressed moving image decoding / display according to claim 6, further comprising a table creating means for creating a table in which the cue reproduction records sequentially output from the video cue reproduction record storage means are collected and outputting the table as a file. apparatus. 11. An international standard M for video coding
The PEG-compressed MPEG system stream is subjected to code analysis from the beginning of the stream, a pack start code and a packet start code are detected, the positions of the pack start code and the packet start code in the stream are stored, and the system separated video is output. A system code detecting means for transmitting an audio packet, and when a video packet is detected by the system code detecting means, stream analysis is performed by regarding video data divided into a plurality of packets as one video stream, A video packet analyzing means for detecting a picture code and a packet including a GOP code; a video frame number calculating means for counting the number of picture codes detected by the preceding video packet analyzing means; If the packet including the GOP code is detected by the packet analysis means, the absolute offset bytes from the head of the stream to the pack header code including the packet, the relative offset bytes from the pack header code to the packet header code, the number of frames in the packet A video cueing / playback record storing means for recording and storing the total number of frames up to the GOP code as one record; and, when an audio packet is detected by the system code detecting means, is divided into a plurality of packets. The audio data is analyzed as a single audio stream and stream analysis is performed.
Audio packet analyzing means for detecting a header code, and AA detected by the audio packet analyzing means.
Means for calculating the number of audio frames for counting U, each time a packet header is detected, the absolute offset byte number from the head of the stream to the pack header code of the pack including the audio packet, Offset byte number from to packet header code, AA in packet
An audio cueing / playback record storage means for storing the number of U's and the cumulative number of AAUs from the head of the stream as a single record; Means for adding a packet identifier for distinguishing both to the cue reproduction record sequentially output from the means.
A contract having a table creation means for creating a cue reproduction table for audio use and outputting this as a file
The compressed video decoding / display device according to claim 6. 12. A cue reproduction table creation unit stores information on a title of a compressed moving image stream, a total number of frames and a reproduction time length, decoding parameters, a title still image, a representative still image, a comment, and a search keyword. 12. The compressed moving picture decoding / display according to claim 10, further comprising a header creating means for adding a variable length area in the header which can be added as compressed moving picture stream information used at the time of retrieval. apparatus. 13. A scene change information storage means for reading a scene change detection result file in a predetermined format in which the frame numbers of start and end frames of each scene in a compressed moving image stream are recorded, and outputting a frame still image decoding instruction. And a frame still image storage means for temporarily storing the decoded still image by reducing or compressing the decoded still image, and sequentially outputting the decoded still image, and information on the scene start / end frame numbers, the number of scenes, and the image size of each scene. 13. The apparatus according to claim 1, further comprising a header creating unit for creating a variable-length header, and a file creating unit for combining outputs of the header creating unit and the frame still image storage unit to create an index image file. The compressed video decoding / display apparatus according to the above. 14. An index image list display means for reading an index image file in which a still image extracted from a compressed moving image stream and a frame number of the still image are summarized, and displaying a list of index images, and an index displayed in the list. 14. The compressed video decoding / display apparatus according to claim 1, further comprising: a playback section determination unit that determines a playback section corresponding to an index image arbitrarily selected from the image group. 15. A scene change detection result file in a predetermined format in which the frame numbers of the start and end frames of each scene in the compressed moving image stream are recorded, and a frame still image display instruction or a moving image reproduction instruction is output. A scene change information storage unit, an index image list display unit for reducing the still images sequentially output from the cue reproduction / reproducible compressed moving image decoding / display unit, and displaying the reduced list, and an index image list display A playback section determining means for determining a playback section corresponding to an index image arbitrarily selected from the group with reference to the scene change information; determining a moving image section playback instruction and a frame still image display instruction; 13. A compressed moving image according to claim 1, further comprising: display content control means for controlling the compressed video. Decoding / display device. 16. The compressed moving image according to claim 14, which has a frame-forward playback function of the compressed moving image stream.
Decoding and displaying means, while decoding the compressed video stream, all the frame images included in the scene corresponding to one index image arbitrarily specified in the index image displayed in the list or a continuous portion thereof, It has a detailed frame image list display means for thinning out at equal intervals, displaying a list in a reduced size, and a normal moving image reproduction, frame-by-frame moving image reproduction function from an arbitrary designated index image in the index images displayed in the list, and a detailed frame Use the image list display function to select a frame image that is not displayed as an index image, replace that frame image with the previously specified index image, add it as a new index image, Index image that can delete any index image in the index image And the step, an index image file editing means capable of re-editing the index image file in accordance with the index image modification, or the scene change detection result file editing means for re-editing the scene change detection result file according to the index image modification Compressed moving image
Easy editing device. 17. The compression according to claim 16, further comprising a scene search keyword registration unit for adding a variable length scene search keyword registration area corresponding to each scene in the index image file or the scene change detection result file. Simple moving image editing device. 18. A plurality of continuous index images selected from the index image group displayed in a list are stored as one moving image reproducing section, and the reproducing order of the plurality of set moving image reproducing sections is determined. 17. A simple editing means which regards each moving image playback section as one scene, extracts a first frame number and a last frame number of the scene, and arranges them based on a playback order to make simple editing information. Or a simplified compressed moving image editing apparatus according to claim 17. 19. A list of index images of a plurality of compressed moving image streams can be displayed, and a plurality of continuous index images selected from a list of index images displayed as one moving image playback section. The playback order of a plurality of set moving image playback sections is determined, each moving image playback section is regarded as one scene, and a compressed moving image stream file name corresponding to this scene and the first and last frame numbers of the scene are stored. 18. The compressed moving image simple editing apparatus according to claim 16, further comprising a simple editing unit that extracts the information and arranges them on the basis of the reproduction order and uses the information as simple editing information. 20. A simple editing means having a function of creating simple editing information to which a variable-length scene search keyword area is added in correspondence with a moving image file name and a reproduction start / end frame number. 20. The simplified compressed moving image editing apparatus according to claim 18, wherein 21. A simplified editing information storage means for reading a simplified editing information file created by the simplified compressed moving image editing apparatus according to claim 18 and storing the same, and a compressed moving image stream A file management unit for managing a file for cueing reproduction and a compressed moving image stream based on the simple editing information stored in the simple editing information storage unit; and the simple editing information storage unit 16. The compressed video decoding / display device according to claim 1, further comprising: decoding control means for performing decoding control in accordance with a reproduction section output from the compressed video. 22. Index image list display means for displaying a list of still images of the first frame of each scene recorded in the simple edit information stored in the simple edit information storage means, and 22. The compressed moving picture decoding / display apparatus according to claim 21, further comprising a playback section determining means for determining a compressed moving picture file name and a playback section corresponding to the frame image selected from the above. 23. An index image with simple editing information for reducing or compressing the first frame still image of each scene recorded in the simple editing information into one file, and creating a file to which the simple editing information is added as a header. 21. The simplified compressed moving image editing apparatus according to claim 18, further comprising a file creation unit. 24. A simple edit information storage means for reading the simple edit information of the index image file with the simple edit information created by the compressed moving image simple edit apparatus according to claim 23 and storing the same, and an index with the simple edit information. Index image list display means for displaying a list of index images of image files, and playback section determination means for determining a compressed moving image file name and a playback section corresponding to a frame image selected from the list of index images. The compressed video decoding / display device according to any one of claims 1 to 15, wherein 25. A compressed moving picture decoding and decoding device having the same function as the compressed moving picture decoding and displaying device according to claim 21 or 24.
24. The simplified compressed video editing apparatus according to claim 19, further comprising a display unit. 26. A file management means for managing a compressed moving image stream and ancillary files such as a scene change detection result file, an index image file, a cue reproduction table file and a simple editing information file, and the file management means. Keyword management means for collectively managing all keywords of all the compressed moving image streams, and search means for performing a compressed moving image stream search or a scene search within a compressed moving image with reference to a keyword added to a predetermined file. 25. The compressed moving image decoding apparatus according to claim 21, further comprising a search result display unit for displaying text information such as a file name and a frame number as search results and still image information such as an index image.
Display device. 27. File management means for managing a compressed moving picture stream file and attached files such as a scene change detection result file, an index image file, a cue reproduction table file, and a simple editing information file, and a retrieval according to a request from a client. Device having keyword search means for performing a search, a communication control means for transmitting a file, a stream or a part of a stream in response to a request from a client, and a command transmission control means for transmitting a search / data transfer request to the server device Client device having communication control means for receiving data necessary for reproducing moving image information requested by a search / browser, and compressed moving image decoding / reproducing means for decoding received data and reproducing an image Are connected by a communication line. 27. Any of up to 15 or 27.
3. The compressed video decoding / display device according to 1.