JP2004221918A - Data preparation device and data reproducing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prepare compact auxiliary information data for random access to video data multiplexed in an MPEG2 TS file. <P>SOLUTION: Auxiliary information is divided into a table indicating the display continuation time of each GOP and a table indicating the number of TS packets required for reading out I frames in each GOP and display continuation time tables are collectively recorded when the display continuation time of the tables is the same time in respective GOPs. The number of TS packets for reading out I frames is divided by scale information and a value obtained by rounding up a decimal point is recorded. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
[Field of the Invention]
The present invention relates to auxiliary data for enabling easy access to video data when reproducing video data obtained by packet-multiplexing a compression-encoded video frame sequence.
[0002]
[Prior art]
As an example of auxiliary data for data access of packet-multiplexed video data, a program stream directory is defined in ISO / IEC 13818-1 (MPEG2 system standard). The auxiliary data is data used for improving data access at the time of fast-forward playback or jump playback of video data. In a playback device, in order to access data at high speed, the data is loaded into a memory inside the device, and is resident in the memory during playback of video data. For this reason, when receiving and reproducing video data via the Internet, or when reproducing video data recorded on a magneto-optical disk, the auxiliary data is first downloaded and loaded into the working memory.
[0003]
[Problems to be solved by the invention]
The above-mentioned program stream directory has a problem that the data size becomes very large in a long time. In general, a GOP (Group Of Pictures), which is a group of video frames according to ISO / IEC 13818-2 (MPEG2 video standard), is configured in units of 0.5 seconds, and approximately 14400 video data of 2 hours are used. GOP. When recording an entry of the program stream directory for each GOP, one program stream directory has a maximum of 3640 entries, so four program stream directories are required, and the size is about 260 kilobytes. When downloading this data at 1 Mbps, a little over 2 seconds is required. In addition, since a large amount of work memory area is consumed in the playback device, a large-capacity memory needs to be mounted, and the playback device becomes expensive.
[0004]
[Means for Solving the Problems]
A data generating apparatus according to the present invention (claim 1) is a device for converting video data, which is composed of a plurality of compression-encoded video frames and packet-multiplexed from data consisting of a continuous video group composed of the plurality of video frames, into packets. It is used to record and output information used to determine the display start time and / or data start position of the video group.
[0005]
According to a second aspect of the present invention, in the data creation apparatus according to the first aspect, a first information field indicating a display time of a first video frame in the video data is recorded as auxiliary data. Is what you do.
[0006]
A data creation device according to a third aspect of the present invention is the data creation device according to the second aspect, wherein the first information entry includes a plurality of first information entries as the auxiliary data. A second information field for determining the number of video groups composed of continuous video frames and a third information field indicating the sum of display durations of video frames included in the one video group are recorded. Is what you do.
[0007]
The data creation device according to the present invention (claim 4) is the data creation device according to claim 3, wherein the display duration of the video group is different from the display duration of the immediately preceding video group. This is to add one information entry.
[0008]
According to a fifth aspect of the present invention, there is provided the data creating apparatus according to the first aspect, wherein the auxiliary information includes M second information entries, and the second information entry includes a video group. A fourth information field indicating the number of a packet including the first data of the first group, and fifth information indicating a value equal to or greater than the difference between the number of the packet including the last data in the first frame of the video group and the fourth information field. And records the fields.
[0009]
In the data creation device according to the present invention (claim 6), in the data creation device according to claim 5, the fifth information field includes a packet number including last data in a first frame of the video group and the number of the packet. The value greater than or equal to the difference from the information field of No. 4 is divided by the integer value S, and the value after the decimal point is rounded up.
[0010]
A data creation device according to a seventh aspect of the present invention is the data creation device according to the sixth aspect, wherein a sixth information field indicating the integer value S is recorded.
[0011]
According to a second aspect of the present invention, in the file creation method according to any one of the first to seventh aspects, the auxiliary data and the video data according to the first to seventh aspects are input, and the video data is stored in the file. To play.
[0012]
BEST MODE FOR CARRYING OUT THE INVENTION
The data creation device according to the first embodiment of the present invention is a device equipped with a limited memory capacity, for example, a magneto-optical disk reproducing device, an Internet terminal, and a portable information terminal, which are MPEG2, ISO / IEC 14496 (MPEG4), ITU- TH. 263; H.264, etc., is input as video multiplexed data such as MPEG2 program stream or transport stream of video coded data by packet multiplexing of video signal and audio signal, and fast forward playback or jump playback is performed while analyzing the contents of the packet. Auxiliary information for enabling high-speed and easy operation is generated compactly. Further, the data reproducing apparatus according to the second embodiment of the present invention inputs data obtained by packet-multiplexing a video signal and an audio signal while referring to the auxiliary data, and decodes and reproduces the data.
[0013]
In the embodiment of the present invention, the case where packetization is performed by using an MPEG2 transport stream has been described. However, the present invention is not limited to this. For example, a program stream, ISO / IEC 14496 (MP4), IETF RFC1889, etc. It can be applied to a stream packetized by (RTP).
[0014]
Hereinafter, embodiments of the present invention will be described.
(Embodiment 1)
First Embodiment A data creation device according to a first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a functional block diagram of an example of a data creating apparatus, which includes a TS input unit 1, a TS packet counting unit 2, a PID determination unit 3, a PES packet analysis unit 4, a video header detection unit 5, an auxiliary data recording unit 6. , And a file output means 7. The TS input means 1 inputs one TS packet from a TS file. Note that the TS file may be an extended TS packet to which a 4-byte extended header preceding the TS packet is added.
[0015]
FIG. 2 shows a configuration example of a TS packet including a video signal. It should be noted that the data fields shown in FIG. 2 show only a part and do not show all the data fields. The TS packet includes a TS packet header, a PES packet header, video data, and the like. The PES packet header exists only when the PUSI (payload unit start indicator) of the TS packet header is 1.
[0016]
The TS packet starts with Sync Byte. The PID (packet ID) of the TS packet header indicates the type of data included in the TS packet, and has different values among video signals, audio signals, different video signals, and the like. When the PUSI is 1, the PES packet starts from the head of the data part of the TS packet. The PES packet contains data of one video frame.
[0017]
The PES packet header starts with a SCP (start code prefix). PTS (presentation time stamp) is a display time of a video signal (frame). The video frame starts with a picture start code and includes a PCT (picture coding type) indicating the encoding type of the video frame. The PCT includes an I picture, a P picture, a B picture, and the like.
[0018]
The TS packet counting means 2 adds the number of input TS packets. The PID determining means 3 compares the PID of the video data specified from the outside with the PID of the input TS packet. The PES packet analysis means 4 analyzes the PES packet to obtain a PTS. The video header analysis means 5 acquires the encoding type PCT of the video frame. The auxiliary data recording unit 6 inputs the packet number, PTS, and PCT from the TS packet counting unit 2, the PES packet header analysis unit 4, and the video header analysis unit 5, and collects auxiliary data. The file output means 7 formats the auxiliary data according to the data structure shown in FIG. 3 and outputs an auxiliary data file.
[0019]
FIG. 3 is a data structure diagram showing a part of the auxiliary data file. The auxiliary data file Supplemental_info has the number of For loops Nof_Entry1, Packet_ID (i) for the number indicated by Nof_Entry1, Duration_Table (i), and Size_table (i). Packet_ID (i) indicates which PID the next two tables are for.
[0020]
The Size_Table includes a Scale used for scaling, the number of loops Nof_Entry2, and Nof_TSP_GP (i) and Nof_TSP_RP (i) for the number indicated by Nof_Entry2. Nof_TSP_GP (i) indicates the number of TS packets between the TS packet including the leading byte of the i-th I picture and the TS packet including the leading byte of the (i + 1) -th I picture. Nof_TSP_RP (i) indicates the number of TS packets between the TS packet including the head byte of the i-th I picture and the TS packet including the last byte of the I picture.
[0021]
Note that Nof_TSP_RP (i) may be recorded as a value obtained by dividing by Scale and rounding up decimal places. For example, if the number of the TS packet including the first byte of the first I picture is 3 and the number of the TS packet including the last byte of the I picture is 102, the number of TS packets between them is 100, but the Scale is In the case of 8, Nof_TSP_RP (1) becomes 13.
[0022]
Nof_TSP_RP is the number of TS packets necessary to read all video data constituting an I picture. It is common that a playback device reads a fixed number of TS packets by one file access. Since it is sufficient to be able to guarantee that the picture is read, it is possible to compress the data field length required for Nof_TSP_RP instead of reducing the accuracy by the above-described method. In this example, this field is 8 bits.
[0023]
Duration_table is composed of Time_Offset, the number of times of for loop Nof_Entry3, and Nof_GP (i) and Duration_GP (i) for the number indicated by Nof_Entry3. Duration_GP (i) indicates a difference between PTSs of two I-picture sections, that is, a display duration time. Nof_GP (i) indicates how many continuous sections are the display duration indicated by Duration_GP (i). For example, assuming that the display duration of the section is 100, 200, 200, and 100 in order, Nof_Entry3 is 3, that is, three entries, Nof_GP (1) = 1, Nof_GP (2) = 2, and Nof_GP. (3) = 1, Duration_GP (1) = 100, Duration_GP (2) = 200, and Duration_GP (3) = 100.
[0024]
In general, the display duration of a GOP (group of pictures) of MPEG2 is fixed at about 0.5 seconds, and therefore, the data size is significantly increased by this method as compared with the case where the time is described for each GOP. It is possible to compress. Time_Offset gives the display start time of the first section, which coincides with the PTS of the first I frame.
[0025]
A detailed operation flow for creating the auxiliary data having the above data configuration will be described with reference to the flowcharts of FIGS.
FIG. 4 is an operation flow of a TS file analysis process required to generate the Size_Table entry of FIG. In FIG. 4, PNC indicates the current TS packet counter, PNI indicates the TS packet number including the first byte of the I picture, Prev PCT indicates the coding type of the immediately preceding picture, and i indicates the i-th entry.
[0026]
S101: Initialize PNC, PNI, i.
S102: A TS packet is input, and one packet counter PNC is added. S103: Compare the PID of the instructed video signal with the PID of the TS packet header. If they match, the process proceeds to S104 because the TS packet is the target of the auxiliary information to be created. Therefore, the process returns to S102.
[0027]
S104: Check the PUSI of the TS packet header, and if it is 1, proceed to S105 because there is a PES packet header, and if it is 0, return to S102.
S105: The PCT included in the video data is checked. If it is an I picture, the process proceeds to S106; otherwise, the process proceeds to S110. Although omitted in this flowchart for simplification of the drawing, in the case of the first I picture, the process does not proceed to S106 or S110, but returns to S102. Before returning to S102, Prev PCT is set to a value indicating an I picture. The process proceeds to S106 when the second and subsequent I pictures are detected.
[0028]
S110: If the immediately preceding picture coding type PrevPCT is an I picture, the process proceeds to S111; otherwise, the process proceeds to S109.
S111: Substitute the difference between PNC and PNI into Nof_TSP_RP (i). Now, when i is 1, Nof_TSP_GP (1) corresponds to the number of packets between TS packets including the first byte of the second frame from the beginning of the TS file. When i is 2, Nof_TS_RP (2) is the number of packets between the TS packet including the first byte of the second I picture and the TS packet including the first byte of the frame next to the second I picture. Is equivalent to
[0029]
Note that the difference value may be a value obtained by dividing using Scale, as described with reference to FIG. Alternatively, the number of packets between the TS packet including the first byte of the I picture and the TS packet including the last byte of the I picture may be obtained. This can be easily performed by storing the number of the TS packet whose PID is Video PID immediately before the TS packet including the first byte of the frame next to the I picture.
[0030]
S106: Substitute the difference between PNC and PNI into Nof_TSP_GP (i). Now, if i is 1, Nof_TSP_GP (1) corresponds to the number of packets between TS packets including the first byte of the second I picture from the beginning of the TS file. When i is 2, Nof_TSP_GP (2) corresponds to the number of packets between TS packets including the first byte of the second and third I picture. In this step, PNC is substituted for PNI.
[0031]
S107: Record Nof_TSP_GP (i) and Nof_TSP_RP (i) obtained in S111 and S106.
S108: The counter i of the entry is incremented by one.
[0032]
S109: Substitute PCT into PrevPCT. Then, the process returns to S102.
The above steps are repeated until the last TS packet.
FIG. 5 is an operation flow of a process of analyzing a TS file required to generate Duration_Table of FIG. In FIG. 5, i represents the i-th entry.
[0033]
S201: i, Initialize Nof_GP (1).
S202: Input a TS packet.
S203: Compare the PID of the instructed video signal with the PID of the TS packet header. If they match, the process proceeds to S204 because the TS packet is the target of the auxiliary information to be created. Therefore, the process returns to S202.
[0034]
S204: Check the PUSI of the TS packet header, and if it is 1, proceed to S205 because there is a PES packet header, and if it is 0, return to S202.
S205: Check the PCT included in the video data, and if it is an I picture, proceed to S206; otherwise, return to S202.
[0035]
S206: Analyze the PES packet header and obtain the PTS. Although not shown for simplicity, in the case of the first I picture, this value is substituted for PTS_PG, and the process returns to S202. That is, the process proceeds to S207 from the second and subsequent I pictures.
[0036]
S207: The difference value PTS-PTS_PG of the PTS between the two I pictures is assigned to D. Then, PTS is substituted for PTS_PG.
S208: Compare the difference value D with Duration_GP (j). Although not shown, when S208 is entered for the first time, D is substituted for Duration_GP (j) instead of performing comparison, and the process proceeds to S209. When the process enters S208 for the second time or later, the process proceeds to S209 if the comparison result matches, and proceeds to S210 if the comparison result does not match.
[0037]
S209: Add the value of Nof_GP (j), and return to S202.
S210: j is incremented by one. That is, since the difference value is different from the immediately preceding difference value, an entry is newly added.
[0038]
S211: Initialize the newly added entry. That is, D is assigned to Duration_GP (j), and 1 is assigned to Nof_GP (j). Thereafter, the process returns to S202.
[0039]
The above processing is repeated up to the last TS packet. The creation of the two tables shown in the flowcharts of FIGS. 4 and 5 can be performed at the same time, which is more efficient. However, for simplicity of explanation, the explanation is separately given here.
[0040]
FIG. 6 shows an example in which the above processing is applied to virtual data. The video frame sequence is composed of a total of 14402 GOPs. The frame rate is 29.97 Hz, the first GOP is 12 frames / GOP, the 2nd to 14401th GOP is 15 frames / GOP, and the 14402th GOP is 6 frames / GOP. This corresponds to approximately two hours. Since the frame rate is constant, the number of entries in Duration_Table is three as shown in FIG. 7A, and the size of Duration_Table is 23 bytes.
[0041]
Also, as shown in FIG. 7B, the number of entries of the Size_Table is 14,402, and the size of the Size_Table is 43,214 bytes. When the TS file includes one video sequence, Nof_Entry1 = 1, and the auxiliary information data has 43 and 243 bytes. This is about 1/6 of the case where the conventional program stream directory is used.
[0042]
The data creation device according to the first embodiment of the present invention has been described.
It should be noted that the data structure of FIG. 3 is merely an example, and various modifications can be made to the arrangement order of data. Further, the Scale field does not exist, and may be divided by a fixed value. Although the field length increases, the number of a TS packet including an I picture (a numerical value indicating the number of the TS packet from the beginning) may be described instead of Nof_TSP_GP.
[0043]
Further, in the present embodiment, the description has been given as being realized by hardware, but these processes can be executed as software programs operating on the CPU.
[0044]
Further, in the present embodiment, the image data is generated for each GOP (for each I picture), but is not limited to this, and may be generated for an arbitrary number of GOPs. For example, if two GOPs are created in units of one GOP, if one GOP is at 0.5 second intervals, the table will have an accuracy of every second. Further, the frame rate of the video may be variable, and the interval between GOPs may be variable. Further, even in the case where there is no GOP structure as in MPEG4, the data creating apparatus according to the present embodiment can be used. Furthermore, the present invention can be applied to a TS or PS file in which a video stream defined by MPEG4 AVC (Advanced Video Coding) including IDR pictures and I pictures is multiplexed.
[0045]
(Embodiment 2)
A data reproducing device according to a second embodiment of the present invention will be described with reference to FIG.
FIG. 8 is a functional block diagram showing an example of the data reproducing apparatus. The auxiliary data input unit 11, the TS data input unit 12, the buffer memory 13 for storing the TS data, the auxiliary data are read, and the auxiliary data is referred to. The CPU 14 for controlling the TS data in the buffer memory 13. The decoding unit 15 receives the TS data in the buffer memory 13 based on an instruction from the CPU 14 and decodes each signal while multiplexing and demultiplexing the video signal and the audio signal. , And display means 16.
[0046]
The auxiliary data input by the auxiliary data input means 11 is based on the data structure shown in FIG. 3. Based on this data, high-speed reproduction for selectively decoding only I pictures and decoding from a certain I picture are started. A so-called trick play such as jump play is realized. The CPU 14 interprets the trick play by interpreting the external control signal 1a.
The data reproducing apparatus according to the second embodiment of the present invention has been described.
[0047]
【The invention's effect】
According to the data creation device of the present invention described above, the auxiliary data is divided into a table indicating the display duration of each GOP and the number of TS packets required to read the I frame in each GOP. If the display duration is the same between GOPs, they are recorded together, and the number of TS packets for reading out I frames is divided by the scale information to record a value rounded up to the decimal point. It becomes possible. Therefore, the time required to download the auxiliary data relating to the long TS file is reduced, and the initial delay time when downloading and reproducing the data from the network is reduced. In addition, since the auxiliary data can be batch-loaded into the memory with a small memory capacity, the cost of the reproducing apparatus can be reduced. Further, since the auxiliary data can be read by one reading from the magneto-optical disk, the waiting time until the start of reproduction can be reduced.
[Brief description of the drawings]
FIG. 1 is a functional block diagram showing an example of a data creation device according to a first embodiment of the present invention.
FIG. 2 is a diagram showing an example of a data configuration of a TS packet.
FIG. 3 is a diagram showing an example of a data structure of auxiliary data output by the data creation device according to the first embodiment of the present invention.
FIG. 4 is an explanatory diagram of an operation flow of auxiliary data creation processing according to the first embodiment of the present invention.
FIG. 5 is an explanatory diagram of an operation flow of auxiliary data creation processing according to the first embodiment of the present invention.
FIG. 6 is an example diagram for explaining an example of auxiliary data based on a virtual video sequence.
FIG. 7 is an example diagram for describing a specific example of the content of auxiliary data.
FIG. 8 is a functional block diagram showing an example of a data reproducing device according to a second embodiment of the present invention.
[Explanation of symbols]
Reference Signs List 1 TS input means 2 TS packet counting means 3 PID determination means 4 PES packet header analysis means 5 Video analysis means 6 Auxiliary data recording means 7 File output means 1a External control signal

Claims (8)

Video data composed of a plurality of compression-encoded video frames and packet-multiplexed data of a continuous video group composed of the plurality of video frames is input, and the display start time and / or data of the video group is input. A data creation device for recording and outputting information used for obtaining a start position. 2. The data creation device according to claim 1, wherein a first information field indicating a display time of a first video frame in the video data is recorded as the auxiliary data. The auxiliary information includes N first information entries, wherein the first information entry includes a second information field for determining the number of video groups composed of a plurality of consecutive video frames; 3. The data creation apparatus according to claim 2, wherein a third information field indicating a sum of display continuation times of video frames included in one video group is recorded. 4. The data creation apparatus according to claim 3, wherein when the display duration of the video group is different from the display duration of the immediately preceding video group, one of the first information entries is added. The auxiliary information includes M pieces of second information entries. The second information entry includes a fourth information field indicating the number of a packet including the first data of the video group, and a last information in the first frame of the video group. 3. The data creation apparatus according to claim 2, wherein a fifth information field indicating a value equal to or larger than a difference between a packet number including data and the fourth information field is recorded. The fifth information field is a numerical value obtained by dividing a value equal to or greater than the difference between the packet number including the last data in the first frame of the video group and the fourth information field by an integer value S and rounding up the decimal point. The data creation device according to claim 5, wherein 7. The data creation device according to claim 6, wherein a sixth information field indicating the integer value S is recorded. 8. A data reproducing apparatus which receives the auxiliary data according to claim 1 and said video data and reproduces said video data.

Images