JP3607681B2 - Sub-picture data editing apparatus and editing method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a sub video data editing apparatus and editing method for editing sub video data on a system stream including sub video data such as subtitles.
[0002]
[Prior art]
A large amount of video data, audio data, and sub-video data such as subtitles are compressed using the MPEG method, multiplexed as one bit stream, and recorded on a recording medium such as an optical disk. The MPEG system is a standard defined by the International Standards Organization (ISO) for interpretation of bitstreams and decoding processing thereof, and ISO 13818 is standardized as an MPEG2 standard.
[0003]
In addition, as a method of recording on a recording medium such as an optical disk, compression of sub-video data such as MPEG video that defines a compression encoding method of an image signal, MPEG audio that defines a compression encoding method of an audio signal, and character data An MPEG method (ISO 13818-1) and a DVD video method (DVD Specifications for Read-Only Version 1.0 August 1996) that integrate and multiplex sub-pictures that define an encoding method into a single stream including synchronous reproduction. It is prescribed.
[0004]
In the MPEG video stream, three types of encoded images of I picture, P picture, and B picture are defined. An I picture is an image that can be encoded by itself without referring to other pictures. A P picture is an image that requires a past I picture or a past P picture in terms of time when coding using forward predictive coding. Further, the B picture is an image that is predicted and encoded from past and future I pictures or P pictures in terms of time using bi-directional predictive encoding. When the B picture is encoded, Past and future I pictures or P pictures are required.
[0005]
An MPEG video stream is a collection of these pictures (a collection of GOPs). In general, a GOP is composed of a combination of an I picture, a P picture, and a B picture in order to realize random access and high coding efficiency, and the first coded image of the GOP is an I picture. Since an I picture can be decoded independently without using a reference screen, the GOP can be used as a point for random access from MPEG data.
[0006]
An MPEG audio stream is a set of AAUs (Audio Access Units: minimum units that can be decoded into audio signals one by one), and a sub-picture stream is a SPU (SubPicture Unit: sub-picture unit). = Minimum unit that can be decoded into character data one by one).
[0007]
The MPEG system stream includes an MPEG video stream, an MPEG audio stream, and a sub-picture stream, and information on when to decode and reproduce is added for each playback unit of video, audio, and sub-picture so that they can be reproduced synchronously. Thus, the encoded stream is multiplexed into one.
[0008]
There is a need for an editing system that cuts out a part of such an MPEG system stream and creates a new MPEG system stream.
[0009]
Some conventional editing systems create a new system stream by cutting out the section of the editing range designated by the user from the system stream read from the DVD disc.
[0010]
In such a case, the amount of sub-video data is much smaller than that of the video data, so it can be sent to the buffer a little before the display time during playback. Are often arranged before the video data at the same time.
[0011]
Therefore, when a new system stream corresponding to the cutout range from the editing point is created, there is no subvideo data to be displayed in the cutout range, and the subvideo data is not displayed.
[0012]
In order to solve this problem, for example, a method of separating a system stream read from a DVD disk into a video stream, an audio stream, and a sub-video stream using a demultiplexer (separation device) is considered.
[0013]
Each separated stream is displayed as a new stream by a display that reproduces image data, subtitles, etc., a speaker that reproduces audio data, a keyboard, a mouse, and other input means for the user to specify edit points. Are divided into packs of different sizes, multiplexed, and recorded on a DVD disc.
[0014]
[Problems to be solved by the invention]
According to this conventional method, there is a problem that a sub-video may not be displayed in a system stream created by cutting out a predetermined range from a system stream.
[0015]
Furthermore, in order to display sub-pictures, there is also a method of separating and editing an existing system stream and multiplexing again. However, in this method, a large amount of work time is spent on data processing such as editing and multiplexing. There is a problem of end.
[0016]
Therefore, the present invention provides a sub-picture data editing apparatus and editing method capable of editing sub-picture data so as to be synchronized with video data without separating and editing the system stream and multiplexing again. With the goal.
[0017]
[Means for Solving the Problems]
In order to solve the above-mentioned problems, the sub-picture data editing apparatus according to the present invention edits sub-picture data on a system stream including packet data other than video data, audio data, and sub-picture data. The means for determining and the existence position of the sub video data corresponding to the editing range determined by the determining means are confirmed, and if the sub video data exists outside the editing range, it exists within the editing range. An information rewriting means for writing the sub-picture data at the position of the packet data is provided.
[0018]
Further, the sub-picture data editing apparatus according to the present invention edits the sub-picture data on the system stream, and determines the editing range and the data within the editing range determined by the determining means. An editing unit that deletes data in units of frames that cannot be decoded only with data within this range, and confirms the position of sub-picture data corresponding to the editing range determined by the determining unit, and When the sub-picture data exists outside the editing range, an information rewriting means for writing the sub-picture data in the data area deleted by the editing means is provided.
[0019]
That is, in the present invention, when editing a system stream including sub-video data, if necessary sub-video data exists outside the editing range, the packet data in the editing range is rewritten to sub-video data. Therefore, the sub-picture data can be accurately displayed by eliminating the lack of display of the sub-picture data.
[0020]
Also, in the present invention, when editing a system stream including sub-video data, if necessary sub-video data exists outside the editing range, data that cannot be decoded only with data within the editing range is deleted. In addition, by writing the sub-picture data in the area, it is possible to eliminate the lack of display of the sub-picture data and display the sub-picture data accurately.
[0021]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
First, pack data constituting an MPEG system stream will be described with reference to FIG.
[0022]
The MPEG system stream is composed of a pack PA as a unit, and is composed of a pack PA, a pack header PH, and one or more packets PK.
[0023]
The pack header PH is provided with an SCR field in which a value of an SCR (System Clock Reference) serving as a time reference when decoding an encoded stream is recorded.
[0024]
The packet PK includes a packet header PKH and packet data PKD in which a stream of video data, audio data, or sub-video data is recorded. Further, the packet data PKD includes an area in which video data, audio data, and sub-video data streams are not recorded, for example, only a header is recorded. Such packet data may be used as an area in which sub-picture data existing outside the editing range is written in processing to be described later.
[0025]
Further, the packet header PKH is provided with a DTS field in which PTS (Presentation Time Stamp), which is time management information for outputting video, audio, and sub-picture included in the packet data PKD for each playback unit, is recorded. ing.
[0026]
Next, the configuration of a sub-picture data editing apparatus that cuts out a part of such an MPEG system stream and creates a new MPEG system stream will be described.
[0027]
FIG. 2 shows the configuration of a sub-picture data editing apparatus according to an embodiment of the present invention.
[0028]
This sub-picture data editing device is composed of a reading device 2 that reads a system stream from a DVD disk 1, an editing device 3 that edits the read system stream, and a playback device 4 that plays back the edited system stream. . The editing device 3 includes an editing range determination unit 5, a sub-video data replacement unit 6, and a stream cutout unit 7.
[0029]
The reading device 2 reads the system stream from the DVD disk 1 and supplies it to the editing device 3.
[0030]
The editing device 3 edits the system stream read by the reading device 2. First, the editing range determination unit 5 determines a section of video data in the system stream corresponding to the editing range as a cutout range based on a range (GOP unit) specified by the user and desired to be edited. The edit range determination unit 5 supplies information on the cutout range to the sub-video data replacement unit 6.
[0031]
The sub-video data replacement unit 6 checks the sub-video data on the system stream corresponding to the cut-out range based on the cut-out range information supplied from the edit range determination unit 5. At this time, if the sub-picture data exists outside the cutout range, the data is replaced with other packet data within the cutout range, and the replaced system stream is supplied to the stream cutout unit 7. In this case, the area of the packet data in which the sub-video data is written is an area in which only the header information is recorded, for example, where video data, audio data, and sub-video data are not recorded.
[0032]
Further, the stream cutout unit 7 provided in the editing apparatus 3 cuts out the cutout range determined by the edit range determination unit 5 from the replaced system stream and creates a new system stream.
[0033]
The playback device 4 plays back (displays, etc.) the new system stream generated by the stream cutout unit 7.
[0034]
Next, the display timing of video data and sub-video data, the clipping range on the system stream, and the sub-video data replacement method will be described with reference to FIG.
[0035]
The system stream includes a video pack V, a sub-picture pack S, and a dummy pack (pack including other packets) P. In FIG. 3, the system stream corresponding to the cutout range is the video pack V5 and later. On the other hand, since the arrangement (sub-picture pack S2) of the sub-video unit SP2 to be displayed in the cutout range is placed before the video pack V5, the dummy pack (video data) existing in the cutout range is placed. , Packet data area in which audio data and sub-video data are not recorded) P3 is replaced with sub-picture pack S2. In FIG. 3, the sub-picture unit SPn corresponds to the sub-picture pack Sn.
[0036]
The system stream in which the dummy pack P3 is replaced with the sub picture S2 is supplied to the stream cutout unit 7. The stream cutout unit 7 performs cutout processing according to the cutout range and creates a new system stream. The created system stream is sent to the playback device 4 and displayed.
[0037]
As described above, according to the sub-picture data editing apparatus, the sub-picture data existing position corresponding to the edit range (cutout range) on the system stream is confirmed, and the sub-picture data exists outside the edit range. The sub-picture data editing process is executed by writing the sub-picture data at the position of packet data (an area in which video data, audio data, and sub-picture data are not recorded) existing in the editing range. .
[0038]
Therefore, it is possible to edit existing system streams without separating, editing and re-multiplexing them, and without sub-picture display being lost. Processing can be performed.
[0039]
Next, the configuration of the second embodiment of the sub-picture data editing apparatus according to the present invention will be described with reference to FIG.
[0040]
This sub-picture data editing apparatus can display sub-picture data smoothly even if there is no packet data in which video data, audio data, and sub-picture data are not recorded at a position in time for the sub-picture display timing. It is the composition to do.
[0041]
In this sub-picture data editing apparatus, as shown in FIG. 4, a reading apparatus 12 that reads a system stream from a DVD disk 11, an editing apparatus 13 that edits the read system stream, and a reproducing apparatus 14 that reproduces the edited system stream. And is composed of.
[0042]
The editing device 13 that edits the system stream includes a configuration of an editing range determination unit 15, a detection / stream editing unit 16, a sub-video data replacement unit 17, a time information rewriting unit 18, and a stream cutout unit 19.
[0043]
In the editing device 13, the editing range determination unit 15 applies the system stream read from the DVD disk 11 supplied from the reading device 12 to the editing range specified by the user (in GOP units). A section of video data in the system stream corresponding to the edit range is determined as a cut-out range.
[0044]
Next, the detection / stream editing unit 16 removes data that cannot be decoded from the data in the cutout range determined by the edit range determination unit 15 from the cutout range in units of frames, and sub-video that is out of the cutout range. Reserve space to replace with data.
[0045]
Further, the sub-video data replacement unit 17 confirms the sub-video data on the system stream corresponding to the cut-out range based on the cut-out range information determined by the editing range determination unit 15. At this time, if the sub-picture data exists outside the cut-out range, the sub-system data is replaced with the space created by the detection / stream editing unit 16 and the replaced system stream is supplied to the time information rewriting unit 18.
[0046]
The time information rewriting unit 18 changes the time management information for decoding the system stream based on the fact that the detection / stream editing unit 17 has secured a space on the system stream.
[0047]
The stream cutout unit 19 cuts out the cutout range determined by the editing range determination unit 15 from the converted system stream and creates a new system stream.
[0048]
Details of the editing process in the detection / stream editing unit 16 will be described with reference to FIGS.
[0049]
FIG. 5 is a diagram for explaining a non-reproducible picture of the video stream at the editing point. Since a B picture is an image that is predicted and encoded from past and future I pictures or P pictures in terms of time using bi-directional predictive coding, in the first GOP of the clipping range, the B picture is Decoding processing cannot be performed, and data cannot be reproduced.
[0050]
Therefore, in the sub-picture data editing apparatus shown in the second embodiment, this B picture data is deleted, an area for writing sub-picture data existing outside the editing range is created, and system stream editing processing is executed. To do.
[0051]
Next, a method of editing data by deleting a B picture on the system stream will be described with reference to FIG.
[0052]
As shown in this figure, each picture (I picture, P picture, B picture) has a plurality of packet PAs, and the packet PA is not necessarily completed in units of pictures, and different pictures are included in one packet. May be mixed. FIG. 6 shows an example in which I picture and B picture are mixed.
[0053]
In the case of such a data structure, the detection / stream editing unit 16 first obtains the size (V (B1) _2) of the B picture deleted from the pack 6. Next, the data size of the I picture (V (I2) _1 + V (I2) _2 + V (I2) _3) is acquired and newly packetized. At this time, packets are packetized from the back in time, and the packs are arranged back-packed. Therefore, since the first pack allocates packet data PKH from the rear, if all of them are not satisfied, it is set as the data area of the padding packet PP (dummy data area). FIG. 6 shows an example in which pad 6 is assigned from pack 6 to pack 3, and pack 3 is less than one pack size, and therefore padding packet PP is inserted, and packs 1 and 2 are composed only of padding packet PP. This is a dummy data area.
[0054]
That is, in this example, the rearranged data is rearranged in the deleted B picture area to create an empty area for data, to secure an area for inserting a putting packet, and to be a dummy data area It is.
[0055]
With respect to the system stream edited in this way, the editing apparatus 13 can display the sub-picture data by executing the following sub-picture data editing process.
[0056]
FIG. 7 is a flowchart for explaining the flow of processing for editing the sub-picture data in the editing device 13.
[0057]
As shown in this figure, first, the editing range determination unit 15 determines the range of the video data section in the system stream (FIG. 7A).
[0058]
As described with reference to FIG. 6, the detection / stream editing unit 16 generates a system stream in which dummy data for replacement with sub-video data is secured and supplies the system stream to the sub-video data replacement unit 17 (FIG. 7B). )).
[0059]
The sub-picture data replacing unit 17 detects and stream editing unit when there is a sub-picture pack on the system stream of the sub-picture unit to be displayed within the clipping range outside the clipping range determined by the editing range determining unit 13. Replace with the dummy data generated in step 8. In the example described in this figure, there are two packs of sub-pictures S1 on the system stream of the sub-picture unit SP1, and the two-pack sub-picture S1 is replaced with a dummy pack to form a system stream (FIG. 7). (C)).
[0060]
The system stream replaced by the sub-picture data replacing unit 17 is supplied to the time information rewriting unit 18, and the time base reference value of the system stream and the decoding time management information are shifted backward by the frame value of the deleted B picture. Thus, the value of the SCR field of the pack header PH is rewritten.
[0061]
The time information rewriting unit 18 rewrites the DTS field of the packet header, which is the decoding time information of the MPEG video stream included in the packet, and the stream cutout unit 19 cuts out data according to the cutout range and creates a new system stream. . This system stream is supplied to the playback device 14 and played back (displayed) (FIG. 7D).
[0062]
According to this editing method of sub-picture data, when editing processing of a system stream including sub-picture data, the necessary sub-picture data exists within the editing range even if it is outside the editing range (cutout range). Data that cannot be decoded only by the data to be deleted is deleted, and processing for writing the sub-picture data in that area is performed. By this process, sub-picture data can be accurately displayed without missing display of sub-picture data.
[0063]
Therefore, in this editing process of sub-picture data, it is possible to edit the sub-picture data without separating and editing the existing system stream, and then multiplexing it again, which can greatly reduce the editing time. It becomes possible. Further, even if there is no packet data other than video data, audio data, and sub-video data in the original system stream, sub-picture data can be edited.
[0064]
The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention.
[0065]
For example, in the above embodiment, the system stream read from the DVD disc is used as the data to be edited, but the system stream read from another recording medium can be applied in the same manner. The present invention can also be applied when a digital TV signal or the like is input.
[0066]
【The invention's effect】
As described above, in the present invention, when editing a system stream including sub-video data, the editing range is determined based on the video data, and the necessary sub-video data exists outside the editing range. Is to rewrite the sub-picture data in a predetermined area within the editing range, thereby eliminating the lack of display of the sub-picture data and displaying the sub-picture data accurately.
[Brief description of the drawings]
FIG. 1 is a diagram for explaining packs constituting a system stream processed by a sub-picture data editing apparatus according to an embodiment of the present invention.
FIG. 2 is a diagram showing a configuration example of a sub-picture data editing apparatus according to an embodiment of the present invention.
FIG. 3 is a diagram for explaining an editing method in the sub-picture data editing apparatus shown in FIG. 2;
FIG. 4 is a diagram for explaining another configuration example of the sub-video data editing apparatus according to the embodiment of the present invention.
5 is a diagram showing a GOP configuration at an editing point of a system stream processed by the sub-picture data editing apparatus shown in FIG.
6 is a diagram for explaining an editing method in the sub-picture data editing apparatus shown in FIG. 4; FIG.
7 is a diagram for explaining an editing method in the sub-picture data editing apparatus shown in FIG. 4; FIG.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1, 11 ... DVD disk 2, 12 ... Reading apparatus 3, 13 ... Editing apparatus 4, 14 ... Playback apparatus 5, 15 ... Editing range determination part 6, 17 ... Sub-picture data substitution part 7, 19 ... Stream cut-out part 16 ... Detection / stream editing unit 18 Time information rewriting unit

Claims (10)

In a sub-video data editing device for editing sub-video data on a system stream including packet data other than video data, audio data, and sub-video data,
Means for determining the editing range;
The existence position of the sub video data corresponding to the editing range determined by the determining means is confirmed, and if the sub video data exists outside the editing range, the position of the packet data existing within the editing range. And an information rewriting means for writing the sub-video data to the sub-video data editing apparatus. In a sub video data editing method for editing sub video data on a system stream including packet data other than video data, audio data, and sub video data,
Based on the specified editing range, determine the clipping range of the video data in the system stream,
The sub video data corresponding to the determined cut-out range is confirmed, and if the sub-video data is outside the cut-out range on the system stream, the position of the packet data existing within the cut-out range is set. A sub-picture data editing method, wherein the sub-picture data is written. In the sub-picture data editing device for editing the sub-picture data on the system stream,
Means for determining the editing range;
Of the data within the editing range determined by the determining means, editing means for deleting data that cannot be decoded only with data within this range in units of frames;
The existence position of the sub video data corresponding to the editing range determined by the determining means is confirmed. If the sub video data exists outside the editing range, the data area deleted by the editing means is stored in the data area. Information rewriting means for writing the sub-picture data;
A sub-picture data editing apparatus comprising: The video data of the system stream includes an I picture that is encoded using only that picture without referring to another picture, a P picture that is encoded using a temporally past I picture or P picture, and time In particular, it is composed of a B picture encoded image that is predicted and encoded from past and future I pictures or P pictures, and the editing means deletes B picture data from these pictures. 4. The sub-picture data editing apparatus according to claim 3, wherein the sub-picture data editing apparatus is provided. The editing means further comprises means for inserting dummy data into the area where the B picture data has been deleted, and the information rewriting means replaces the sub-picture data existing outside the editing range in the area where the dummy data has been inserted. 5. The sub-picture data editing apparatus according to claim 4, wherein the sub-picture data editing apparatus performs processing. In order to secure an area for inserting dummy data by rearranging other data with respect to the area from which the B picture data has been deleted, the editing means uses other I picture and / or P picture data. 6. The sub-picture data editing apparatus according to claim 5, wherein rearrangement processing is performed. In the sub video data editing method for editing the sub video data on the system stream,
Based on the specified editing range, determine the clipping range of the video data in the system stream,
From the data within the determined cutout range, data that cannot be decoded only with data within this range is deleted in frame units,
The sub video data corresponding to the cutout range is confirmed, and if the subvideo data is outside the cutout range on the system stream, the subvideo data is stored in the data area deleted in units of frames. A method of editing sub-picture data, characterized by performing a writing process. The video data of the system stream includes an I picture that is encoded using only that picture without referring to another picture, a P picture that is encoded using a temporally past I picture or P picture, and time In particular, it is composed of an encoded image of a B picture that is predicted and encoded from past and future I or P pictures,
8. The data of a B picture is deleted from these pictures when deleting data that cannot be decoded only by data within the cutout range in units of frames. Sub-picture data editing method. 2. When deleting B picture data, dummy data is inserted into the deleted area, and sub-picture data existing outside the editing range is replaced in the area where the dummy data is inserted. 8. The sub-picture data editing method according to 8. When inserting dummy data, another I picture and / or P is used to secure an area for inserting dummy data by rearranging other data for the area from which the B picture data has been deleted. 10. The sub-picture data editing method according to claim 9, further comprising rearranging the picture data.

Images