JP2008193203A - Digital video information data generation apparatus, digital video information recording apparatus, digital video information reproducing apparatus, and digital video information data generation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To display a caption immediately after reproduction even if random access is performed when digital broadcast is recorded, and to simultaneously display a caption during rewind/fast forward reproduction. <P>SOLUTION: Caption data are arranged between a head packet and an end packet of TS packets constituting I pictures existing in a period between a display start time and a display end time of a caption, and the caption is decoded and displayed from the caption data read out first during random access reproduction, and during the rewind/fast forward reproduction, the caption is decoded and displayed from caption data read out simultaneously with the I pictures. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a video information apparatus that records and reproduces video information of a digital broadcast on a video information recording medium. Generation of a transport stream of a video information apparatus capable of displaying subtitles during fast forward / reverse playback, and recording / reproduction thereof It is about.

When playing back encoded video data (video data) written on an optical disc in a format compliant with the MPEG system for digital broadcasting, video data is usually used as a method for displaying subtitles during fast forward / reverse playback. A method has been proposed in which playback is performed from a time point that is set in advance from the playback start point (for example, Patent Document 1).
JP 2002-290895 A (FIGS. 1 and 2, paragraphs 0038 to 0040, 0043)

  As satellite digital broadcasting and terrestrial digital television broadcasting become more popular, recording devices capable of recording these digital broadcast signals are gradually becoming more popular. Unlike conventional analog broadcasts, these digital broadcasts superimpose video data, audio data, caption / text super data, digital data of electronic program guides, etc. on the transport stream (hereinafter abbreviated as TS). Has been sent out. As a recording apparatus for recording the TS, there are a D-VHS, a DVD recorder, and a Blu-ray Disc (hereinafter abbreviated as BD) recorder, but the recording formats are different for each recording apparatus. For example, D-VHS and BD recorders adopt a method of recording a digital broadcasting TS as it is, or selecting and recording a part of the TS, and DVDs use a high definition TS (hereinafter abbreviated as HD). Once the decoding process is performed, the program stream (hereinafter abbreviated as “PS”) of Standard Definition (hereinafter abbreviated as “SD”) is encoded again and then recorded. There is no problem when the above-described recording apparatus records a digital broadcast on which caption display is not superimposed. However, when caption display is superimposed, various problems occur.

  In the case of a D-VHS or BD recorder, since the broadcast TS is recorded as it is or a partial TS selected as a part thereof is recorded, the same TS as when it was broadcast is output as it is during playback. Therefore, it is possible to turn on / off captions for broadcasts with captions, as in the case of receiving broadcasts. However, there is an adverse effect caused by recording the same TS as the broadcast. In general, fast-forward and fast-rewind playback of MPEG-encoded video data is realized by reading and decoding only the I picture data at the head of the GOP. Since the position where the caption data is superimposed is not determined, the data for displaying the caption cannot be read from the disc during fast forward and fast reverse reproduction. For this reason, it is impossible to measure the timing for returning from fast-forward / fast-reverse playback to normal playback while watching the subtitles. In addition, since it is not possible to specify in which part of the TS the caption display data is superimposed, reproduction from a desired position on the TS by the reproduction of a desired scene (called random access), which is a characteristic of the disk device, When playback is performed, there is a problem in that subtitles cannot be displayed until the next subtitle display data can be read.

  For example, when recording on a DVD recorder, video data is converted from HD to SD, and audio data is converted from MPEG2 Audio AAC (Advanced Audio Coding) (hereinafter abbreviated as AAC) to Dolby AC-3 (Audio Code number 3). It is necessary to convert to a system (hereinafter abbreviated as AC-3) or linear PCM (linear pulse code modulation: hereinafter abbreviated as LPCM), and further to PS.

  Video data and caption data encoded in MPEG2 of a digital broadcast signal are demodulated into a digital video signal (hereinafter referred to as a video signal) and once expanded on a memory, then the caption is synthesized, and the video signal on which the caption is superimposed is generated. Generated. Since the video signal on which subtitles are superimposed is HD resolution, it is resized to SD resolution and then encoded again with MPEG2.

  Video data on which subtitles resized to the SD resolution are superimposed is packetized into a video PS, and further converted into a TS together with the audio PS, and then recorded on a DVD-R or DVD-RW. When such recording is performed, subtitles are always displayed, and there is a disadvantage that subtitles cannot be erased for users who do not need subtitles.

  The present invention has been made to solve the above-described problems. When a digital broadcast is recorded, ON / OFF of subtitle display can always be controlled, and subtitles are displayed even during fast forward / rewind playback. It is another object of the present invention to provide a video information generation apparatus capable of immediately starting subtitle display even during random access playback, and a video information recording / playback apparatus using the same.

A digital video information data generating device according to the present invention includes broadcast data separation means for separating caption data, video data, and audio data of a transport stream of digital broadcast, video decoding means for decoding the video data of digital broadcast, Video encoding means for encoding the decoded video data into video data in a recording medium format, video packetizing means for packetizing the encoded video data and adding a video time stamp, and subtitles for the digital broadcast Caption control means for detecting a time stamp for inserting data, caption decoding means for decoding the caption data of the digital broadcast, and caption encoding means for encoding the decoded caption data into caption data in a recording medium format; Packetize the encoded subtitle data And includes a subtitle packetizing means for adding a new subtitle timestamp, and a transport stream unit for generating a transport stream of subtitle and video packets and the recording medium formatted by superimposing,
In the transport stream means, the subtitle packet is superimposed between video packets constituting an I picture existing between a subtitle display start time and an end time based on the video time stamp and the subtitle time stamp. It is characterized by doing.

  According to the present invention, the first TS packet and the last TS of the TS packet constituting the I picture to which the time stamp between the start time and the end time of the display of the caption data superimposed on the digital broadcast is added. Since the subtitle data encoded in the format of the recording medium is superimposed and recorded between the packets, fast-forward playback and fast-rewind playback with the subtitles displayed are possible, and playback from any position is possible with random access. Even in this case, subtitles can be displayed in a short time.

  As a method for recording video data of digital broadcasting, there are a method of recording the broadcast video data as it is, and a method of decoding the compressed video data once, changing the bit rate and encoding method, and encoding again. Similarly, for audio data (audio data), a method for recording broadcast audio data as it is, a method for decoding compressed audio data, and encoding again by changing an encoding bit rate and an encoding method. There is a method.

In the following description, the video data of the broadcast TS is encoded with MPEG2, the encoding method when encoding again is MPEG4, and the audio data of the broadcast TS is encoded with AAC. In the case of encoding again, the description will be made assuming that it is encoded with AC-3 and recorded on the optical disc.
Although an optical disc recording / reproducing apparatus using an optical disc as a video information recording medium will be described as an example, the video information recording medium may be a recording medium such as a hard disk or a semiconductor memory.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a configuration example of a transport stream conversion apparatus according to Embodiment 1 of the present invention. In general, a TS in digital broadcasting is composed of data packets such as video packets, audio packets, caption / text super packets, PSI (Program and System Information) packets, and electronic program guides (hereinafter abbreviated as EPG). The TS analysis unit 1 analyzes the input TS PSI packet. Based on PAT (Program Association Table) and PMT (Program Map Table) information included in the PSI packet, the PID (Packet ID) to be selected is read by the PID filter 2 and the PID filter 2 is set. The PID filter 2 extracts video data, audio data, subtitle / character super data based on the set PID information, converts them into PES packets, the PES video packet 1 is sent to the video processing unit 3, and the PES audio packet 1 is audio. The PES subtitle / character super packet 1 is sent to the subtitle processing unit 5 to the processing unit 4.

  FIG. 2 is a block diagram showing the configuration of the video processing unit 3 shown in FIG. The video data of the PES video packet 1 output from the PID filter 2 is decoded into a video format signal such as YCbCr by the video decoder 7. The decoded video signal is encoded into MPEG4-AVC video format data by the video encoder 8, further PES packetized by the video PES packetizer 9, and output as a PES video packet 2.

  FIG. 3 is a block diagram showing the configuration of the audio processing unit 4 shown in FIG. The audio data of the PES audio packet 1 output from the PID filter 2 is decoded into a linear PCM digital signal by the audio decoder 10 and encoded into AC-3 by the audio encoder 11. Further, the encoded audio data is PES packetized by the audio PES packetizer 12 and output as a PES audio packet 2.

  FIG. 4 is a block diagram showing the configuration of the caption processing unit shown in FIG. The subtitle control unit 13 extracts subtitle data from the PES subtitle / character superpacket 1 output from the PID filter 2. Here, the subtitle is information displayed in synchronization with the video, and for example, subtitle information of a movie corresponds to this. The character superimposition is information displayed asynchronously with the video, and for example, news bulletin corresponds to this. In general, when recording a digital broadcast, only the subtitle is recorded, and the superimpose is not recorded. Therefore, the subtitle control unit 13 extracts the subtitle data and its time stamp, the subtitle decoder 14 decodes the video signal into a video signal format such as YCbCr, RGB, etc., and the subtitle encoder 15 converts the subtitle data and the time stamp into a subtitle data format defined by the recording medium. Converted. Next, the data encoded in the format of the recording medium is added with subtitle display time information, is converted into a PES packet, and is converted into a PES subtitle packet 2. Hereinafter, the method will be described in detail.

A time stamp (Presentation Time Stamp: hereinafter abbreviated as PTS) of video data re-encoded by the video encoder 8 is different from the PTS at the time of broadcasting, and a new video PTS based on a new system clock is added. . Since the caption display is displayed in synchronization with the reproduction of the video signal, the PTS added to the re-encoded caption data is also added based on the system clock of the video data. For example, for the same frame, the relationship between the PTS (v_before) of the video before being re-encoded and the PTS (v_after) after being re-encoded is
PTS (v_after) = PTS (v_before) + ΔT (1)
In the case of the following equation, the relationship between the subtitle PTS (st_before) before re-encoding and the sub-title PTS (st_after) after re-encoding is
PTS (st_after) = PTS (st_before) + ΔT (2)
Indicated by That is, the audio PES packetizer 12 and the subtitle PES packetizer 16 generate a PES packet by using the system clock generated by the video encoder 8 as described above as a new system clock.

  The subtitle PES packetizer 16 generates a new subtitle PTS from the PES data extracted from the subtitle control unit 13 and the system clock information sent from the video processing unit 3, and adds it to the PES packet. The PTS added here is used as the timing for starting subtitle display in the same manner as subtitle data for digital broadcasting.

  Next, a method for generating a PES packet for storing caption data will be described in detail. The period for displaying subtitles is described in the subtitle control information attached to the subtitle data as in the case of digital broadcasting. Hereinafter, description will be made assuming that the display start time of subtitle data is “st_start_time” and the display end time is “st_end_time”. The display period information of the caption data is extracted from the caption control information in the caption data by the caption control unit 13. In general, the display time of subtitles is often set to about several seconds to 10 seconds.

FIG. 5 is a diagram illustrating a temporal relationship between the PTS of the I picture and the PES data of the caption, and illustrates the temporal relationship between the caption display time and the I picture displayed therebetween. In FIG. 5, PTS (v_I_1), PTS (v_I_2), PTS (v_I_3), and PTS (v_I_4) indicate the PTSs of the four I pictures.
Hereinafter, the display period of caption data, that is, the interval between “st_start_time” and “st_end_time” is 2 seconds, the video GOP length is 0.5 seconds, and the video data I displayed while the caption is displayed A case where there are four pictures will be described.

  The caption encoder 15 generates caption data whose display end time is “st_end_time”, and the caption PES packetizer 16 generates a PES caption packet PES (st_start_time) 17 whose PTS is “st_start_time” for the caption data. To do. Next, a PES video packet PES (v_I_1) 18 having a display start time “PTS (v_I_1)” and a display end time “st_end_time” is created. Here, the subtitles displayed by decoding these two PES packets are exactly the same, and the data excluding the subtitle control information is exactly the same. Similarly, the display start times are PTS (v_I_2), PTS (v_I_3), and PTS (v_I_4), respectively, and the display end time is “st_end_time”. , PES (v_I_4) 21 are sequentially created. As described above, the caption PES packet converted into the recording medium format is generated for the caption data of the digital broadcast to which the PTS of “st_start_time” is added.

  Next, a method for multiplexing the subtitled PES packet generated above will be described in detail. In FIG. 1, the multiplexer 6 again multiplexes the encoded video data, audio data, and caption data into one TS, but the video processing unit 3 prevents the video and audio from being interrupted during reproduction. Then, the PES video packet 2 and the PES audio packet 2 PES subtitle packet 2 that have been made into PES packets by the audio processing unit 4 and the subtitle processing unit 5 are rearranged, and a new PSI packet in the format of the recording medium is added.

FIG. 6 is a diagram illustrating the state of the TS packet at the head of the GOP on which the caption data is superimposed. The structure of the GOP of the TS file on which the PES (v_I_1) 18 to which the PTS (v_I_1) is added by the multiplexer 6 is superimposed. Is shown. Each square in the figure represents a TS packet, and each shaded square represents a TS packet other than video and subtitles. V (I_1) to V (I_9) are TS packets constituting the I picture of the GOP, and ST (1) to ST (4) are TS packets constituting the PES (v_I_1) 18. In the multiplexer 6, the TS in FIG. 6 is created by superimposing data so as to satisfy the following two conditions.
(1) The first TS packet constituting the PES packet of caption data to which the same PTS as that of the I picture is added is placed after the TS packet constituting the head of the I picture of the GOP.
(2) The last TS packet constituting the PES packet of caption data to which the same PTS as that of the I picture is added is arranged before the last TS packet constituting the I picture of the GOP.
That is, in the case of FIG. 6, the multiplexer 6 is subtitled so that the ST (1) packet is placed after the V (I_1) TS packet and the ST (4) is placed before the V (I_9). Superimpose data. The data is superimposed on the above conditions only for caption data to which the same PTS as the PTS of the I picture is added. In FIG. 5, PES (v_I_1) 18, PES (v_I_2) 19, PES (v_I_3) 20 , PES (v_I_4) 21 is a target, and PES (st_start_time) 17 is a target.

  In this description, the PTS of the I picture and the PTS (PES (v_I_1), etc.) of the caption data to be superimposed are set to the same time, but it is not always necessary to set the same time, and is set to a time near the PTS of the I picture. It doesn't matter.

  As described above, video data, audio data, and caption data included in digital broadcasting TS data are converted to TS data of the recording / playback apparatus and recorded on the recording medium. At this time, the data of the digital broadcasting TS is recorded. In addition, information necessary for random access playback, fast forward, fast reverse playback, etc. specific to the recording / playback apparatus is recorded on the recording medium.

  Decoding of data encoded by a method such as MPEG2 must be performed from the first I picture of the GOP. Therefore, in order to perform random access, fast-forward playback, etc., access information is required to know at which position in the TS data recorded on the optical disc the I picture exists. FIG. 7 is a diagram showing the data structure of an access information file necessary for random access, and shows the structure of an access information file indicating where in the TS file an I picture exists.

  “Num_of_GOP” represents the total number of GOPs present in the TS file. The for loop in the next line is repeated as many times as “num_of_GOP”. “PTS_I (k)” means the PTS time of the first I picture of the kth GOP, and “SPN_I (k)” means the number of packets from the beginning of the TS file to the first TS packet of the I picture. “I_size (k)” means the number of packets from the first TS packet of the GOP to the last TS packet constituting the I picture. These data are converted together with the TS file into a TS (hereinafter abbreviated as DVDTS) of the optical disc recording / reproducing apparatus, and recorded on the optical disc.

  Next, a method for reproducing from an arbitrary position of the TS file recorded on the optical disc will be described. In the description, it is assumed that data on the optical disk is read out by the optical disk recording / reproducing apparatus. However, a semiconductor memory such as FLASH or other optical drives may be used.

  FIG. 8 is a diagram showing the configuration of a playback apparatus that plays back TS data recorded on an optical disk. When a playback instruction is issued from an arbitrary position from the user, the CPU 23 reads the access information file of FIG. 7 from the drive 22 and is closest to the playback time specified by the user and exists before the specified time. Search for “PTS_I (k)”. Next, the drive 22 is instructed to read data from the “SPN_I (k)” th packet counted from the head of the TS file to be reproduced. The read PES packet data of the DVDTS is separated by the PID filter 24 and sent to the video decoder 26, the audio decoder 27, and the caption decoder 28. The detailed operation of the data separation is the same as that described in FIG. Omitted. The video signal combined by the video decoder 26 and the caption data decoded by the caption decoder 28 are superimposed by the video mixer 29 and output to the display device 30 connected to the outside.

  FIG. 9 is a diagram showing the relationship between the reproduction designation position and the PTS of the superimposed caption data, and is a diagram showing the relationship between the time when the user designates reproduction and the display time of the caption data. When there is a time relationship as shown in FIG. 9, “PTS_I (k)” searched by the CPU 23 is PTS (v_I_1). Since the video data can be decoded only from the I picture at the head of the GOP, the video decoder 26 does not output the decoded video until the picture at the time specified by the user, and the video from the picture after the specified time is not output. Output. At this time, since the PES (v_I_1) 18 is superimposed on the GOP to which the PTS of the PTS (v_I_1) is added as shown in FIG. 6, it is possible to display subtitles from the time specified by the user.

  On the other hand, as in the present embodiment, subtitle data having the same display data is superimposed on each I picture included in the time between “st_start_time” and “st_end_time”, and further, the first TS constituting the I picture By superimposing the subtitle TS packet after the packet, the subtitle can be displayed immediately after random access.

  On the other hand, if only PES (st_start_time) 17 is superimposed on the TS file, such as subtitle data superimposed on the digital broadcast, the subtitles that are displayed when playing from the beginning of the TS file are random access. Can no longer be displayed. For this reason, the user may not be able to grasp the content of the video being reproduced and displayed until the display of the next subtitle starts.

  Next, playback methods for fast forward playback and fast reverse playback will be described. In general, in fast forward and fast reverse playback, a method of decoding and displaying only an I picture is common. When a fast-forward command is issued from the user, the CPU 23 searches the access information file for “PTS_I (k)” after the current playback time, and “SPN_I (k) from the beginning of the TS file being played back. A command to read “I_size (k)” TS packets from the “th TS packet” is issued to the drive 22. Subsequent operations are the same as those in the random access reproduction, and thus description thereof is omitted.

  When the I picture of “PTS_I (k)” is displayed, the CPU 23 causes the drive 22 to display “I_size (k + 1)” from the “SPN_I (k + 1)” th TS packet from the beginning of the TS file being reproduced. A command to read “number of TS packets” is issued. Thereafter, in the same manner, only I picture data is read one after another, decoded and displayed. However, since the interval for reading out the I picture varies depending on the number of times specified by the user and the access speed of the drive, not all I pictures are reproduced. Fast forward playback is performed as described above.

  In this way, fast-forward / fast-rewind playback is performed by selecting and reading out only I-picture data. The fast-rewinding operation is the same except that the PTS of the I picture to be read is reproduced in the past direction, and a description thereof will be omitted. If only the PES (st_start_time) 17 is superimposed on the TS file like the caption data superimposed on the digital broadcast, the PES (st_start_time) 17 exists in the TS packet of the I picture read by chance. Except for cases, subtitles cannot be displayed. On the other hand, as in the present embodiment, subtitle data having the same display data is superimposed on each I picture included in the time between “st_start_time” and “st_end_time”, and further, the first TS constituting the I picture By superimposing the subtitle TS packet between the packet and the final TS packet, the subtitle can always be displayed even during fast-forward playback.

  As described above, the caption data of digital broadcasting is superimposed on the I picture existing while the caption is displayed, between the first TS packet and the last TS packet constituting the I picture. Thus, for random access playback, subtitles can be displayed immediately after playback, and subtitles can always be displayed even when fast forward / rewind playback is performed.

It is a block diagram which shows the structural example of the transport stream converter by Embodiment 1 of this invention. It is a block diagram which shows the structure of the video processing part of FIG. It is a block diagram which shows the structure of the audio processing part of FIG. It is a block diagram which shows the structure of the caption processing part of FIG. It is a figure which shows the time relationship between PTS of I picture and PES data of subtitles by Embodiment 1 of this invention. It is a figure which shows the state of TS packet of the GOP head with which the subtitle data by Embodiment 1 of this invention was superimposed. It is a figure which shows the data structure of the access information file required in order to perform random access by Embodiment 1 of this invention. It is a block diagram which shows the structure of the reproducing | regenerating apparatus which reproduces | regenerates the TS file recorded on the recording medium by Embodiment 1 of this invention. It is a figure which shows the relationship between the reproduction | regeneration specification position by Embodiment 1 of this invention, and PTS of the superimposed subtitle data.

Explanation of symbols

  1 TS analysis unit, 2 PID filter, 3 video processing unit, 6 multiplexer, 13 subtitle control unit, 14 subtitle decoder, 15 subtitle encoder, 16 subtitle PES packetizer, 22 DVD drive, 23 CPU, 26 video decoder, 27 audio decoder , 28 subtitle decoder, 29 video mixer, 30 display device

Claims (4)

First data separation means for separating caption data, video data, and audio data of a transport stream of digital broadcasting;
First video decoding means for decoding video data of the digital broadcast;
First video encoding means for encoding the decoded video data into video data in a recording medium format;
First video packetizing means for packetizing the encoded video data and adding a video time stamp;
Subtitle control means for detecting a time stamp for inserting the subtitle data of the digital broadcast;
First subtitle decoding means for decoding the digital broadcast subtitle data;
Subtitle encoding means for encoding the decoded subtitle data into subtitle data in a recording medium format;
Subtitle packetizing means for packetizing the encoded subtitle data and adding a new subtitle time stamp;
Transport stream generating means for generating a transport stream of the recording medium format by superimposing the caption packet and the video packet;
In the transport stream generating means, based on the video time stamp and the subtitle time stamp, the subtitle packet is present between video packets constituting an I picture that exist between a subtitle display start time and an end time. A digital video information data generation device characterized by superimposing.   A digital video information recording apparatus comprising the digital video information data generating apparatus according to claim 1. The subtitle packet and video of the recording medium format are recorded from the transport stream recorded on the recording medium by superimposing the subtitle packet between the video packets constituting the I picture, which exist between the display start time and the end time of the subtitle. A second data separation means for separating packets;
Second video decoding means for decoding a video signal from a video packet in the recording medium format;
A digital video information reproducing apparatus comprising: second subtitle decoding means for decoding a subtitle signal from a subtitle packet in the recording medium format. A video decoding step of decoding the digital broadcast video data;
A video encoding step of encoding the decoded video data into video data in a recording medium format;
Video packetizing step of packetizing the encoded video data and adding a video time stamp;
A time stamp detecting step of detecting a time stamp for inserting the subtitle data of the digital broadcast;
Subtitle decoding step for decoding the digital broadcast subtitle data;
A subtitle encoding step of encoding the decoded subtitle data into subtitle data in a recording medium format;
Subtitle packetizing step for packetizing the encoded subtitle data and adding a new subtitle time stamp;
Based on the video time stamp and the subtitle time stamp, after the subtitle packet is superimposed between video packets constituting an I picture that exist between the subtitle display start time and end time, the subtitle packet and video A digital video information data generation method comprising: a transport stream generation step of a recording medium format for generating a transport stream by superimposing packets.

Images