JP2021034085A - Video reproducing device - Google Patents

Abstract

To provide a video reproducing device that reproduces multiple component data in synchronous even if stream data does not include time information packet required to synchronize and reproduce the multiple component data.SOLUTION: In a video reproducing device, a stream control portion acquires first information from stream data and second information from the stream data or data other than the stream data. A time information packet generation portion acquires time information from the second information and generates a time information packet including the time information by using the first information. An extraction portion extracts the time information from the time information packet. A synchronous reproduction portion includes a reference generation portion that generates a reference for synchronously reproducing multiple component data. The synchronous reproduction portion performs synchronous processing for synchronously reproducing the multiple component data according to the reference, and sets the reference according to the time information.SELECTED DRAWING: Figure 2

Description

The present invention relates to a video reproduction device.

In recent years, the resolution of television broadcasting has been increasing. Along with this, multiplexing methods such as MMT (Mpeg Media Transport) have come to be adopted for broadcasting data used for television broadcasting.

Broadcast receivers that receive television broadcasts often use storage such as hard disk drives (HDD: Hard Disk Drive) and Blu-ray discs (BD: Blu-ray (registered trademark) Disc) to receive the broadcast. You can record / play television broadcasts. In many cases, when the broadcast receiving device reproduces the television broadcast recorded in the storage, the stream data to be reproduced is reproduced by using the time information packet included in the stream data obtained by converting the television broadcast to be reproduced into data. Synchronously play back multiple component data obtained from. The time information packets used are multimedia multiplexing transport protocol (MMTP) including Network Time Protocol (NTP) packets and MPEG-H-Time Offset Table (MH-TOT). : Multimedia Multiplexing Transport Protocol) Packets, etc. The plurality of component data to be acquired are video data, audio data, subtitle data, and the like.

In the technique described in Patent Document 1, in order to solve the problem that the reference time information is missing from the content and the playback process cannot be performed correctly in time synchronization, the content recorded on the recording medium is configured. The first reference time information among the one or more reference time information included in the plurality of first packets is recorded as the reference time of the content and the playback start time of the content included in the content management information (paragraph 0030-0034). ).

JP-A-2018-133124

A video playback device that synchronizes and reproduces multiple component data acquired from stream data using the time information packet included in the stream data has the time information packet required to synchronize and reproduce multiple component data. If it is not included in the stream data, multiple component data cannot be played back in synchronization. For example, if the time information packet is not included in the stream data at all, a plurality of component data cannot be played back in synchronization. Even if the time information packet is included in the stream data, when the playback of the stream data is started from an arbitrary position, the position where the playback is started and the time information packet of a plurality of component data are included. The part between the position and the position cannot be played back in synchronization. For example, the part in front of the position containing the time information packet cannot be played back synchronously.

The present invention has been made in view of this problem. The present invention synchronizes a plurality of component data even when the stream data does not include a time information packet necessary for synchronously reproducing a plurality of component data acquired from the stream data recorded on the recording medium. It is to provide a video reproduction apparatus which can reproduce.

The video reproduction device includes a stream control unit, a time information packet generation unit, an extraction unit, and a synchronous reproduction unit.

The stream control unit acquires the first information from the stream data recorded on the recording medium, and acquires the second information from the stream data or data other than the stream data recorded on the recording medium.

The time information packet generation unit acquires time information from the second information and generates a time information packet including the time information using the first information.

The extraction unit extracts time information from the time information packet.

The synchronous reproduction unit includes a reference generation unit that generates a reference for synchronously reproducing a plurality of component data acquired from stream data. The synchronous playback unit performs synchronous processing for synchronously playing back a plurality of component data according to the reference, and sets the reference according to the time information.

According to the present invention, the time information packet required for synchronously reproducing a plurality of component data acquired from the stream data recorded on the recording medium is transferred to the stream data recorded on the recording medium or the recording medium. Generated from recorded stream data and data other than stream data. Therefore, even when the time information packet is not included in the stream data, a plurality of component data can be reproduced in synchronization.

The objects, features, aspects and advantages of the present invention will become more apparent with the following detailed description and accompanying drawings.

It is a block diagram which illustrates the video reproduction apparatus of Embodiment 1. FIG. It is a figure which illustrates the data processed by the image reproduction apparatus of Embodiment 1. FIG. It is a flowchart which illustrates the flow of the initial operation executed by the image reproduction apparatus of Embodiment 1. FIG. It is a flowchart which illustrates the flow of the decoding operation executed by the video reproduction apparatus of Embodiment 1. FIG. It is a flowchart which illustrates the flow of the output operation executed by the video reproduction apparatus of Embodiment 1. FIG.

1 Embodiment 1
1.1 Video Reproduction Device FIG. 1 is a block diagram illustrating the video reproduction device of the first embodiment. FIG. 2 is a diagram illustrating data processed by the video playback device of the first embodiment.

As shown in FIG. 1, the video reproduction device 1 of the first embodiment reads the stream data 61 and the clip information data 62 from the recording medium 2. Further, the video reproduction device 1 acquires video data, audio data, and subtitle data from the read stream data 61. Further, the video reproduction device 1 generates superimposed image data according to the acquired video data and subtitle data, and outputs the generated superimposed image data to the external liquid crystal panel 3. As a result, the liquid crystal panel 3 reproduces the superimposed video in which the video corresponding to the video data and the subtitles corresponding to the subtitle data are superimposed on each other. Further, the video reproduction device 1 generates pulse code modulation (PCM) data according to the acquired audio data, and outputs the generated PCM data to the external speaker 4. As a result, the voice corresponding to the voice data is reproduced by the speaker 4. Further, the video reproduction device 1 executes a synchronization process for synchronously reproducing the acquired video data, subtitle data, and audio data according to the read stream data 61 and clip information data 62. The video playback device 1 acquires a plurality of component data different from the three component data composed of video data, subtitle data, and audio data, and executes a synchronization process for synchronously reproducing the acquired plurality of component data. You may. The video reproduction device 1 may output the superimposed image data to a display device other than the liquid crystal panel 3. The video reproduction device 1 may output the PCM data to a sounding device other than the speaker 4.

Stream data 61 and clip information data 62 are recorded on the recording medium 2.

The recording medium 2 is a hard disk drive (HDD: Hard Disk Drive), a digital multipurpose disc (DVD: Digital Versatile Disc), a Blu-ray disc (BD: Blu-ray (registered trademark) Disc), or the like.

The stream data 61 conforms to the Type Length Value (TLV) / MPEG Media Transport (MMT) standard and the like. Therefore, the stream data 61 includes a TLV packet. The TLV packet includes a TLV header, an Internet Protocol (IP) header, a User Datagram Protocol (UDP) header, and residual data. The remaining data includes Multimedia Multiplexing Transport Protocol (MMTP) packets.

As shown in FIG. 2, the stream data 61 is used to generate a time information packet 63 necessary for synchronously reproducing video data, audio data, and subtitle data acquired from the stream data 61. Contains information 72 of 1. The first information 72 includes a packet sequence number 73 indicating the order of the first MMTP packets 71 included in the stream data 61.

The clip information data 62 is an example of data other than the stream data 61. The clip information data 62 includes data necessary for reproducing the stream data 61.

As shown in FIG. 2, the clip information data 62 is used to generate the time information packet 63 necessary for synchronously reproducing the video data, the audio data, and the subtitle data acquired from the stream data 61. The second information 81 is included. The second information 81 includes a reproduction start time 82 indicating a time when the reproduction of the stream data 61 is started.

As shown in FIG. 1, the video reproduction device 1 includes a processor 11, a video / audio output unit 12, and a memory 13. The operating system 21 and the storage driver 22 are installed in the video playback device 1.

The processor 11 executes the operating system 21 and the storage driver 22. Further, the processor 11 functions as the video reproduction processing unit 23 by executing the firmware. All or part of the video reproduction processing unit 23 may be configured by hardware that does not execute the program.

As shown in FIG. 1, the video reproduction processing unit 23 includes a stream control unit 31, a time information packet generation unit 32, a packet parser 33, a demultiplexer 34, a decoder 35, a subtitle generation unit 36, and a video subtitle superimposition. A unit 37 is provided.

The Demuxer 34 includes a take-out unit 41. The decoder 35 includes a synchronous output counter 42. The subtitle generation unit 36 includes an internal clock 43. The decoder 35 and the subtitle generation unit 36 constitute a synchronous playback unit 51. The synchronous output counter 42 and the internal clock 43 form a reference generation unit 52.

The stream control unit 31 reads the stream data 61 from the recording medium 2. Further, the stream control unit 31 acquires the first information 72 from the read stream data 61, and transmits the acquired first information 72 to the time information packet generation unit 32. Further, the stream control unit 31 transmits the read stream data 61 to the packet parser 33.

Further, the stream control unit 31 reads the clip information data 62 from the recording medium 2, acquires the second information 81 from the read clip information data 62, and sends the acquired second information 81 to the time information packet generation unit 32. Send. The stream control unit 31 may acquire the second information 81 from data other than the clip information data 62. For example, the stream control unit 31 may acquire the second information 81 from the stream data 61.

Further, the stream control unit 31 transmits a time information packet generation instruction instructing the time information packet 63 to be generated to the time information packet generation unit 32. Further, the stream control unit 31 transmits the time information packet 63 generated by the time information packet generation unit 32 that has received the time information packet generation instruction to the packet parser 33.

The time information packet generation unit 32 receives the transmitted first information 72 and second information 81. The time information packet generation unit 32 acquires the time information 92 from the received second information 81 in conjunction with receiving the time information packet generation instruction, and acquires the time information 92 using the received first information 72. The time information packet 63 including the time information 92 is generated, and the generated time information packet 63 is written to the memory 13. The time information packet 63 generated is an MMTP packet or the like including an Empeg-H-time offset table (MH-TOT). When the time information packet 63 is an MMTP packet including the MH-TOT, the time information 92 is included in the MH-TOT.

Further, the time information packet generation unit 32 starts with the packet sequence number 73 indicating the order of the first MMTP packet 71 included in the stream data 61 included in the first information 72, and the packet sequence indicating the order of the time information packet 63. Get the number 91. The packet sequence number 91 indicating the order of the acquired time information packets 63 is made smaller than the packet sequence number 73 indicating the order of the first MMTP packet 71 included in the stream data 61. Therefore, the order of the acquired time information packets 63 is set before the order of the first MMTP packet 71 included in the stream data 61. Further, the time information packet generation unit 32 acquires the time information 92 from the reproduction start time 82, which is included in the second information 81 and indicates the time when the reproduction of the stream data 61 is started. The acquired time information 92 indicates a time before the reproduction start time 82, which indicates the time when the reproduction of the stream data 61 is started. As a result, it is possible to execute the same synchronization processing as the synchronization processing that can be executed when the stream data at the head of the time information packet 63 is read.

The packet parser 33 includes an internal buffer. Further, the packet parser 33 executes data reception processing and data parsing processing.

Further, when executing the data reception process, the packet parser 33 receives the transmitted data, stores the received data in the internal buffer, and stream-controls the reception completion notification indicating that the data reception is completed. It is transmitted to the unit 31. However, when the packet parser 33 receives the transmitted data, if the internal buffer does not have a free space larger than the data size of the received data, the packet parser 33 uses the free space larger than the data size of the received data as the internal buffer. The data reception process is stopped until the data is received.

When executing the data parsing process, the packet parser 33 reads the TLV header from the data stored in the internal buffer, acquires the packet size of the TLV packet including the TLV header from the read TLV header, and acquires the acquired packet. Check if data with size is stored in the internal buffer. When the packet parser 33 confirms that the data having the acquired packet size is stored in the internal buffer, the packet parser 33 removes the TLV header, the IP header, and the UDP header from the TLV packet, and transfers the remaining data to the Demuxer 34. Send. If the packet parser 33 confirms that the data having the acquired packet size is not stored in the internal buffer, the packet parser 33 stops the data parsing process until the next data is received.

The Demuxer 34 receives the transmitted data and classifies the received data into video data, audio data, subtitle data, and control information. As a result, three component data including video data, audio data, and subtitle data are acquired from the stream data 61. A plurality of component data different from the three component data may be acquired from the stream data 61. The classified control information is MMT-Signaling Information (MMT-SI: MMT-Signaling Information) and the like.

Further, the Demuxer 34 transmits the classified video data and audio data to the decoder 35. Further, the Demuxer 34 transmits the control information related to the video data and the audio data included in the classified control information to the decoder 35. Further, the Demuxer 34 transmits the classified subtitle data to the subtitle generation unit 36. Further, the Demuxer 34 transmits the control information regarding the subtitle data included in the classified control information to the subtitle generation unit 36. Each of the control information related to the video data and the audio data and the control information related to the subtitle data is an MMT Package Table (MPT) or the like.

Further, the extraction unit 41 provided in the Demuxer 34 extracts the time information 92 from the control information including the time information 92 included in the classified control information, and transmits the extracted time information 92 to the subtitle generation unit 36. As a result, the extraction unit 41 can extract the time information 92 included in the time information packet 63 from the time information packet 63. The control information including the time information 92 is MH-TOT or the like.

The decoder 35 receives the transmitted video data and audio data. Further, the decoder 35 decodes the received video data into image data, and transmits the decoded image data to the video subtitle superimposing unit 37. If the subtitle data cannot be acquired from the stream data 61, the decoder 35 may transmit the image data to the video / audio output unit 12. Further, the decoder 35 decodes the received audio data into PCM data, and transmits the decoded PCM data to the video / audio output unit 12.

The synchronous output counter 42 provided in the decoder 35 counts the count value for synchronously reproducing the video data and the audio data. The decoder 35 executes a synchronization process for synchronously reproducing the video data and the audio data according to the count value to be counted and the presentation time indicating the time presented by the video data and the audio data. As a result, the decoder 35 transmits the image data and the PCM data so that the video data and the audio data can be reproduced in synchronization. Further, when the decoder 35 receives the transmitted time information 92, the decoder 35 sets the count value counted by the synchronous output counter 42 according to the received time information 92.

The subtitle generation unit 36 receives the transmitted subtitle data and generates the subtitle image data of the subtitle image from the received subtitle data. Further, the subtitle generation unit 36 transmits the generated subtitle image data to the video subtitle superimposition unit 37 when the time engraved by the internal clock 43 becomes the subtitle image display time indicating the time when the subtitle image is displayed. Further, the subtitle generation unit 36 instructs to hide the subtitle when the time ticked by the internal clock 43 becomes the subtitle image erasure time indicating the time when the subtitle image is erased. Is transmitted to the video subtitle superimposing unit 37.

The internal clock 43 provided in the subtitle generation unit 36 ticks the time for synchronously reproducing the video data and the subtitle data. The subtitle generation unit 36 executes a synchronization process for synchronizing and reproducing the video data and the subtitle data according to the time of engraving. As a result, the subtitle generation unit 36 transmits the subtitle image data and the subtitle non-display instruction so that the video data and the subtitle data can be reproduced in synchronization. Further, when the transmitted time information 92 is received, the subtitle generation unit 36 sets the time to be ticked by the internal clock 43 according to the received time information 92.

Therefore, the reference generation unit 52 including the synchronous output counter 42 and the internal clock 43 generates a reference including the count value and the time for synchronously reproducing the video data, the audio data, and the subtitle data. Further, the synchronous reproduction unit 51 including the decoder 35 and the subtitle generation unit 36 executes a synchronization process for synchronously reproducing the video data, the audio data, and the subtitle data according to the generated reference. Further, when the synchronous reproduction unit 51 receives the transmitted time information 92, the synchronous reproduction unit 51 sets a reference generated by the reference generation unit 52 according to the received time information 92.

The video subtitle superimposition unit 37 receives the transmitted image data and subtitle image data, superimposes the received image data and subtitle image data on each other to generate superimposition image data, and outputs the generated superimposition image data as video / audio. It is transmitted to the unit 12. Further, when the video subtitle superimposing unit 37 receives the new image data, the video subtitle superimposing unit 37 updates the image data superimposed on the subtitle image data with the received new image data. For example, when the video subtitle superimposing unit 37 superimposes the image data A and the subtitle image data X on each other to generate the superimposing image data A + X, transmits the generated superimposing image data A + X, and then receives the new image data B. Generates superimposed image data B + X by superimposing new image data B and subtitle image data X on each other, and transmits the generated superimposed image data B + X. Further, when the new subtitle image data is received, the video subtitle superimposing unit 37 updates the subtitle image data superimposed on the image data with the received new subtitle image data. For example, the video subtitle superimposing unit 37 superimposes the image data A and the subtitle image data X on each other to generate the superimposing image data A + X, transmits the generated superimposing image data A + X, and then receives the new subtitle image data Y. In this case, the image data A and the new subtitle image data Y are superposed on each other to generate the superimposed image data A + Y, and the generated superimposed image data A + Y is transmitted. When the video subtitle superimposing unit 37 receives the transmitted subtitle non-display instruction, the video subtitle superimposing unit 37 transmits only the received image data to the video / audio output unit 12.

The video / audio output unit 12 receives the transmitted superimposed image data or image data, and outputs the received superimposed image data or image data to the liquid crystal panel 3. Further, the video / audio output unit 12 receives the transmitted PCM data and outputs the received PCM data to the speaker 4.

1.2 Playback operation The video playback device 1 executes a playback operation when playing back a video. The reproduction operation includes an initial operation, a decoding operation, and an output operation. After the initial operation is completed, the video reproduction device 1 repeatedly executes the decoding operation and the output operation until the reproduction of the video is finished or stopped. The video playback device 1 executes the decoding operation and the output operation in parallel.

1.3 Initial operation FIG. 3 is a flowchart illustrating the flow of the initial operation executed by the video reproduction device of the first embodiment.

The video reproduction device 1 starts reproduction of an image when receiving a reproduction instruction. The video reproduction device 1 executes steps S100 to S104 shown in FIG. 3 in conjunction with starting reproduction of the video.

In the packet generation information preparation process of step S100, the stream control unit 31 stores the head data of the stream data 61 recorded on the recording medium 2 and the clip information data 62 recorded on the recording medium 2 as a storage driver. Read to the memory 13 via 22. When reading the head data of the stream data 61, the stream control unit 31 determines the playback start position from the file name of the stream data 61 and the beginning of the file, and the head portion for the number of seconds set from the calculated playback start position. Read the data.

Further, in the packet generation information preparation process in step S100, the stream control unit 31 extracts the packet identifier packet_id of the MMTP packet of the video data, audio data, and subtitle data to be reproduced from the clip information data 62 read into the memory 13. Then, the extracted packet identifier packet_id is notified to the Demuxer 34.

Further, in the packet generation information preparation process in step S100, the stream control unit 31 starts with the packet sequence number 73 of the first MMTP packet 71 included in the stream data 61 from the first data of the stream data 61 read into the memory 13. Packet sequence number packet_sequence_number) is acquired. Further, the stream control unit 31 acquires the reproduction start time 82 from the clip information data 62 read into the memory 13. Further, the stream control unit 31 notifies the time information packet generation unit 32 of the acquired packet sequence number 73 (packet sequence number packet_sequence_number) and the reproduction start time 82. Further, the stream control unit 31 transmits a time information packet generation instruction to the time information packet generation unit 32.

The packet identifier packet_id is information that identifies the type of data stored in the payload of the packet containing it. The packet sequence number packet_sequence_number indicates the order of the MMTP packets containing the packet_sequence_number.

In the time information packet generation process of the following step S101, the time information packet generation unit 32 includes MH- including time information 92 (time information JST_time) indicating a time set time before the notified playback start time 82. The TOT data is generated, and the M2 short section message data storing the generated MH-TOT data is generated. Further, the time information packet generation unit 32 includes the generated M2 short section message data as a payload, and subtracts 1 from the notified packet sequence number 73 (packet sequence number packet_sequence_number) to generate the packet sequence number 91 (packet sequence number packet_sequence_number). The time information packet 63, which is an MMTP packet to have, is generated. Further, the time information packet generation unit 32 generates a TLV packet including the generated time information packet 63 as a payload, writes the generated TLV packet to the memory 13, the start position of the TLV packet written to the memory 13, and the TLV packet size. , And a time information packet generation completion notification indicating that the generation of the time information packet 63 is completed is notified to the stream control unit 31.

In the time information packet transmission process of the following step S102, the stream control unit 31 transmits data corresponding to the packet size of the TLV packet notified from the head position of the notified TLV packet to the packet parser 33.

Further, in the time information packet transmission process in step S102, the packet parser 33 receives the transmitted data, stores the received data in the internal buffer, and notifies the data reception completion indicating that the data reception is completed. Notify the stream control unit 31.

In the time information packet analysis process of the following step S103, the packet parser 33 reads the TLV packet from the data stored in the internal buffer, removes the TLV header, the IP header, and the UDP header from the read TLV packet, and the remaining MMTP packet. The time information packet 63 is transmitted to the Header 34. The MMTP packet transmitted contains M2 short section message data.

In the time synchronization process of the following step S104, the Demuxer 34 receives the transmitted time information packet 63, and the time information 92 (from MT-TOT included in the M2 short section message data included in the received time information packet 63). The time information JST_time) is taken out, and the taken out time information 92 (time information JST_time) is notified to the decoder 35 and the subtitle generation unit 36.

Further, in the time synchronization process in step S104, the decoder 35 sets the count value counted by the synchronized output counter 42 because of the notified time information 92 (time information JST_time), and operates the synchronous output counter 42. To start.

Further, in the time synchronization process in step S104, the subtitle generation unit 36 sets the time to be ticked by the internal clock 43 according to the notified time information 92 (time information JST_time), and starts the operation of the internal clock 43. As a result, the synchronization process for synchronously reproducing the video data, the audio data, and the subtitle data according to the packet sequence number 73 and the reproduction start time 82 (time information JST_time) is completed.

1.4 Decoding operation FIG. 4 is a flowchart illustrating the flow of the decoding operation executed by the video playback apparatus of the first embodiment.

After completing the initial operation, the video reproduction device 1 executes steps S200 to S212 shown in FIG.

In the stream data read processing in step S200, the stream control unit 31 reads the data constituting the stream data 61 from the recording medium 2 into the memory 13 and transmits the data read into the memory 13 to the packet parser 33.

Further, in the stream data read processing in step S200, the packet parser 33 receives the transmitted data, stores the received data in the internal buffer, and notifies the data reception completion indicating that the data reception is completed. Notify the stream control unit 31.

In the following step S201, the packet parser 33 determines whether or not the accumulated amount is sufficient. At that time, the packet parser 33 reads the TLV header from the data stored in the internal buffer, and acquires the packet size of the TLV packet including the TLV header from the read TLV header. Further, the packet parser 33 determines that the accumulated amount is sufficient when the data having the acquired packet size is accumulated in the internal buffer, and determines that the accumulated amount is not sufficient otherwise. If it is determined that the accumulated amount is sufficient, step S202 is executed. If it is determined that the accumulated amount is not sufficient, step S200 is executed again. Therefore, when the packet parser 33 determines that the accumulated amount is not sufficient, the packet parser 33 stops the TLV parsing process executed in step S202 until the next data is received.

In the TLV parsing process in step S202, the packet parser 33 removes the TLV header, the IP header, and the UDP header from the TLV packet, and transmits the remaining data to the Demuxer 34.

In the data classification of the following step S203, the Demuxer 34 receives the transmitted data and classifies the MMTP packet into control message data and component data according to the data type payload_type of the payload of the MMTP packet included in the received data. .. The component data includes video data, audio data or subtitle data. The control message data includes control information. If the MMTP packet is classified as control message data, step S204 is executed. If the MMTP packet is classified as component data, step S207 is executed.

In the table confirmation in step S204, the Demuxer 34 confirms the table included in the control message data. As a result of checking the table, if the received MMTP packet contains MPT, step S205 is executed. On the other hand, if the received MMTP packet contains MH-TOT, step S206 is executed.

In the media presentation unit (MPU: Media Presentation Unit) _presentation time (MPU_PT: MPU Presentation Time) list saving process in step S205, the Demuxer 34 combines the MPU_PT of the video data to be reproduced and the corresponding MPU sequence number mpu_sequence_number. The video MPU_PT list that lists the above is saved in the memory 13. Further, the Demuxer 34 stores in the memory 13 an audio MPU_PT list that lists the combinations of the MPU_PT of the audio data to be reproduced and the corresponding MPU sequence number mpu_sequence_number.

In the MH-TOT transmission process in step S206, the Demuxer 34 extracts the time information from the MH-TOT and notifies the subtitle generation unit 36 of the extracted time information.

In the component classification in step S207, the Demuxer 34 classifies the data included in the MMTP packet into subtitle data, video data, and audio data. If the data is classified as subtitle data, step S208 is executed. If the data is classified as video data, step S209 is executed. If the data is classified as voice data, step S210 is executed.

In the subtitle data transmission process of step S208, the Demuxer 34 transmits the media fragment unit (MFU: Media Fragment Unit) data included in the subtitle data to the subtitle generation unit 36.

In the subtitle generation process of the following step S212, the subtitle generation unit 36 receives the transmitted MFU data, acquires a presentation time indicating the time presented by the subtitle data from the received MFU data, and acquires the presentation time. And the set of the subtitle image data is saved in the memory 13.

In the video data transmission process of step S209, the Demuxer 34 searches the video MPU_PT list stored in the memory 13 for the MPU sequence number mpu_sequence_number that matches the MPU sequence number mpu_sequence_number of the video data, and the searched MPU sequence number mpu_sequence_number corresponds to the MPU sequence number mpu_sequence_number. The set of MPU_PT and video data is transmitted to the decoder 35, and the MPU_PT and the MPU sequence number mpu_sequence_number are deleted from the video MPU_PT list.

In the voice data transmission process of step S210, the Demuxer 34 searches the voice MPU_PT list stored in the memory 13 for the MPU sequence number mpu_sequence_number that matches the MPU sequence number mpu_sequence_number of the voice data, and the searched MPU sequence number mpu_sequence_number corresponds to it. The pair of MPU_PT and voice data is transmitted to the decoder 35, and the MPU_PT and the MPU sequence number mpu_sequence_number are deleted from the voice MPU_PT list.

In the decoding process of step S211 following steps S209 and S210, the decoder 35 receives the transmitted video data, and if the received video data can be decoded, decodes the video data into image data and decodes the video data. The image data is stored in the memory 13.

Further, in the decoding process of step S211, if the decoder 35 receives the transmitted audio data and can decode the received audio data, the audio data is decoded into PCM data, and the decoded PCM data is used. Save in memory 13.

1.5 Output operation FIG. 5 is a flowchart illustrating a flow of an output operation executed by the video reproduction device of the first embodiment. 5 (a), 5 (b), 5 (c) and 5 (d) illustrate the flow of output operations executed by the decoder, subtitle generation unit, video subtitle superimposition unit and video / audio output unit, respectively. It is a flowchart to be done.

The decoder 35 executes steps S300 and S301 illustrated in FIG. 5A when executing the output operation.

In the output operation of step S301, when the decoder 35 stores the image data having the MPU_PT corresponding to the count value counted by the synchronous output counter 42 in the memory 13, the image data is stored in the video subtitle superimposing unit. Send to 37.

In the audio output processing of the following step S302, when the decoder 35 stores the PCM data having the MPU_PT corresponding to the count value counted by the synchronous output counter 42 in the memory 13, the PCM data is stored in the memory 13. It is transmitted to the output unit 12.

As a result, the video data and the audio data are reproduced according to the count value counted by the synchronous output counter 42. As a result, the video data and the audio data are reproduced in synchronization.

The subtitle generation unit 36 executes steps S302 and S303 shown in FIG. 5B when executing the output operation.

In the subtitle output process of step S302, when the time ticked by the internal clock 43 becomes the presentation time indicating the time presented by the subtitle image data stored in the memory 13, the subtitle generation unit 36 corresponds to the present time. The subtitle image data is transmitted to the video subtitle superimposing unit 37.

In the subtitle hiding process of the following step S303, the subtitle generation unit 36 outputs a subtitle hiding instruction instructing to hide the subtitle when the time ticked by the internal clock 43 has passed the presentation period. It is transmitted to the subtitle superimposing unit 37.

The video subtitle superimposing unit 37 executes step S304 shown in FIG. 5C when executing the output operation.

In the image superimposition process of step S304, the video subtitle superimposition unit 37 receives the transmitted image data and subtitle image data, and the received image data and subtitle image data are superimposed so that the subtitle image is superimposed on the image. Are superimposed on each other to generate superimposed image data, and the generated superimposed image data is output to the video / audio output unit 12.

As a result, the subtitle data is reproduced according to the time ticked by the internal clock 43. As a result, the video data and the subtitle data are reproduced in synchronization.

The video / audio output unit 12 executes steps S305 and S306 shown in FIG. 5D when executing the output operation.

In the video output process of step S305, the video / audio output unit 12 receives the transmitted superimposed image data, generates a signal having a signal waveform corresponding to the received superimposed image data, and generates the generated signal on the liquid crystal panel. Output to 3.

In the audio output processing of the following step S306, the video / audio output unit 12 receives the transmitted PCM data, generates a signal having a signal waveform corresponding to the received PCM data, and transmits the generated signal to the speaker 4. Output.

1.6 Effect of the invention of the first embodiment According to the invention of the first embodiment, the video data, the audio data, and the subtitle data acquired from the stream data 61 recorded on the recording medium 2 are reproduced in synchronization with each other. The time information packet 63 required for is generated from the stream data 61 and the clip information data 62 recorded on the recording medium 2. The generated time information packet 63 can be used in the same manner as the time information packet included in the stream data 61. Therefore, even when the time information packet is not included in the stream data 61, the video data, the audio data, and the subtitle data can be reproduced in synchronization. For example, even when the time information packet is not included in the stream data 61 at all, the video data, the audio data, and the subtitle data can be reproduced in synchronization. Further, when the playback of the stream data 61 is started from an arbitrary position when the time information packet is included in the stream data 61, the position where the playback is started and the time information packet of the video data, the audio data, and the subtitle data are displayed. The part between the included position can be played back in synchronization. For example, the part in front of the position including the time information packet can be reproduced in synchronization.

1.7 Modification Example In the first embodiment, the generated time information packet 63 is an MMTP packet containing the MH-TOT. However, the generated time information packet 63 may be a Network Time Protocol (NTP) packet.

In the first embodiment, the playback start time 82 notified by the stream control unit 31 to the time information packet generation unit 32 is acquired from the clip information data 62. However, the reproduction start time 82 may be acquired from the MPU_TU of the reproduction start image acquired from the MPT included in the stream data 61.

In the present invention, the embodiments can be appropriately modified or omitted within the scope of the invention.

Although the present invention has been described in detail, the above description is exemplary in all aspects and the invention is not limited thereto. It is understood that innumerable variations not illustrated can be assumed without departing from the scope of the present invention.

1 Video playback device, 2 Recording media, 31 Stream control unit, 32 Time information packet generation unit, 33 Packet parser, 34 Demultiplexer, 35 Decoder, 36 Subtitle generation unit, 37 Video subtitle superimposition unit, 41 Extraction unit, 42 Synchronous output counter, 43 Internal clock, 51 Synchronous playback unit, 52 Reference generator, 61 Stream data, 62 Clip information data, 63 Time information packet, 71 First MMTP packet, 72 First information, 73 Packet sequence number , 81 Second information, 82 Playback start time, 91 Packet sequence number, 92 Time information.

Claims (9)

A stream control unit that acquires the first information from the stream data recorded on the recording medium and acquires the second information from the stream data or data other than the stream data recorded on the recording medium.
A time information packet generation unit that acquires time information from the second information and uses the first information to generate a time information packet including the time information.
A take-out unit that extracts the time information from the time information packet,
It is provided with a reference generation unit that generates a reference for synchronously reproducing a plurality of component data acquired from the stream data, and performs a synchronization process for synchronously reproducing the plurality of component data according to the reference. , A synchronous playback unit that sets the reference according to the time information,
Video playback device equipped with. The plurality of component data includes video data and audio data.
The reference includes a count value for synchronously reproducing the video data and the audio data.
The reference generation unit includes a synchronous output counter that counts the count value.
The video reproduction unit according to claim 1 includes a decoder that performs synchronous processing for synchronously reproducing the video data and the audio data according to the count value, and sets the count value based on the time information. apparatus. The plurality of component data includes video data and subtitle data.
The reference includes a time for synchronously playing back the video data and the subtitle data.
The reference generator includes an internal clock that keeps track of the time.
Claim 1 or 2 includes a subtitle generation unit that performs synchronous processing for synchronously reproducing the video data and the subtitle data according to the time, and sets the time according to the time information. Video playback device. The first information includes the order of the first MMTP packet included in the stream data.
The video reproduction device according to any one of claims 1 to 3, wherein the time information packet generation unit acquires the order of the time information packets from the order of the first MMTP packets included in the stream data. The video reproduction device according to claim 4, wherein the order of the time information packets is earlier than the order of the first MMTP packet included in the stream data. The second information is any video playback device according to any one of claims 1 to 5, which includes a time when playback of the stream data is started. The video playback device according to claim 6, wherein the time information indicates a time before the time when the playback of the stream data is started. The stream control unit is any video reproduction device according to any one of claims 1 to 7, which acquires the second information from the stream data. The data other than the stream data is clip information data, and is
The stream control unit is any video reproduction device according to any one of claims 1 to 7, which acquires the second information from the clip information data.

Images