JP7034926B2 - Information processing equipment, information processing methods, and programs - Google Patents

Description

The present technology relates to information processing devices, information processing methods, and programs, and in particular, information processing devices and information that enable content formats to be converted from a first format having a different time axis to a second format. Regarding processing methods and programs.

Content transmission in next-generation broadcasting standards is performed by MMT (MPEG Media Transport) instead of the conventional MPEG-2 TS. MMT makes it possible to transmit content using not only broadcast wave transmission lines but also communication transmission lines such as the Internet.

Japanese Unexamined Patent Publication No. 2016-103745

When receiving and recording MMT content, it is possible to record MMT content as it is, but recording with MPEG-2 TS is also being considered. By recording with MPEG-2 TS, it becomes possible to utilize the assets of MPEG-2 TS that are already widespread.

The technology to convert MMT content to MPEG-2 TS content has not been established.

The present technology has been made in view of such a situation, and makes it possible to convert the format of the content from the first format having a different time axis to the second format.

The information processing device on one aspect of this technology is a clock converter that converts NTP-format time information contained in MMT data, which is MMT -multiplexed data including video and audio streams, into MPEG-2 TS PCR . Of the time stamp conversion unit that converts PTS and DTS contained in the MMT data into PTS and DTS of MPEG-2 TS, respectively, and the video data constituting the video stream and the audio data constituting the audio stream. A PES packet is generated by combining the video data and the audio data belonging to the same MPU in the MPU sequence number, and the related PTS and DTS among the converted PTS and DTS are based on the MPU sequence number. It is provided with a multiplexing unit that generates TS data by adding the data.

The time information in NTP format can be expressed as a 32-bit counter value in which the time in seconds of 1 second or more is represented by 32 bits and the accuracy of 1 second or less is 2 ⁿ Hz (n is an integer of 24 or more). .. Further, the PCR of MPEG-2 TS can be represented by 33 bits with a precision of 90 kHz for a time of 1 second or longer, and a 42-bit counter value represented with 9 bits with a precision of 27 MHz for a time of 1 second or less. can.

In the multiplexing unit, a TS packet containing PCR can be added to a TS packet for storing the video data and a TS packet for storing the audio data to generate the TS data.

The PTS and DTS included in the MMT data can be represented by 32 bits for a time in seconds of 1 second or longer, and can be a 32 + n-bit counter value with an accuracy of 1 second or less as 2 ⁿ Hz. Further, the PTS and DTS of the MPEG-2 TS can be used as counter values expressed in 33 bits with an accuracy of 90 kHz.

The time stamp conversion unit performs different calculations depending on whether the time indicated by the PTS and DTS contained in the MMT data is larger or smaller than 26.5h, and the MPEG-corresponding to the PTS and DTS included in the MMT data. 2 TS PTS and DTS can be calculated .

A recording control unit for recording the TS data on a recording medium can be further provided.

The recording medium can be a Blu-ray (registered trademark) Disc.

In one aspect of the present technology, the NTP format time information contained in the MMT data obtained by multiplexing the data including the video stream and the audio stream by MMT is converted into the PCR of MPEG-2 TS and included in the MMT data. PTS and DTS are converted to MPEG-2 TS PTS and DTS, respectively . Further, a PES packet is generated by combining the video data and the audio data belonging to the same MPU among the video data constituting the video stream and the audio data constituting the audio stream based on the MPU sequence number. TS data is generated by adding the related PTS and DTS among the converted PTS and DTS based on the MPU sequence number .

According to the present technology, the format of the content can be converted from the first format having a different time axis to the second format.

The effects described herein are not necessarily limited, and may be any of the effects described in the present disclosure.

It is a figure which shows the example of the format conversion by the information processing apparatus which concerns on one Embodiment of this technique. It is a figure which shows the example of the conversion of a counter value. It is a figure which shows the protocol stack of the broadcasting system which uses MMT. It is a figure which shows the structure of the NTP format. It is a figure which shows the arrangement of the TLV packet which stores the IP packet containing the time information of NTP format. It is a figure which shows the example which constructs an MMTP payload from a video / audio signal. It is a figure which shows the structure of the MMTP packet. It is a figure which shows the meaning of the value of payload_type. It is a figure which shows the structure of MMTP_payload (). It is a figure which shows the structure of MMTP_payload (). It is a figure which shows the structure of MMT-SI. It is a figure which shows the type and function of a message of MMT-SI. It is a figure which shows the structure of a PA message. It is a figure which shows the meaning of the value of message_id. It is a figure which shows the structure of MPT. It is a figure which shows the structure of the MPU time stamp descriptor. It is a figure which shows the structure of the MPU extended time stamp descriptor. It is a figure which shows the calculation method of PTS and DTS in MMT. It is a figure which shows the structure of a TS packet. It is a figure which shows the structure of the adaptation field. It is a figure which shows the structure of a PES packet. It is a block diagram which shows the configuration example of an information processing apparatus. It is a block diagram which shows the configuration example of the MMT signal processing unit and the recording control unit of FIG. 22. It is a figure which shows the relationship between the counter value and time in MPEG-2 TS. It is a figure which shows the relationship between the counter value and time in MMT. It is a figure which superimposes the characteristic of a 33-bit counter of MPEG-2 TS and the characteristic of a 32 + nbit counter of MMT. It is a figure which shows the structure of PCR. It is a flowchart explaining the conversion process of an information processing apparatus. It is a block diagram which shows the configuration example of a computer.

Hereinafter, a mode for implementing the present technology will be described. The explanation will be given in the following order.
1. 1. Example of format conversion 2. About MMT 3. About MPEG-2 TS 4. Configuration example of information processing device 5. Conversion of counter value 6. Operation of information processing device 7. Modification example

<1. Format conversion example>
FIG. 1 is a diagram showing an example of format conversion by an information processing apparatus according to an embodiment of the present technology.

The information processing device 1 is, for example, a device such as a recording device having a built-in recording medium such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), a television receiver, a personal computer, or a set-top box. The information processing device 1 receives content such as a television program including video and audio, which is broadcast (distributed) via a broadcast wave transmission line or a communication transmission line such as the Internet. Broadcast wave transmission lines include terrestrial broadcasting and satellite broadcasting transmission lines.

Content is transmitted via the broadcast wave transmission line or the communication transmission line by MMT. The information processing apparatus 1 receives the multiplexed data obtained by multiplexing the video stream and the audio stream with MMT.

The information processing device 1 is a device corresponding to the reception of MMT contents compliant with ARIB STD-B60 (“media transport method by MMT in digital broadcasting”). The information processing device 1 is also a device that supports content transmission in accordance with NEXTVF TR-0004 (“Advanced Broadband Satellite Digital Broadcasting Operation Regulations”) specified in the Next Generation Broadcasting Promotion Forum.

The information processing device 1 has a function of reproducing the received MMT content and outputting the video and audio of the content from a television receiver (not shown) connected to the information processing device 1.

Further, the information processing apparatus 1 has a function of converting the received MMT content into the MPEG-2 TS content. MPEG-2 TS is specified by ITU-H.222.0.

At the time of conversion of the format from MMT to MPEG-2 TS, the information processing apparatus 1 acquires the video data and audio data stored in the MMTP (MMT Protocol) packet and stores them again in the TS packet. The information processing apparatus 1 generates a TS (transport stream) by multiplexing a video TS packet and an audio TS packet together with a TS packet storing control information such as SI (Service Information).

At the time of format conversion, the information processing apparatus 1 converts the time information included in the MMT contents into the time information of the MPEG-2 TS. MMT and MPEG-2 TS are formats with different time axes. The difference in the time axis means that at least one of the number of bits and the accuracy (frequency) of the counter value that defines the time is different.

MMT content and MPEG-2 TS content include counter values that represent the reference time that is the basis of operation, and counter values that represent PTS (Presentation Time Stamp) and DTS (Decode Time Stamp) as time information. ..

FIG. 2 is a diagram showing an example of conversion of a counter value.

As shown on the left side of FIG. 2, in MMT, the reference time is included in the time information in NTP format.

Details will be described later, but in the NTP format, the reference time is represented by a counter value of 32 + nbit / 2 ⁿ Hz. Here, 32 + nbit / 2 ⁿ Hz indicates that the time of 1 second or more is represented by using 32 bits in seconds, and the time of 1 second or less is represented by using nbit. The accuracy of 1 second or less is 2 ^nm Hz.

NEXTVF_TR-0004 stipulates that n is 24 or more for NTP format operation.

In MMT, PTS and DTS are obtained from the time represented by the counter value of 32 + nbit / 2 ⁿ Hz.

On the other hand, as shown on the right side of FIG. 2, in MPEG-2 TS, PCR, which is a reference time as a reference time for operation, is represented by a counter value of 42 bits / 27 MHz.

Further, in MPEG-2 TS, PTS and DTS are represented by counter values of 33 bits / 90 kHz, respectively.

The information processing apparatus 1 converts the reference time of the MMT represented by the counter value of 32 + ^nbit / 2 nm into the PCR of the MPEG-2 TS represented by the counter value of 42 bit / 27 MHz.

Further, the information processing apparatus 1 converts PTS and DTS in MMT into PTS and DTS of MPEG-2 TS represented by counter values of 33 bits / 90 kHz, respectively.

The information processing apparatus 1 generates a TS including the time information of the MPEG-2 TS converted in this way.

Returning to the description of FIG. 1, the information processing apparatus 1 records the content of the MPEG-2 TS generated by converting the content of the MMT on the recording medium 11 mounted on the information processing apparatus 1. The recording medium 11 is, for example, a Blu-ray (registered trademark) Disc (hereinafter, appropriately referred to as BD). BD is a recording medium that can record MPEG-2 TS files.

The information processing device 1 can record the content transmitted by the MMT on the BD. A playback device that supports BD playback can play back the content written on the BD by the information processing device 1.

Further, the information processing apparatus 1 transmits the content of the MPEG-2 TS generated by converting the content of the MMT via a network 12 such as the Internet. A playback device that supports playback of the content of the MPEG-2 TS can reproduce the content generated by the information processing device 1.

By enabling the MMT content to be converted into the MPEG-2 TS content in this way, it is possible for various devices capable of processing the MPEG-2 TS to process the content provided by the MMT. Become. In other words, it will be possible to utilize the assets of MPEG-2 TS.

<2. About MMT ＞
<2-1. MMT packet structure>
Describe MMT. Here, a part of the provisions of ARIB STD-B60 that is related to the processing of the information processing apparatus 1 will be described. Unless otherwise specified, it is basically based on the provisions of ARIB STD-B60.

FIG. 3 is a diagram showing a protocol stack of a broadcasting system using MMT.

The code of the video signal and audio signal of the program is divided into transmission units of MFU (Media Fragment Unit) / MPU (Media Processing Unit). An MPU, which is a unit containing data of one or more access units, is configured to include a plurality of MFUs.

The MFU is placed in the MMTP payload, and the MMTP header is added to form the MMTP packet. MMTP packets are transmitted using IP packets. One MMTP packet is transmitted by one IP packet.

MMT-SI, which is control information indicating the configuration of broadcast programs, is also placed in the MMTP payload as an MMT control message format, converted into MMTP packets, and transmitted as IP packets.

The above-mentioned NTP format time information is also transmitted using an IP packet.

FIG. 4 is a diagram showing an NTP format configuration.

NTP format IP packets specified in IETF RFC 5905 “Network Time Protocol Version 4: Protocol and Algorithms Specification” are used to provide Coordinated Universal Time (UTC).

NTP uses the cumulative number of seconds from 0:00:00 on January 1, 1900. The NTP format includes reference_timestamp.

reference_timestamp indicates the time when the system time was last corrected in NTP length format. The NTP length format is composed of a 32-bit field indicating a second unit and a 32-bit field indicating 1 second or less.

NEXTVF_TR-0004 stipulates that the time of 1 second or less is indicated by using nbit (n: 24 or more). n is 32 or less.

The time indicated by NTP is represented as a counter value of a 32 + nbit counter, which is the sum of 32 bits representing a time of 1 second or longer in seconds and nbit representing a time of 1 second or less. The accuracy of 1 second or less is 2 ^nm .

N, which has an accuracy of 1 second or less, is indicated by the precision in FIG. n indicates log2 of the accuracy (in seconds) of the system clock.

In ARIB STD-B60, the transmission system and receiver in the broadcasting system using MMT must meet the following requirements.
-The sending system must maintain a clock based on the NTP time stamp.
-In the transmission system, the above clock and the clock that encodes the video and audio must be synchronized.
-The receiver shall play and hold the clock based on the received NTP format time information.
-In the receiver, the system clock for holding the above clock and the clock for decoding video / audio, etc. must be synchronized.

FIG. 5 is a diagram showing the arrangement of TLV (Type Length Value) packets for storing IP packets including NTP format time information in STD-B44.

STD-B323_1 stipulates that IP packets, including IP packets containing NTP-format time information, should be transmitted as TLV packets. The TLV packet that stores the IP packet containing the time information in NTP format is placed at the beginning of the first slot among the slots assigned for each TLV stream ID.

By arranging the TLV packet that stores the IP packet including the NTP format time information in this way, it is possible to suppress the delay fluctuation of the IP packet including the NTP format time information.

The information processing device 1 extracts an IP packet from the TLV packet constituting the TLV stream, and acquires NTP-format time information stored in the IP packet or an MMTP packet of video or audio.

FIG. 6 is a diagram showing an example of constructing an MMTP payload from video / audio signals.

In the processing of video signals and audio signals, the MPU is the unit of processing. As shown by the broken line in the middle of FIG. 6 as sample data, the MPU includes one or more access units, and is a unit capable of performing video and audio decoding processing by the MPU alone. The size of the MPU is arbitrary and can include any number of access units. For video signals encoded using inter-frame prediction, the MPU must be in GOP (Group of Picture) units.

As shown in the middle of FIG. 6, the MPU is composed of MPU metadata including information on the configuration of the entire MPU, movie fragment metadata including information on encoded media data, and sample data which is encoded media data. To. Since one MPU can be composed of one or more movie fragments, there may be a plurality of movie fragment metadata and sample data.

The same sequence number is added to the MPU for each MPU belonging to the same asset. By using the asset ID that identifies the asset and the sequence number of the MPU, it is possible to distinguish each MPU.

MFU is a smaller unit than MPU. MFU can be extracted from the sample data that composes MPU. As a method of configuring the MFU, there is a method of configuring the MFU in units of NAL units or access units as shown by the downward arrow in FIG. By configuring the MFU with the media in mind, it is possible to suppress the propagation of errors when transmission quality degradation such as packet loss occurs.

For media such as video components and audio components for which the presentation time is specified, the presentation time (PTS in MMT described above) is indicated in MPU units. The presentation time of the MPU is specified using a common time axis based on UTC. Therefore, the device on the receiving side can synchronously present the necessary media components regardless of the difference in the transmission path between broadcasting and communication and the difference in the transmission source.

As described above, the elements constituting the coded signal in MMT include MPU, MFU, MMTP packet, and MMTP payload.

Note that FIG. 6 shows an example in which one MFU is stored in one MMTP payload. Multiple MFUs may be stored in one MMTP payload, and conversely, one MFU may be stored in multiple MMTP payloads.

FIG. 7 is a diagram showing the configuration of the MMTP packet.

An MMTP packet consists of an MMTP header and an MMTP payload. Of the descriptions shown in FIG. 7, the description from the beginning to before MMTP_payload () constitutes the MMTP header.

The payload_type included in the MMTP header indicates the data type of the MMTP payload.

FIG. 8 is a diagram showing the meaning of the value of payload_type.

As shown in FIG. 8, that the value of payload_type is 0x00 indicates that the MMTP payload contains MPUs such as video and audio.

Also, a payload_type value of 0x02 indicates that the MMTP payload contains a control message.

Returning to the description of FIG. 7, the packet_id (packet identifier) included in the MMTP header is information for identifying the data type of the MMTP payload.

NEXTVF_TR-0004 stipulates that when the MMTP payload contains an MPU (when the payload_type value is 0x00), the packet_id of the asset shown in the MMT package table described later is described.

9 and 10 are diagrams showing the configuration of MMTP_payload ().

FIG. 9 shows a description when the value of payload_type is 0x00, that is, the MPU is included in the MMTP payload. On the other hand, FIG. 10 shows a description when the value of payload_type is 0x02, that is, the control message is included in the MMTP payload.

In the description shown in FIG. 9, fragment_type indicates the fragment type of the information stored in the MMTP payload.

For example, a value of fragment_type of 0 indicates that the MMTP payload contains MPU metadata (FIG. 6). The MPU_metadata_byte in FIG. 9 described when the value of fragment_type is 0 corresponds to the MPU metadata.

A fragment_type value of 1 indicates that the MMTP payload contains movie fragment metadata. The movie_fragment_metadata_byte in FIG. 9 described when the value of fragment_type is 1, corresponds to the movie fragment metadata.

A fragment_type value of 2 indicates that the MMTP payload contains timed media data sample data (NAL units, access units), subsample data, or timeless media data items that make up the MFU. .. For example, the MFU_data_byte in FIG. 9 described when the value of fragment_type is 2 and the value of timed_flag is 1, corresponds to the sample data of the media data with time constituting the MFU.

Further, in the description shown in FIG. 9, the MPU_sequence_number (MPU sequence number) indicates the sequence number of the MPU to which the MMTP payload stores the MPU metadata, the movie fragment metadata, and the MFU.

In the description shown in FIG. 10, the message_length (message data length) described when the value of payload_type is 0x02 indicates the size of one control message placed immediately after this field in bytes.

message_byte (message data) is a data byte of a control message. Hereinafter, the control message is also simply referred to as a message as appropriate.

<2-2. MMT control message>
Here, the control message stored in the MMTP payload will be described.

FIG. 11 is a diagram showing the structure of MMT-SI, which is control information.

As shown in FIG. 11, MMT-SI, which shows information related to the configuration of MMT packages and broadcasting services, includes messages that store tables and descriptors, tables with elements and attributes that indicate specific information, and details. It is composed of three layers of descriptors showing various information.

As mentioned above, the message is stored in the MMTP payload and transmitted using MMTP packets. The value of payload_type when the message constitutes the MMTP payload is 0x02. Multiple messages are not stored in one MMTP payload, one message is stored in one MMTP payload.

FIG. 12 is a diagram showing the types and functions of MMT-SI messages.

As shown in FIG. 12, the messages mainly include Package Access (PA) messages, M2 section messages, CA messages, M2 short section messages, and data transmission messages. For example, the PA message becomes the entry point of MMT-SI and is used to transmit various tables of MMT-SI.

The type of such a message is identified by the message_id (message identifier) described in the MMTP payload as message_byte.

FIG. 13 is a diagram showing the structure of the PA message.

message_id is set as 0x0000 as shown in FIG.

number_of_tables indicates the number of tables stored in this PA message.

table_id (table identification) indicates the identifier of the table stored in this PA message.

The table is the table in which this PA message is stored. One of the tables, MPT (MMT package table), is described in table () of the PA message.

The description of such a PA message is included in the MMTP payload (FIG. 10) as message_byte.

FIG. 15 is a diagram showing the configuration of the MPT.

The MPT provides information that makes up the package, such as a list of assets and the location of the assets on the network. The MPT contains a descriptor.

MPT_descriptors_length (MPT descriptor length) in the description shown in FIG. 15 indicates the length of the MPT descriptor area in bytes.

The MPT_descriptors_byte (MPT descriptor area) is an area for storing MPT descriptors. The descriptors stored in MPT_descriptors_byte include, for example, an MPU time stamp descriptor (MPU_Timestamp_Descriptor ()) and an MPU extended time stamp descriptor (MPU_Extended_Timestamp_Descriptor ()).

The MPU time stamp descriptor is a descriptor indicating the presentation time of the first access unit in the presentation order in the MPU. The MPU extended time stamp descriptor is a descriptor that provides the decryption time of the access unit in the MPU.

FIG. 16 is a diagram showing the structure of the MPU time stamp descriptor.

The descriptor_tag of the MPU timestamp descriptor is set as 0x001.

mpu_sequence_number (MPU sequence number) indicates the sequence number of the MPU that describes the time stamp in the MPU time stamp descriptor. As described above, the same sequence number is added to each MPU for each MPU belonging to the same asset.

mpu_presentation_time (MPU presentation time) indicates the presentation time of MPU in NTP format (indicated by a counter value of 32 + nbit / 2 ⁿ Hz).

FIG. 17 is a diagram showing the structure of the MPU extended time stamp descriptor.

The descriptor_tag of the MPU extended timestamp descriptor is set as 0x8026.

mpu_sequence_number (MPU sequence number) indicates the sequence number of the MPU including the access unit that specifies the decoding time, etc. in the MPU extended time stamp descriptor.

mpu_decoding_time_offset (MPU decoding time offset) is the absolute value of the difference between the decoding time of the first access unit in the decoding order and the presentation time of the first access unit in the presentation order in the MPU specified by the MPU sequence number. Shown in time units shown on a scale.

num_of_au (number of access units) indicates the number of access units that specify the decryption time, etc. in the MPU extended time stamp descriptor.

dts_pts_offset (decoding time / presentation time offset) indicates the time from the decoding time of the access unit to the presentation time in the time unit indicated by the time scale.

pts_offset (PTS offset) shows the difference in presentation time between the previous access unit and the current access unit in the presentation order in the same MPU, in the time unit shown on the time scale.

The MPU timestamp descriptor identifies the presentation time of the first access unit in the presentation order of the MPU. In addition, the decryption time of each access unit is specified from the MPU extended time stamp descriptor.

FIG. 18 is a diagram showing a method of calculating PTS and DTS in MMT.

Each rectangle shown in FIG. 18 shows a picture (access unit) constituting GOPn included in a certain MPU. The upper part of FIG. 18 shows the arrangement of each access unit in the decoding order, and the lower part shows the arrangement of each access unit in the presentation order. The I / P / B alphabet indicates the picture type, and the numbers indicate the number of each picture in the order of presentation.

The mpu_presentation_time in the MPU timestamp descriptor indicates the presentation time of the first access unit in the presentation order in the MPU. The presentation time indicated by the MPU time stamp descriptor is the time indicated by the upward triangle in the lower left of FIG.

On the other hand, the MPU extended time stamp descriptor is presented by mpu_decoding_time_offset with the decryption time of the first access unit in the decryption order and the first access unit in the presentation order in the MPU, as shown by the dashed arrow on the left side of FIG. The difference value from the time is shown.

In addition, the MPU extended time stamp descriptor indicates the difference value between the decryption time and the presentation time of each access unit by dts_pts_offest as a value with the time scale as the unit time. Furthermore, the MPU extended time stamp descriptor indicates the difference between the presentation time of the access unit immediately before in the presentation order and the presentation time of the current access unit in the same MPU by pts_offset as a value with the time scale as the unit time. ..

Using these times and values, the decoding time DTS (n) and the presentation time PTS (n) of the nth access unit in the MPU are calculated by the following equations (1) and (2), respectively.

In this way, the PTS and DTS of each access unit are specified from the MPU time stamp descriptor and the MPU extended time stamp descriptor. Since it is obtained using mpu_presentation_time, which is a counter value of 32 + nbit / 2 ⁿ Hz, it can be said that the DTS of the MPU is also the time indicated by the counter value of 32 + nbit / 2 ⁿ Hz.

In the information processing apparatus 1, the MPU time stamp descriptor and the MPU extended time stamp descriptor are acquired from the control message of the MMTP packet, respectively, and PTS and DTS are obtained based on the MPU time stamp descriptor and the MPU extended time stamp descriptor. Be done. The calculation of PTS and DTS is performed before the start of decryption of each access unit constituting the MPU transmitted using other MMTP packets.

<3. About MPEG-2 TS>
Next, the MPEG-2 TS will be described.

FIG. 19 is a diagram showing a configuration of TS packets constituting the TS.

One TS packet is composed of 184 bytes of payload, adaptation field, or both added to a 4 bytes header. The header contains, for example, a PID that is identification information of the type of data contained in the payload. The configuration of such TS packets is specified in, for example, ISO / IEC 138180-1.

The adaptation field of the TS packet includes PCR, as shown shaded in FIG. FIG. 20 is a diagram showing the configuration of an adaptation field.

At the time of conversion from MMT to MPEG-2 TS, in the information processing apparatus 1, the PCR obtained by converting the reference time of MMT represented by the time information in NTP format is set in the adaptation field of the TS packet. ..

FIG. 21 is a diagram showing a configuration of a PES (Packetized Elementary Stream) packet.

In MPEG-2 TS, video and audio streams are divided into predetermined units and stored in PES packets. One PES packet is divided into a plurality of TS packets. A header is added to the PES packet, and PTS and DTS are described as shown in FIG.

At the time of conversion from MMT to MPEG-2 TS, in the information processing apparatus 1, the PTS and DTS obtained by converting the PTS and DTS of the MMT calculated by analyzing the MMT-SI are set in the PES packet. To.

<4. Information processing device configuration example>
FIG. 22 is a block diagram showing a configuration example of the information processing apparatus 1.

The information processing device 1 is composed of a receiving unit 31, an MMT signal processing unit 32, a recording control unit 33, a drive 34, an output control unit 35, and a communication unit 36.

The receiving unit 31 receives a broadcast wave signal from an antenna (not shown) and performs various processes such as demodulation processing and error correction processing. The receiving unit 31 outputs the TVL stream obtained by performing various processes to the MMT signal processing unit 32 as MMT contents.

Further, when the content is transmitted via the transmission path of the communication, the receiving unit 31 controls the communication unit 36 to receive the MMT content, and MMT the TVL stream obtained by performing a predetermined process. It is output to the signal processing unit 32.

The MMT signal processing unit 32 reproduces the contents of the MMT, outputs the video and audio of the contents from the television receiver, and converts the contents of the MMT into the contents of the MPEG-2 TS.

For example, when the MMT signal processing unit 32 reproduces the contents of the MMT, the MMT signal processing unit 32 extracts video data and audio data from the TLV stream supplied from the receiving unit 31 and decodes them. The MMT signal processing unit 32 outputs the video and audio data of the content obtained by decoding to the output control unit 35.

Further, when the MMT signal processing unit 32 converts the MMT content into the MPEG-2 TS content, the MMT signal processing unit 32 stores the data stored in the MMTP packet extracted from the TLV stream supplied from the receiving unit 31 in the TS packet. Generate TS by fixing it. When the MMT signal processing unit 32 records the content of the MPEG-2 TS on the recording medium 11, the generated TS is output to the recording control unit 33, and when the content is transmitted to another device via the network 12, the generated TS is generated. Is output to the communication unit 36.

The recording control unit 33 controls the drive 34 and causes the TS generated by the MMT signal processing unit 32 to be recorded on the recording medium 11 mounted on the drive 34.

The output control unit 35 supplies the video and audio data supplied from the MMT signal processing unit 32 to, for example, a television receiver connected to the information processing device 1, and outputs the data from the display and the speaker, respectively.

The communication unit 36 communicates with various devices via the network 12. For example, the communication unit 36 receives the MMT content distributed from the server via the network 12 and outputs it to the reception unit 31. Further, the communication unit 36 transmits the TS generated by the MMT signal processing unit 32 to another device via the network 12.

FIG. 23 is a block diagram showing a configuration example of the MMT signal processing unit 32 and the recording control unit 33 of FIG. 22.

The demultiplexer 51 extracts an IP packet from the TLV packet constituting the TLV stream supplied from the receiving unit 31, and performs DEMUX processing of the MMT data transmitted using the IP packet.

For example, the demultiplexer 51 extracts an IP packet including NTP-format time information transmitted using the TLV packet and outputs the IP packet to the comparison unit 52. As described with reference to FIG. 5, the IP packet containing the time information in the NTP format is placed at the beginning of the first slot among the slots constituting the TLV stream.

Further, the demultiplexer 51 outputs the MMTP packet storing the MMT-SI among the MMTP packets transmitted using the IP packet to the MMT-SI analysis unit 55. The demultiplexer 51 outputs an MMTP packet storing video data and audio data constituting the MPU to the MPU / MMT packet analysis unit 58. The type of data stored in the MMTP payload is identified by the payload_type of the MMTP packet (Fig. 8).

The comparison unit 52 outputs the reference time (32 + nbit counter value) included in the NTP format time information stored in the IP packet supplied from the demultiplexer 51 to the NTP clock counter 53, and sets it as the initial value of the counter value. ..

Further, the comparison unit 52 outputs a signal indicating an error between the time information supplied from the NTP clock counter 53 and the time information in the NTP format supplied from the demultiplexer 51 to the NTP clock counter 53.

The NTP clock counter 53 has a VCO (Voltage Controlled Oscillator). The NTP clock counter53 counts the 2 ⁿ Hz clock signal generated by the VCO and outputs a 2 ⁿ Hz system clock signal representing a 32 + nbit counter value. The system clock signal output from the NTP clock counter 53 is supplied to the comparison unit 52, the clock conversion unit 54, and the video / audio synchronization control unit 56.

The clock conversion unit 54 converts the 32 + nbit counter value represented by the 2 ⁿ Hz system clock signal supplied from the NTP clock counter 53 into a 42-bit counter value (27 MHz) indicating PCR of MPEG-2 TS. The details of the conversion by the clock conversion unit 54 will be described later. The clock conversion unit 54 outputs the 42-bit counter value obtained by conversion as PCR to the TS conversion processing unit 62.

The MMT-SI analysis unit 55 acquires and analyzes the MMT-SI from the MMTP packet stored in the IP packet supplied from the demultiplexer 51. For example, the MMT-SI analysis unit 55 acquires the MPT transmitted by the PA message stored in the MMTP packet, and has the MPU time stamp descriptor (FIG. 16) and the MPU extended time stamp descriptor (FIG. 16) described in the MPT. 17) is acquired.

Further, the MMT-SI analysis unit 55 calculates (specifies) the PTS of each MPU based on the MPU time stamp descriptor. The MMT-SI analysis unit 55 calculates the DTS of the access unit constituting each MPU by using the PTS of each MPU and the description of the MPU extended time stamp descriptor as described with reference to FIG. ..

The MMT-SI analysis unit 55 outputs the calculated PTS and DTS together with the mpu_sequence_number indicating the sequence number of the MPU to the video / audio synchronization control unit 56 and the PTS / DTS conversion unit 57. As described above, mpu_sequence_number included in the MPU time stamp descriptor indicates the sequence number of the MPU that specifies the presentation time (FIG. 16). Further, mpu_sequence_number included in the MPU extended time stamp descriptor indicates the sequence number of the MPU including the access unit for specifying the decoding time and the like (FIG. 17).

The video / audio synchronization control unit 56 operates according to the system clock signal from the NTP clock counter 53, and controls the timing of decoding and output by the video / audio decoder 60 based on the PTS and DTS calculated by the MMT-SI analysis unit 55. .. The video / audio decoder 60 is controlled by the video / audio synchronization control unit 56 based on metadata such as MPU metadata and movie fragment metadata supplied from the MPU / MMT packet analysis unit 58.

The PTS / DTS conversion unit 57 converts the 32 + nbit counter values representing PTS and DTS calculated by the MMT-SI analysis unit 55 into 33-bit PTS and DTS counter values (90 kHz) of MPEG-2 TS, respectively. do. Details of the conversion by the PTS / DTS conversion unit 57 will be described later. The PTS / DTS conversion unit 57 outputs the converted MPEG-2 TS PTS and DTS to the TS conversion processing unit 62.

From the PTS / DTS conversion unit 57, mpu_sequence_number representing the sequence number of the MPU related to the PTS and DTS of the MPEG-2 TS obtained by conversion is output. Further, in the PTS / DTS conversion unit 57, SI that needs to be included in the MPEG-2 TS is appropriately generated based on the MMT-SI and output to the TS conversion processing unit 62.

The MPU / MMT packet analysis unit 58 acquires the data of each MPU by analyzing the MMTP packet stored in the IP packet supplied from the demultiplexer 51. The MPU metadata, the movie fragment metadata, and the MFU, which are the data of each MPU, are specified by the MPU_sequence_number (FIG. 9), which is information indicating the sequence number of the MPU to which they belong.

The MPU / MMT packet analysis unit 58 outputs metadata such as MPU metadata and movie fragment metadata of each MPU acquired from the MMTP payload to the video / audio synchronization control unit 56 when the MMT content is reproduced. Further, the MPU / MMT packet analysis unit 58 outputs the video data and audio data of the MFU constituting each MPU acquired from the MMTP payload to the decoder buffer 59.

On the other hand, the MPU / MMT packet analysis unit 58 acquires the video data (VIDEO ES) and audio data (AUDIO ES) of the MFU constituting each MPU acquired from the MMTP payload when converting the format from MMT to MPEG-2 TS. Is output to the buffer 61 together with the MPU_sequence_number.

The video / audio decoder 60 decodes and outputs the video data and audio data recorded in the decoder buffer 59 according to the control by the video / audio synchronization control unit 56. The data output from the video / audio decoder 60 is supplied to the output control unit 35, and the video and audio of the content are output.

The TS conversion processing unit 62 constitutes PES by collecting the video data and audio data recorded in the buffer 61 that have the same value as the MPU_sequence_number. Further, the TS conversion processing unit 62 adds the PTS and DTS related to each PES supplied from the PTS / DTS conversion unit 57 to each PES, and generates a PES packet. Which PES each PTS / DTS is added to is specified based on the mpu_sequence_number supplied from the PTS / DTS conversion unit 57.

The TS conversion processing unit 62 converts the generated PES packet into a TS packet. The TS conversion processing unit 62 generates TS based on the TS packet in which the data of the PES packet is stored in the payload and the TS packet including the PCR generated by the clock conversion unit 54 in the adaptation field.

When the Partial TS is generated, the TS conversion processing unit 62 inputs the SI of the MPEG-2 TS such as PAT, PMT, DAT, SIT, etc. specified by ARIB STD-B14 to the information supplied from the PTS / DTS conversion unit 57. Embed in TS based on.

PAT (Program Association Table) is information that specifies a packet identifier of a TS packet that transmits a PMT related to Partial TS. PMT (Program Map Table) is information that specifies a packet identifier of a TS packet that transmits each coded signal constituting a broadcast program. PAT and PMT are retransmitted at least once every 100 ms.

DIT (Discontinuity Information Table) is information indicating a change point where the program arrangement information of the program transmitted by Partial TS may be discontinuous. SIT (Selection Information Table) is information that indicates information about a program transmitted by Partial TS.

The TS conversion processing unit 62 outputs the TS obtained by performing the TS conversion processing as described above to the recording control unit 33.

The recording control unit 33 is composed of a T-STD 81, a PLL 82, an Arrival time clock counter 83, a Source packetizer 84, and a Write Buffer 85. The configuration of the recording control unit 33 shown in FIG. 23 is a conceptual model for defining the recording process. The recording control unit 33 has a function of adding an ATS (Arrival_time_stamp) to the TS packets constituting the TS and generating a sequence of source packets according to this recorder model.

The i-th byte of the TS supplied from the MMT signal processing unit 32 is input to the T-STD (Transport stream system target decoder) 81 and the Source packetizer 84 at time t (i).

The T-STD 81 is a virtual decoder in the recorder model. The decoder corresponding to T-STD81 does not exist in the recording control unit 33.

The PLL 82 generates a 27 MHz clock locked to the value of the TS PCR.

The Arrival time clock counter 83 is a binary counter that counts the pulse of the frequency of 27 MHz output by the PLL 82. Arrival_time_clock (i) is the counter value of Arrival time clock counter 83 at time t (i).

The Source packetizer 84 adds TP_extra_header to all TS packets and generates a source packet. Further, the Source packetizer 84 adds ATS to the TS packet.

Specifically, the Source packetizer 84 sets the value of ATS in the TP_extra_header of the source packet based on the Arrival_time_clock (i) supplied from the Arrival time clock counter 83. ATS represents the time when the first byte of the TS packet arrives at both the T-STD 81 and the Source packetizer 84.

The Write Buffer 85 records the source packet generated by the Source packetizer 84. Each source packet output from the Write Buffer 85 is supplied to the drive 34 and recorded on the recording medium 11.

<5. About conversion of counter value>
<5-1. PTS / DTS conversion of PTS / DTS conversion unit 57>
Here, the conversion of the counter value indicating PTS / DTS will be described.

FIG. 24 is a diagram showing the relationship between the counter value and time in the MPEG-2 TS.

The horizontal axis of the graph shown in FIG. 24 indicates the time, and the vertical axis indicates the counter value. The same applies to FIGS. 25 and 26.

Each PTS / DTS of MPEG-2 TS is expressed with reference to PCR. PCR is represented by 42 bits (27 MHz) and PTS / DTS is represented by 33 bits (90 kHz). Since PTS / DTS is represented by 33 bits, 33 bits is sufficient as the amount of information for representing time information.

When the ³³ -bit counter is operated with an accuracy of 90 kHz, the wrap around of the counter value occurs at the time of the point P1 at which the counter value becomes 233. The time at the point P1 is obtained as approximately 26.5 h as shown in the following equation (3).

The second wraparound occurs at the time of point P2 and the third wraparound occurs at the time of point P3. The relationship between the count value of the 33-bit counter with an accuracy of 90 kHz and the real time is shown in FIG. 24, in which wraparound occurs at the time of points P1, P2, and P3.

The straight line connecting the origin and the point P11 shows the relationship between the real time and the counter value when the 42-bit counter, which has the same number of bits as the PCR, is operated with an accuracy of 90 kHz.

FIG. 25 is a diagram showing the relationship between the counter value and time in MMT.

As described above, each PTS / DTS of MMT is represented by 32 + nbit (accuracy of 1 second or less is 2 ⁿ Hz) in NTP format.

When the 32 + nbit counter is operated with an accuracy of 2 ⁿ Hz, the wraparound of the counter value occurs at the time of the point P21 where the counter value becomes 232 ^{+ n} . The time at point P21 is expressed as 1193046h as shown in the following equation (4).

The relationship between the counter value of the 32 + nbit counter with an accuracy of 2 ^nm Hz and the real time is shown by the alternate long and short dash line where wraparound occurs at the time of point P21.

FIG. 26 is a diagram showing the characteristics of the 33-bit counter of the MPEG-2 TS and the characteristics of the 32 + nbit counter of the MMT in an overlapping manner.

When the PTS / DTS of the MMT indicates the time T which is a predetermined time, the PTS / DTS of the corresponding MPEG-2 TS obtains the counter value of the 33-bit counter representing the time T based on the counter value of the 32 + nbit counter. Identified by that.

Specifically, the MPEG-2 TS counter value X'is obtained by the following procedure based on the MMT counter value X.

・ Procedure 1
Obtain the MMT counter value A at the wraparound time in MPEG-2 TS.

The MMT counter value A is a value corresponding to a time of 233 / 90k ( ^≈26.5h ) in real time. Assuming that n = 24, the counter value A is obtained as in the following equation (5).

・ Procedure 2
The MPEG-2 TS counter value X'indicating the same time as the MMT counter value X is obtained by the following equation (6) when the time indicated by the MMT counter value X is smaller than 26.5 h. Be done.

On the other hand, when the time indicated by the MMT counter value X is larger than 26.5 h, the MPEG-2 TS counter value X'is lower, using the value of the remainder mod (X / A) as the MMT counter value X. It is obtained as in equation (7).

In this way, the PTS / DTS conversion unit 57 in FIG. 23 shows the ratio of the real time to the counter value in the characteristics of the 32 + nbit counter of MMT and the ratio of the real time to the counter value in the characteristics of the 33 bit counter of MPEG-2 TS. Is used to convert the counter value.

For example, to further explain the case where the time indicated by the MMT counter value X is smaller than 26.5 h, the relationship of the following equation (8) is derived from the characteristics of the MMT 32 + nbit counter, and from the characteristics of the MPEG-2 TS 33-bit counter. The relationship of the following equation (9) is derived.

The t1 of the equations (8) and (9) is the time when the counter value of the MMT becomes X and the counter value of the MPEG-2 TS becomes X'.

The PTS / DTS conversion unit 57 substitutes the PTS / DTS value of MMT as the value of X into the equations (8) and (9), and obtains the counter value X'by the same calculation as the above equation (6). It will be. Even when n is a value other than 24, it is possible to convert the counter value in the same manner.

<5-2. Clock conversion of clock conversion unit 54>
Next, the conversion of the counter value representing the reference time will be described.

In MMT, the reference time included in the time information in NTP format is represented as a counter value of 32 + nbit.

The clock conversion unit 54 converts a 32 + nbit counter value as a counter value of a 33-bit counter with an accuracy of 90 kHz as described with reference to FIG. 26.

Further, the clock conversion unit 54 adds a counter value of a 9-bit counter with an accuracy of 27 MHz to the LSB side of the counter value of 33 bits obtained by conversion, so that the accuracy of 27 MHz is as shown in FIG. 27. 42-bit counter value is generated. In FIG. 27, the right side is the LSB side.

The 9-bit counter with an accuracy of 27 MHz is, for example, a counter possessed by the clock conversion unit 54 as a self-propelled free-run counter. The upper limit of the counter value of this 9-bit counter is set so as to take a value in the range of 0 to 300.

The clock conversion unit 54 outputs the 42-bit counter value thus generated to the TS conversion processing unit 62 as PCR. PCR is expressed as ³³ bits (90 kHz counter 0 to (233 -1)) + 9 bits (27 MHz counter 0 to 300).

<6. Information processing device operation>
Next, a series of processes of the information processing apparatus 1 for converting the contents of the MMT into the contents of the MPEG-2 TS and recording the contents on the recording medium 11 will be described with reference to the flowchart of FIG. 28.

In step S1, the receiving unit 31 acquires the contents of the MMT supplied via the broadcast wave transmission line or the communication transmission line. The receiving unit 31 outputs the TVL stream, which is the data of the MMT contents, to the MMT signal processing unit 32.

In step S2, the demultiplexer 51 of the MMT signal processing unit 32 extracts an IP packet from the TLV packets constituting the TLV stream and performs DEMUX processing. That is, the IP packet including the time information in the NTP format is supplied to the comparison unit 52, and the MMTP packet storing the MMT-SI is supplied to the MMT-SI analysis unit 55. The MMTP packet storing the video data and audio data constituting the MPU is supplied to the MPU / MMT packet analysis unit 58.

In step S3, the MPU / MMT packet analysis unit 58 extracts the video data and audio data of the MFU constituting each MPU from the MMTP packet. The extracted video data and audio data are output to the buffer 61 together with the MPU_sequence_number and stored.

In step S4, the comparison unit 52 sets the initial value of the NTP clock counter 53 based on the reference time (32 + nbit counter value) included in the time information in the NTP format.

In step S5, the NTP clock counter 53 outputs a system clock signal representing a counter value of 32 + nbit.

In step S6, the clock conversion unit 54 converts the 32 + nbit counter value represented by the 2 ⁿ Hz system clock signal supplied from the NTP clock counter 53 into a 42-bit counter value indicating PCR of MPEG-2 TS. The 42-bit counter value obtained by conversion is supplied to the TS conversion processing unit 62 as PCR.

In step S7, the MMT-SI analysis unit 55 acquires the MPU time stamp descriptor and the MPU extended time stamp descriptor by analyzing the MMT-SI stored in the MMTP packet, and obtains the PTS of each MPU and each MPU. Calculate the DTS of the access units that make up.

In step S8, the PTS / DTS conversion unit 57 converts the 32 + nbit counter values that represent PTS and DTS calculated by the MMT-SI analysis unit 55 into the counter values that represent the 33-bit PTS and DTS of the MPEG-2 TS, respectively. Convert.

In step S9, the TS conversion processing unit 62 configures a PES using video data and audio data in which the same value is set as MPU_sequence_number, and generates a PES packet by adding PTS and DTS related to each PES. ..

In step S10, the TS conversion processing unit 62 converts the PES packet into a TS packet.

In step S11, the TS conversion processing unit 62 inserts a TS packet containing the PCR generated by the clock conversion unit 54 in the adaptation field into the TS packet in which the data of the PES packet is stored in the payload, and generates a TS. .. As appropriate, a TS packet containing the SI generated based on the MMT-SI is also inserted into the TS.

In step S12, the recording control unit 33 controls the drive 34 and causes the recording medium 11 to record the TS generated by the MMT signal processing unit 32.

By the above processing, MMT contents having different time axes can be converted into MPEG-2 TS contents and recorded on a recording medium such as BD.

<7. Modification example>
<7-1. About inverse conversion ＞
In the above, the case of converting the MMT content to the MPEG-2 TS content has been described, but it is also possible to convert the MPEG-2 TS content to the MMT content.

In this case, the information processing apparatus 1 acquires video data and audio data from the TS packet constituting the TS of the content of the MPEG-2 TS to be converted, and stores the video data and the audio data in the MMTP packet again.

Further, the information processing apparatus 1 extracts the PCR represented as a 42-bit counter value from the TS packet and converts it into an NTP format reference time represented as a 32 + nbit counter value.

Further, the information processing apparatus 1 extracts the MPEG-2 TS PTS / DTS (90 kHz) represented as a 33-bit counter value from the PES packet and converts it into the MMT PTS / DTS represented as a 32 + nbit counter value. ..

The information processing apparatus 1 generates an MMTP packet for storing video data and audio data, and an MMTP packet for storing a control message for transmitting a descriptor used for calculating PTS / DTS of MMT. Further, the information processing apparatus 1 generates a TLV stream by multiplexing an IP packet containing the generated MMTP packet and an IP packet including time information in NTP format.

Here, the reverse conversion of PTS / DTS and the reverse conversion of the reference time are performed as follows.

That is, the MMT counter value X is obtained by the following procedure based on the MPEG-2 TS counter value X'.

・ Procedure 1
Obtain the MMT counter value A at the wraparound time in MPEG-2 TS.

The MMT counter value A is a value corresponding to a time ( ^≈26.5 h) of 233 / 90k in real time. Assuming that n = 24, the counter value A is obtained as in the above equation (5).

The process of this procedure 1 is the same as the process of converting the MMT content into the MPEG-2 TS content.

・ Procedure 2
When the time indicated by the MPEG-2 TS counter value X'is smaller than the time indicated by the MPEG-2 TS counter value X', the MMT counter value X indicating the same time as the time indicated by the following equation (10) Is required.

On the other hand, when the time indicated by the MPEG-2 TS counter value X'is larger than 26.5 h, the MMT counter value X sets the value of the remainder mod (X'/ A) to the MPEG-2 TS counter value X'. It is obtained as shown in the following equation (11).

The PTS / DTS of the MPEG-2 TS is converted to the PTS / DTS of the MMT by such an inverse conversion process. Further, in the PCR of the MPEG-2 TS having the configuration shown in FIG. 27, the 33-bit counter value on the MSB side is converted into a 32 + n-bit counter value by such an inverse conversion process, and is used as a reference time.

From this procedure, it is possible to convert only to a counter value of A or less, but it is also possible to convert to a counter value of A or more by using an internal counter and adding a counter value as appropriate.

<7-2. Other examples>
In the above, the conversion from MMT to MPEG-2 TS and the reverse conversion from MPEG-2 TS to MMT have been described, but the format of the content to be converted / reverse conversion is limited to MMT and MPEG-2 TS. It's not a thing. The above-mentioned processing can be used for conversion / inverse conversion of various formats having different time axes.

Although it is assumed that the content to be converted / inverse converted includes both video data and audio data, the above-mentioned processing may be applied to the conversion / inverse conversion of the format of either content. .. Further, it may be applied to conversion / inverse conversion of formats of data of types other than video data and audio data.

-Computer configuration example The above-mentioned series of processes can be executed by hardware or software. When a series of processes are executed by software, the programs constituting the software are installed from a program recording medium on a computer embedded in dedicated hardware, a general-purpose personal computer, or the like.

FIG. 29 is a block diagram showing an example of hardware configuration of a computer that executes the above-mentioned series of processes programmatically.

The CPU (Central Processing Unit) 1001, the ROM (Read Only Memory) 1002, and the RAM (Random Access Memory) 1003 are connected to each other by the bus 1004.

An input / output interface 1005 is further connected to the bus 1004. An input unit 1006 including a keyboard, a mouse, and the like, and an output unit 1007 including a display, a speaker, and the like are connected to the input / output interface 1005. Further, a storage unit 1008 including a hard disk and a non-volatile memory, a communication unit 1009 including a network interface, and a drive 1010 for driving the removable media 1011 are connected to the input / output interface 1005.

In the computer configured as described above, the CPU 1001 loads the program stored in the storage unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004 and executes the above-mentioned series of processes. Is done.

The program executed by the CPU 1001 is recorded on the removable media 1011 or provided via a wired or wireless transmission medium such as a local area network, the Internet, or a digital broadcast, and is installed in the storage unit 1008.

The program executed by the computer may be a program in which processing is performed in chronological order according to the order described in the present specification, in parallel, or at a necessary timing such as when a call is made. It may be a program in which processing is performed.

The embodiment of the present technology is not limited to the above-described embodiment, and various changes can be made without departing from the gist of the present technology.

For example, the present technology can be configured as cloud computing in which one function is shared by a plurality of devices via a network and jointly processed.

Further, each step described in the above-mentioned flowchart may be executed by one device or may be shared and executed by a plurality of devices.

Further, when a plurality of processes are included in one step, the plurality of processes included in the one step can be executed by one device or shared by a plurality of devices.

The embodiment of the present technology is not limited to the above-described embodiment, and various changes can be made without departing from the gist of the present technology.

-Example of combination of configurations This technology can also have the following configurations.
(1)
A receiver that receives the first multiplexed data obtained by multiplexing the data including the video stream and the audio stream in the first format.
The first counter value of the number of bits specified in the first format, which is included as time information in the first multiplexed data, is converted into the second counter value of the number of bits specified in the second format. Conversion part and
The video data constituting the video stream and the audio data constituting the audio stream are each stored in the packet of the second format, and the second counter value is included as time information, the second of the second format. An information processing device equipped with a multiplexing unit that generates multiplexed data.
(2)
The first format is MMT.
The second format is MPEG-2 TS. The information processing apparatus according to (1) above.
(3)
The first counter value representing the reference time is 32 + n bits in which the time in seconds of 1 second or more is represented by 32 bits and the accuracy of 1 second or less is 2 ⁿ Hz (n is an integer of 24 or more). It is a counter value
The second counter value representing the reference time is a 42-bit counter value in which a time of 1 second or longer is represented by 33 bits with an accuracy of 90 kHz and a time of 1 second or less is represented by 9 bits with a precision of 27 MHz. The information processing apparatus according to (2) above.
(4)
The multiplexing unit adds a TS packet containing PCR as the second counter value representing a reference time to the TS packet for storing the video data and the TS packet for storing the audio data, and the first The information processing apparatus according to (3) above, which generates the multiplexed data of 2.
(5)
The first counter value representing the presentation time and the first counter value representing the decoding time are 32 + n bits in which the time in seconds of 1 second or more is represented by 32 bits and the accuracy of 1 second or less is 2 ⁿ Hz. Is the counter value of
The information processing apparatus according to (3) or (4), wherein the second counter value representing the presentation time and the second counter value representing the decoding time are counter values represented by 33 bits with an accuracy of 90 kHz.
(6)
The multiplexing unit represents the PTS as the second counter value indicating the presentation time and the decoding time in the TS packet for storing the video data and the PES packet which is the source of the TS packet for storing the audio data. The information processing apparatus according to (5) above, which adds a DTS as a second counter value.
(7)
The information processing apparatus according to (6), wherein the multiplexing unit generates a PES packet containing the video data and the audio data belonging to the same MPU based on the MPU sequence number.
(8)
The information processing apparatus according to any one of (2) to (7), further comprising a recording control unit for recording TS as the second multiplexed data on a recording medium.
(9)
The information processing apparatus according to (8) above, wherein the recording medium is a Blu-ray (registered trademark) Disc.
(10)
Receives the first multiplexed data, which is the multiplexing of the data including the video stream and the audio stream in the first format.
The first counter value of the number of bits specified in the first format, which is included as time information in the first multiplexed data, is converted into the second counter value of the number of bits specified in the second format. ,
The video data constituting the video stream and the audio data constituting the audio stream are each stored in the packet of the second format, and the second counter value is included as time information, the second of the second format. An information processing method that includes steps to generate multiplexed data for.
(11)
Receives the first multiplexed data, which is the multiplexing of the data including the video stream and the audio stream in the first format.
The first counter value of the number of bits specified in the first format, which is included as time information in the first multiplexed data, is converted into the second counter value of the number of bits specified in the second format. ,
The second of the second format, which stores the video data constituting the video stream and the audio data constituting the audio stream in the packet of the second format, and includes the second counter value as time information. A program that causes a computer to perform a process that includes a step to generate the multiplexed data of.

1 Information processing device, 11 Recording medium, 32 MMT signal processing unit, 33 Recording control unit, 51 Demultiplexer, 52 Comparison unit, 53 NTP clock counter, 54 clock conversion unit, 55 MMT-SI analysis unit, 56 Video / audio synchronization control Unit, 57 PTS / DTS conversion unit, 58 MPU / MMT packet analysis unit, 59 decoder buffer, 60 video / audio decoder, 61 buffer, 62 TS conversion processing unit

Claims (9)

A clock converter that converts NTP format time information contained in MMT data, which is MMT -multiplexed data including video streams and audio streams, into MPEG-2 TS PCR , and
A time stamp conversion unit that converts PTS and DTS contained in the MMT data into PTS and DTS of MPEG-2 TS, respectively.
A PES packet is generated by combining the video data constituting the video stream and the audio data belonging to the same MPU among the audio data constituting the audio stream based on the MPU sequence number, and after conversion. An information processing device equipped with a multiplexing unit that generates TS data by adding related PTSs and DTSs based on the MPU sequence number . The time information in NTP format is represented by 32 bits for the time in seconds of 1 second or more, and is represented by a 32 + n-bit counter value in which the accuracy of 1 second or less is 2 ⁿ Hz (n is an integer of 24 or more).
The MPEG-2 TS PCR is represented by a 42-bit counter value in which a time of 1 second or longer is represented by 33 bits with an accuracy of 90 kHz and a time of 1 second or less is represented by 9 bits with a precision of 27 MHz. The information processing apparatus according to 1 . The information processing according to claim 1 or 2 , wherein the multiplexing unit adds a TS packet containing PCR to a TS packet for storing the video data and a TS packet for storing the audio data to generate the TS data. Device. The PTS and DTS included in the MMT data are represented by 32 bits for the time in seconds of 1 second or more, and are represented by a 32 + n-bit counter value with an accuracy of 1 second or less as 2 ⁿ Hz.
The information processing apparatus according to any one of claims 1 to 3, wherein the PTS and DTS of the MPEG-2 TS are represented by a counter value represented by 33 bits with an accuracy of 90 kHz. The time stamp conversion unit performs different calculations depending on whether the time indicated by the PTS and DTS contained in the MMT data is larger or smaller than 26.5h, and the MPEG-corresponding to the PTS and DTS included in the MMT data. 2 The information processing apparatus according to any one of claims 1 to 4 for obtaining PTS and DTS of TS . The information processing apparatus according to any one of claims 1 to 5, further comprising a recording control unit for recording the TS data on a recording medium. The information processing apparatus according to claim 6 , wherein the recording medium is a Blu-ray (registered trademark) Disc. MMT -multiplexed data including video stream and audio stream NTP format time information contained in MMT data is converted to MPEG-2 TS PCR .
The PTS and DTS contained in the MMT data are converted into PTS and DTS of MPEG-2 TS, respectively.
A PES packet is generated by combining the video data constituting the video stream and the video data belonging to the same MPU among the audio data constituting the audio stream based on the MPU sequence number, and after conversion. Generate TS data by adding the related PTS and DTS of the PTS and DTS based on the MPU sequence number .
Information processing method. MMT -multiplexed data including video stream and audio stream NTP format time information contained in MMT data is converted to MPEG-2 TS PCR .
The PTS and DTS contained in the MMT data are converted into PTS and DTS of MPEG-2 TS, respectively.
A PES packet is generated by combining the video data constituting the video stream and the video data belonging to the same MPU among the audio data constituting the audio stream based on the MPU sequence number, and after conversion. Generate TS data by adding the related PTS and DTS of the PTS and DTS based on the MPU sequence number .
A program that causes a computer to perform processing .

Images