JP5958008B2 - Stream processing apparatus, stream processing method, and stream processing program - Google Patents

Description

  The present invention relates to a stream processing device, a stream processing method, and a stream processing program, and more particularly to a stream processing device, a stream processing method, and a stream processing program for adjusting the frequency of a reference clock without increasing a delay time.

  Japanese Unexamined Patent Application Publication No. 2007-104085 (Patent Document 1) discloses a technique for adjusting the frequency of a reference clock by monitoring the occupation amount of packets accumulated in a buffer. That is, when the packet of the second data stream is converted into a format corresponding to the first data stream, when the packet is temporarily buffered, the packet of the second data stream is stored in the buffer, and Check packet occupancy. When the occupancy is below the predetermined range, the pace at which packets are read from the buffer is slowed down. When the occupancy is above the specified range, the pace at which packets are read out from the buffer is increased, and the occupancy is within the specified range. In some cases, packets are read from the buffer at a predetermined pace.

JP 2007-104085 A

  However, in the technique described in Patent Document 1, in order to adjust the frequency of the reference clock, it is necessary to accumulate packets in a buffer up to a predetermined specified range. For example, the received packets are sequentially accumulated in the buffer, and after the packets are accumulated to a predetermined specified range, the first received packet is taken out from the buffer and moved to the next processing. Therefore, a residence time from when the packet enters the buffer until it leaves the buffer is generated, and even if the packet is received, the packet cannot be processed until the residence time elapses. That is, a delay time until the received packet is transferred to the next process occurs.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a stream processing apparatus capable of adjusting the frequency of the reference clock without increasing the delay time in a stream processing configuration. It is to provide a stream processing method and a stream processing program.

  (1) In order to solve the above problems, a stream processing apparatus according to an aspect of the present invention is a stream processing apparatus for processing a stream obtained by dividing information of one or a plurality of programs into a plurality of packets, The packet includes a header and a payload, at least some headers of the plurality of packets include information for clock recovery, each header of the plurality of packets includes time information, and clock generation for generating a reference clock And a counter for updating the count value based on the reference clock, and a clock for adjusting the frequency of the reference clock by controlling the clock generation unit based on the count value and the time information And an adjustment unit.

  As described above, the count value and the time information can be acquired from the time when the stream processing apparatus receives the packet, so that the processing for controlling the clock generation unit and the processing for decoding can be started from the time, In addition, the buffer at the gate is not necessary, and the video decoding process can be started immediately. Therefore, it is possible to suppress an increase in delay time from the time when the stream processing apparatus receives the packet until the packet is decoded, for example.

  Further, by using the count value based on the clock in the stream processing device and the time information based on the clock in the station side device such as a VOD (Video On Demand) server, the clock in the stream processing device is Can be synchronized to the clock.

  In this way, since the count value is synchronized with the clock in the station side device, the stream processing device performs processing based on the clock in the stream processing device, so that the stream created based on the clock in the station side device is Each packet of the stream can be processed with the same timing relationship as the processing timing of each packet of the stream in the station side device.

  (2) Preferably, the clock adjustment unit calculates an evaluation value of a virtual buffer for the packet based on the count value and the time information, and the clock generation unit is set according to the calculated evaluation value. By controlling the frequency of the reference clock.

  With such a configuration, an evaluation value of a virtual buffer can be calculated without actually providing a buffer, so that the clock in the stream processing apparatus can be appropriately controlled.

  Since the stream processing apparatus does not mount a buffer, the received packet can be transferred to the next process without staying in the buffer.

  (3) Preferably, the clock adjustment unit adjusts the frequency of the reference clock based on the difference between the count value and the time information.

  Thus, by calculating the difference between the count value based on the clock in the stream processing device and the time information based on the clock in the station side device, the operation timing in the stream processing device and the difference in operation timing in the station side device Can be analyzed, so that the clock in the stream processing apparatus can be adjusted accurately.

  In addition, since the count value and time information necessary for calculating the difference are acquired at the time when the packet is received, the process for calculating the difference can be started from the time. Therefore, the stream processing apparatus can reflect the difference in the adjustment of the clock frequency in the stream processing apparatus at the time when the packet is received.

  (4) Preferably, the clock adjustment unit selects one or a plurality of the packets based on the magnitude relationship between the count value and the time information, and the reference clock based on the total data amount of the selected packets. Adjust the frequency.

  With this configuration, it is possible to virtually determine the amount of packets stored in the buffer. By monitoring the amount of storage over time, the operation timing in the stream processing device and the operation timing in the station side device are shifted. Over time can be analyzed. This makes it possible to accurately adjust the clock in the stream processing apparatus.

  (5) More preferably, the clock adjustment unit adjusts the frequency of the reference clock so that the difference becomes a predetermined value.

  If the clock in the stream processing device and the clock in the station side device are synchronized, the difference between the count value based on the clock in the stream processing device and the time information based on the clock in the station side device becomes constant, so the above difference Becomes constant. By adjusting the frequency of the clock in the stream processing device, the clock in the stream processing device can be synchronized with the clock in the station side device.

  (6) More preferably, the clock adjustment unit adjusts the frequency of the reference clock based on a difference between the time information and the count value of a part of the plurality of packets.

  With such a configuration, the difference between the count value based on the clock in the stream processing device and the time information based on the clock in the station side device is calculated from some packets instead of all the packets received by the stream processing device. Therefore, the calculation burden on the stream processing apparatus can be reduced.

  (7) More preferably, the clock adjustment unit adjusts the frequency of the reference clock so that the total becomes a predetermined value.

  In this way, in the packet received by the stream processing device, the number of bytes of the packet is included in the total from the timing when the packet is received until the timing when the count value becomes the value indicated by the time information. When the clock in the stream processing device and the clock in the station-side device are synchronized, the total is constant. Therefore, by adjusting the frequency of the clock in the stream processing device so that the total is constant, the stream processing device Can be synchronized with the clock in the station side device.

  (8) More preferably, the clock adjustment unit calculates a difference between the count value and the time information as a retention time of the packet in a virtual buffer, and based on the calculated retention time, Adjust the frequency.

  With such a configuration, since the virtual residence time can be calculated as the evaluation value of the virtual buffer, the clock in the stream processing apparatus can be accurately adjusted.

  (9) More preferably, the clock adjustment unit calculates a sum of data amounts of one or a plurality of the packets whose time information is equal to or less than the count value as an accumulation amount of the packets in a virtual buffer, The frequency of the reference clock is adjusted based on the calculated accumulation amount.

  With such a configuration, since the virtual accumulation amount can be calculated as the evaluation value of the virtual buffer, the clock in the stream processing apparatus can be adjusted accurately.

  (10) Preferably, the stream processing apparatus further includes a payload processing unit for performing predetermined processing on the payload of the packet, and the payload processing unit stores the clock reproduction information and the time information. If they are synchronized, the predetermined processing is performed according to the timing of the reference clock.

  As described above, when the clock reproduction information and the time information are synchronized, the stream processing apparatus does not need to perform a gate operation on the received packet to reproduce the clock based on the clock reproduction information. Predetermined processing can be performed on the packet.

  In addition, since the operation for regenerating the clock by the information for clock recovery is not necessary, the processing load on the stream processing apparatus can be reduced, and the gate and the buffer in the gate are also unnecessary.

  (11) A stream processing method according to an aspect of the present invention is a stream processing method in a stream processing apparatus for processing a stream obtained by dividing information of one or a plurality of programs into a plurality of packets, and the packet Includes a header and a payload, at least some headers of the plurality of packets include information for clock recovery, each header of the plurality of packets includes time information, and generates a reference clock; and the reference clock And updating the count value based on the above and adjusting the frequency of the reference clock based on the count value and the time information.

  As described above, the count value and the time information can be acquired from the time when the stream processing apparatus receives the packet, so that the processing for controlling the clock generation unit and the processing for decoding can be started from the time, In addition, the buffer at the gate is not necessary, and the video decoding process can be started immediately. Therefore, it is possible to suppress an increase in delay time from the time when the stream processing apparatus receives the packet until the packet is decoded, for example.

  Also, by using the count value based on the clock in the stream processing device and the time information based on the clock in the station side device such as the VOD server, the clock in the stream processing device is synchronized with the clock in the station side device. Can do.

  In this way, since the count value is synchronized with the clock in the station side device, the stream processing device performs processing based on the clock in the stream processing device, so that the stream created based on the clock in the station side device is Each packet of the stream can be processed with the same timing relationship as the processing timing of each packet of the stream in the station side device.

  (12) A stream processing program according to an aspect of the present invention is a stream processing program used in a stream processing apparatus for processing a stream obtained by dividing information of one or a plurality of programs into a plurality of packets, The packet includes a header and a payload; at least some of the headers of the plurality of packets include clock recovery information; each header of the plurality of packets includes time information; and generating a reference clock to the computer And a step of updating a count value based on the reference clock and a step of adjusting the frequency of the reference clock based on the count value and the time information.

  As described above, the count value and the time information can be acquired from the time when the stream processing apparatus receives the packet, so that the processing for controlling the clock generation unit and the processing for decoding can be started from the time, In addition, the buffer at the gate is not necessary, and the video decoding process can be started immediately. Therefore, it is possible to suppress an increase in delay time from the time when the stream processing apparatus receives the packet until the packet is decoded, for example.

  Also, by using the count value based on the clock in the stream processing device and the time information based on the clock in the station side device such as the VOD server, the clock in the stream processing device is synchronized with the clock in the station side device. Can do.

  In this way, since the count value is synchronized with the clock in the station side device, the stream processing device performs processing based on the clock in the stream processing device, so that the stream created based on the clock in the station side device is Each packet of the stream can be processed with the same timing relationship as the processing timing of each packet of the stream in the station side device.

  The present invention provides a stream processing device, a stream processing method, and a stream processing program that can adjust the frequency of a reference clock without increasing the delay time in a stream processing configuration.

It is a figure which shows the structure of the stream processing system which concerns on embodiment of this invention. It is a figure which shows an example of the stream processed in the stream processing apparatus which concerns on embodiment of this invention. It is a figure which shows the structure of the stream processing apparatus which concerns on embodiment of this invention. It is the flowchart which defined the operation | movement procedure at the time of performing the reproduction | regeneration process of a TTS clock based on the average virtual residence time calculated using the virtual buffer in the stream processing apparatus which concerns on embodiment of this invention. It is the flowchart which defined the operation | movement procedure at the time of performing the calculation process of the virtual accumulation | storage amount in a virtual buffer in the stream processing apparatus which concerns on embodiment of this invention. It is the flowchart which defined the operation | movement procedure at the time of performing the reproduction | regeneration process of a TTS clock based on the average virtual accumulation amount calculated using the virtual buffer in the stream processing apparatus which concerns on embodiment of this invention. It is the flowchart which defined the operation | movement procedure at the time of processing the TTS packet received in the stream processing apparatus which concerns on embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals and description thereof will not be repeated.

[Overview of stream processing system]
FIG. 1 is a diagram showing a configuration of a stream processing system according to an embodiment of the present invention.
Referring to FIG. 1, the stream processing system 13 includes a station side device 10 such as a VOD server, a stream processing device 20, and a display device 12. The station apparatus 10 is connected to the stream processing apparatus 20 via the network 11.

  The station-side device 10 transmits a stream including information on one or more programs via the network 11. The network 11 is, for example, the Internet, and the station-side device 10 and the stream processing device 20 transmit and receive streams using IP (Internet Protocol).

FIG. 2 is a diagram showing an example of a stream processed in the stream processing apparatus according to the embodiment of the present invention.
Referring to FIG. 2, (A) in FIG. 2 shows a stream transmitted from a station side apparatus using, for example, a container format of MPEG2-TS (Moving Picture Experts Group 2 Transport Stream) standard. Includes program information.

  MPEG2-TS includes a plurality of TS packets. The TS packet has 188 bytes of data and includes a TS header and a payload. Video TS packets indicated by V1, V2, V3, V4, etc. are, for example, MPEG2 or H.264 in the payload. It includes a video ES (Elementary Stream) which is video data obtained by encoding the program information by H.264 / AVC or the like. Further, the TS header includes information such as a PID (Packet Identifier) corresponding to video data included in the payload.

  The audio TS packet indicated by A1 includes an audio ES (Elementary Stream) that is audio data obtained by encoding audio information in the payload. The TS header includes information such as PID corresponding to the audio data included in the payload.

  A PCR packet indicated by PCR (Program Clock Reference) includes PCR in an adaptation field included in the TS header. The PCR is used, for example, to synchronize the STC (System Time Clock) or the PCR clock of the stream processing device 20 with the PCR clock used in the station side device. Hereinafter, the PCR clock in the station side device 10 is referred to as a station side PCR clock.

  A null packet indicated by NULL is a TS packet not including data, and is indicated by a PID of 0x1FFF. When a TS stream is transmitted by broadcast waves, the transmission bit rate is adjusted by inserting a null packet into the TS stream when the amount of data to be transmitted is small because it occupies a predetermined frequency band and time. Is done.

  On the other hand, when a stream is transmitted from the station-side device 10 via the network 11 such as the Internet as shown in FIG. 1, the frequency band and time required for information transmission can be saved by reducing the amount of data to be transmitted. preferable.

  Therefore, when the station side apparatus 10 retransmits, for example, the MPEG2-TS shown in FIG. 2A via the network 11, the TS packet unrelated to program information such as a null packet is deleted. For example, it is converted into MPEG2-TTS (Timestamped TS) shown in (B) of FIG. Thereby, the station side apparatus 10 can save the frequency band and time required for information transmission.

  A stream transmitted via the network 11 includes a plurality of TTS packets. The TTS packet is a packet in which a 4-byte TTS header is added to the TS packet, and has a size of 192 bytes. The TTS header includes a 4-byte time stamp. Note that the TTS header may be included in the TS header.

  When the stream processing apparatus 20 receives the TTS packet from the station side apparatus 10 via the network 11, the decoder decodes the encoded program information included in the received TTS packet, and the decoded program image and audio information For example, it is reproduced on the display device 12.

  However, when the station-side device 10 transmits a stream via the network 11, the stream is transferred to the stream due to fluctuations in the transmission speed, communication path, processing capability of the relay device, line congestion, etc. in the network 11 and deletion of NULL packets. The transmission time of included packets from the station side device 10 to the stream processing device 20 varies for each packet. For this reason, the following problems occur.

  That is, when the station side device 10 transmits each packet of a stream by a broadcast wave, the station side device 10 indicates the relative transmission timing that is the difference in time at which each packet is transmitted, and the difference in time at which each packet should be input to the decoder. Each packet is transmitted in accordance with the relative decoding timing. Specifically, for example, when the timing at which the second packet following the first packet is to be input to the decoder is 1 second after the first packet, the station-side device 10 transmits the second packet to the first packet. Transmit by broadcast wave 1 second after packet.

  When the station-side device 10 transmits a stream packet by broadcast waves, the transmission time until the packet reaches the stream processing device 20 from the station-side device 10 is constant in all packets.

  Therefore, when receiving the stream by the broadcast wave, the stream processing device 20 inputs each packet of the received stream to the decoder at the received timing, thereby decoding each packet while maintaining the relative decoding timing. Can be thrown into.

  On the other hand, when the station side device 10 transmits a stream via the network 11, the packet transmission time from the station side device 10 to the stream processing device 20 varies for each packet.

  Therefore, even if the stream processing apparatus 20 inputs each packet of the stream received via the network 11 to the decoder at the reception timing, the stream processing apparatus 20 cannot input each packet to the decoder while maintaining the relative decoding timing. . As a result, the stream processing apparatus 20 cannot appropriately reproduce the program information included in the stream.

  For this reason, the station side device 10 generates time information indicating the relative decoding timing at which each TS packet should be input to the decoder based on the TTS clock in the station side device 10 of 27 MHz. The station side device 10 creates a TTS packet by adding a TTS header including the generated time information as a TTS time stamp to the TS packet, and transmits the created TTS packet via the network 11. Hereinafter, the TTS clock in the station side device 10 is referred to as a station side TTS clock.

  If the stream processing apparatus 20 can synchronize its own reference clock with the station-side TTS clock, the stream processing apparatus 20 can input the packet to the decoder at the relative decoding timing based on the TTS time stamp in the received TTS packet.

[Configuration of stream processing device]
FIG. 3 is a diagram showing the configuration of the stream processing apparatus according to the embodiment of the present invention.
Referring to FIG. 3, the stream processing apparatus 20 includes a time stamp acquisition unit 21, a TTS gate unit 22, a virtual buffer calculation unit (clock adjustment unit) 23, a TTS counter unit (counter) 24, and a TTS clock adjustment. Unit (clock adjustment unit) 25, TTS clock generation unit 26, demax unit 27, PCR recovery unit 28, video ES buffer 29, video decoder unit (payload processing unit) 30, audio ES buffer 31, And an audio decoder unit (payload processing unit) 32.

  The time stamp acquisition unit 21 acquires the TTS time stamp from the TTS header of the TTS packet included in the stream received by the stream processing device 20, and outputs the acquired TTS time stamp to the virtual buffer calculation unit 23. Then, the time stamp acquisition unit 21 outputs the TTS packet for which the TTS time stamp has been acquired to the TTS gate unit 22.

  When receiving the TTS packet from the time stamp acquisition unit 21, the TTS gate unit 22 performs a gate operation based on the TTS time stamp of the received TTS packet and the count value in the TTS counter unit 24. More specifically, when receiving the TTS packet, the TTS gate unit 22 monitors the count value from the TTS counter unit 24, and when the count value matches the TTS timestamp of the TTS packet, the TTS header The TS packet is created by deleting, and the created TS packet is output to the demux unit 27.

  Further, the TTS gate unit 22 does not perform the gate operation when the TTS time stamp of the TTS packet in the stream and the PCR of the PCR packet described later are synchronized. Here, the synchronization means that the TTS time stamp and the PCR are count values generated by the same clock.

  For example, the TTS gate unit 22 acquires a PCR from a plurality of PCR packets and a TTS time stamp of the PCR packet. The TTS gate unit 22 determines that the TTS time stamp and the PCR are synchronized when the difference between the PCR and the TTS time stamp in each acquired packet is constant. If it is determined that the stream processing system is synchronized, that is, if it is determined that the station side device 10 is synchronized, there is no need to make a determination.

  The TTS clock generation unit 26 generates a TTS clock that is a reference clock in the stream processing device 20. The TTS clock generated by the TTS clock generation unit 26 is used as a 27 MHz reference clock used for the TTS counter unit 24 or a PCR counter (not shown). Specifically, the TTS clock generation unit 26 includes a VCXO (Voltage Controlled Xtal Oscillator) and is an oscillator using a crystal such as a crystal. By adjusting the voltage output to the VCXO, the resonance frequency of the VCXO changes.

  The TTS clock generation unit 26 outputs a clock generated by, for example, VCXO to the TTS counter unit 24. Further, the voltage output to the VCXO is adjusted by the TTS clock adjustment unit 25, whereby the frequency of the clock output from the TTS clock generation unit 26 to the TTS counter unit 24 is changed. Hereinafter, the TTS clock in the stream processing device 20 may be referred to as a local TTS clock.

  The TTS counter unit 24 updates the TTS count value based on the clock received from the TTS clock generation unit 26, for example. Then, the TTS counter unit 24 outputs the updated count value to the TTS gate unit 22 and the virtual buffer calculation unit 23. Specifically, the TTS count value is incremented every time a clock is received, for example. Therefore, the TTS counter unit 24 increases the count value in synchronization with the TTS clock as time elapses. As a result, the count value can be used as the current time, and the time-series context can be acquired from the magnitude relationship of the count value.

  Based on the TTS count value acquired from the TTS counter unit 24 and the TTS time stamp received from the time stamp acquisition unit 21, the virtual buffer calculation unit 23 uses the virtual buffer residence time and the virtual buffer storage amount as the virtual buffer evaluation values. Calculate Then, the virtual buffer calculation unit 23 determines whether the frequency of the local TTS clock is higher, lower, or appropriate than the local TTS clock according to the calculated virtual buffer residence time and the virtual buffer accumulation amount. The result is notified to the TTS clock adjustment unit 25.

  More specifically, the virtual buffer calculation unit 23 performs the following operation when calculating the virtual buffer residence time. That is, when the virtual buffer calculation unit 23 receives the first TTS time stamp from the time stamp acquisition unit 21 after the user selects a program or an error occurs, the virtual buffer calculation unit 23 uses the count value in the TTS counter unit 24 as the first TTS time stamp. Initialize with. The virtual buffer calculation unit 23 receives a TTS time stamp from the time stamp acquisition unit 21 and acquires a TTS count value indicating the current time from the TTS counter unit 24. The TTS count value indicates the time when the stream processing apparatus 20 receives a TTS packet corresponding to the TTS time stamp.

  Then, the virtual buffer calculation unit 23 calculates the difference between the TTS time stamp and the TTS count value, and sets the calculated difference as Δt indicating the virtual residence time in the virtual buffer. The virtual buffer calculation unit 23 calculates Δt for a plurality of TTS packets, and analyzes the tendency of Δt based on the calculated plurality of Δt.

  At this time, the virtual buffer calculation unit 23 may analyze the tendency of Δt from two calculated Δt, for example. The virtual buffer calculation unit 23 may calculate, for example, AvgΔt indicating a moving average of a plurality of Δt, and analyze the tendency of AvgΔt based on the calculated AvgΔt.

  For example, when Δt or AvgΔt tends to increase, the virtual buffer calculation unit 23 determines that the frequency of the local TTS clock is lower than the frequency of the station-side TTS clock. Details of the determination performed by the virtual buffer calculation unit 23 will be described later.

  For example, when Δt or AvgΔt tends to decrease, the virtual buffer calculation unit 23 determines that the frequency of the local TTS clock is higher than the frequency of the station-side TTS clock. For example, when Δt or AvgΔt is constant, the virtual buffer calculation unit 23 determines that the frequency of the local TTS clock and the frequency of the station-side TTS clock are the same.

  Then, the virtual buffer calculation unit 23 notifies the clock adjustment unit 25 of the determination result indicating whether the frequency of the local TTS clock is higher, lower, or the same as the frequency of the local TTS clock.

  Further, the virtual buffer calculation unit 23 performs the following operations when calculating the virtual buffer accumulation amount. That is, the virtual buffer calculation unit 23 receives the TTS time stamp from the time stamp acquisition unit 21 and adds 192 to Sn which is the virtual accumulation amount. 192 indicates the number of bytes of the TTS packet.

  Then, the virtual buffer calculation unit 23 regards the TTS time stamp as the time when the TTS packet corresponding to the TTS time stamp leaves the virtual buffer, and monitors the TTS count value in the TTS counter unit 24 to thereby obtain the TTS count value. The following operations are performed at the timing indicating the above time.

  That is, when the TTS count value and the TTS time stamp coincide with each other, the virtual buffer calculation unit 23 subtracts 192 from Sn that is the virtual accumulation amount in the virtual buffer. The virtual buffer calculation unit 23 calculates Sn by performing the above processing on a plurality of TTS packets. In other words, the virtual buffer calculation unit 23 selects a TTS packet whose TTS time stamp of the TTS packet is equal to or greater than the TTS count value, and calculates Sn, which is the total size of the selected TTS packet.

  The virtual buffer calculation unit 23 monitors, for example, the TTS count value in the TTS counter unit 24, acquires Sn every time the TTS count value increases a predetermined value, and displays the tendency of Sn based on the plurality of acquired Sn. analyse.

  At this time, the virtual buffer calculation unit 23 may analyze, for example, the calculated two Sn to Sn trends. The virtual buffer calculation unit 23 may calculate, for example, AvgSn indicating a moving average of a plurality of Sns, and analyze the trend of AvgSn based on the calculated AvgSn.

  For example, when Sn or AvgSn tends to increase, the virtual buffer calculation unit 23 determines that the frequency of the local TTS clock is lower than the frequency of the local TTS clock. Details of the determination performed by the virtual buffer calculation unit 23 will be described later.

  For example, when Sn or AvgSn tends to decrease, the virtual buffer calculation unit 23 determines that the frequency of the local TTS clock is higher than the frequency of the station-side TTS clock. For example, if Sn or AvgSn is constant, the virtual buffer calculation unit 23 determines that the frequency of the local TTS clock and the frequency of the station TTS clock are the same.

  Then, the virtual buffer calculation unit 23 notifies the clock adjustment unit 25 of the determination result indicating whether the frequency of the local TTS clock is higher, lower, or the same as the frequency of the local TTS clock.

  The TTS clock adjustment unit 25 adjusts the frequency of the TTS clock in the TTS clock generation unit 26 based on the determination result from the virtual buffer calculation unit 23. Specifically, when the TTS clock adjustment unit 25 receives a determination result indicating that the frequency of the TTS clock is lower than the frequency of the station-side TTS clock from the virtual buffer calculation unit 23, the TTS clock adjustment unit 25 performs the VCTSO in the TTS clock generation unit 26. The frequency of the TTS clock is increased by adjusting the output voltage.

  When the TTS clock adjustment unit 25 receives a determination result indicating that the frequency of the TTS clock is higher than the frequency of the TTS clock on the station side from the virtual buffer calculation unit 23, the voltage output to the VCXO in the TTS clock generation unit 26 Is adjusted to lower the frequency of the TTS clock.

  The demux unit 27 identifies and separates TS packets such as video TS packets, audio TS packets, and PCR packets received from the TTS gate unit 22 by PID, for example. More specifically, when receiving a video TS packet from the TTS gate unit 22, the demax unit 27 separates the received video TS packet into a TS header and a payload, and sends video data included in the separated payload to the video ES buffer 29. Output.

  In addition, when receiving the audio TS packet from the TTS gate unit 22, the demax unit 27 separates the received audio TS packet into a TS header and a payload, and outputs the audio data included in the separated payload to the audio ES buffer 31.

  Further, upon receiving the PCR packet from the TTS gate unit 22, the demax unit 27 separates the received PCR packet into a TS header and a payload, acquires the PCR included in the adaptation field of the separated TS header, and acquires the acquired PCR. Output to the PCR recovery unit 28.

  When receiving the PCR from the demax unit 27, the PCR recovery unit 28 compares the received PCR and a PCR count value based on the PCR clock in the stream processing device 20 (not shown). Hereinafter, the PCR clock in the stream processing device 20 may be referred to as a local PCR clock.

  Since the PCR is a count value given based on the local side PCR clock, the PCR recovery unit 28 sets the frequency of the local PCR clock so that the difference between the PCR and the PCR count value is constant in a plurality of PCR packets. By adjusting, the local PCR clock is synchronized with the local PCR clock.

  The video ES buffer 29 temporarily holds the video data received from the demax unit 27. The video ES buffer 29 outputs the held video data to the video decoder unit 30 in response to a request from the video decoder unit 30.

  The video decoder unit 30 decodes the video data held in the video ES buffer 29. At this time, the video decoder unit 30 uses the TTS clock as a reference clock when the TTS time stamp and the PCR are synchronized in the received stream. In addition, when the TTS time stamp and the PCR are not synchronized in the received stream, the video decoder unit 30 uses the PCR clock reproduced by the PCR recovery unit 28 as a reference clock.

  More specifically, the video decoder unit 30 holds the DTS (Decoding Time Stamp) and PTS (Presentation) included in the PES header of the video PES (Packetized Elementary Stream) included in the payload of the TTS packet held in the video ES buffer 29. See Time Stamp).

  The DTS specifies the timing for decoding the program image data included in the video PES. The PTS designates the timing for presenting a program image created by decoding the program image data.

  When the DTS coincides with the reference clock, the video decoder unit 30 creates a program image by decoding the video data included in the PES corresponding to the DTS. When the PTS matches the reference clock, the video decoder unit 30 outputs a program image created by decoding the video data included in the PES corresponding to the PTS to the display device 12, for example.

  The audio ES buffer 31 temporarily holds the audio data received from the demax unit 27. The audio ES buffer 31 outputs the held audio data to the audio decoder unit 32 in response to a request from the audio decoder unit 32.

  The audio decoder unit 32 decodes the audio data held in the audio ES buffer 31. At this time, the audio decoder unit 32 uses the TTS clock as a reference clock when the TTS time stamp and the PCR are synchronized in the received stream. The audio decoder unit 32 uses the PCR clock reproduced by the PCR recovery unit 28 as a reference clock when the TTS time stamp and the PCR are not synchronized in the received stream.

  More specifically, the audio decoder unit 32 refers to the PTS included in the PES header of the audio PES included in the payload of the TTS packet held in the audio ES buffer 31.

  When the PTS matches the reference clock, the audio decoder unit 32 outputs the audio information created by decoding the audio data included in the PES corresponding to the PTS to the display device 12, for example.

[Synchronization of local TTS clock and station TTS clock]
By the way, the stream processing apparatus 14 that implements the first technique disclosed in Patent Document 1 causes the TTS packet of the received stream to stay in the buffer before being output to the TTS gate unit. Then, the TTS gate unit outputs a TTS packet including a TTS time stamp that matches the TTS count value from the TTS packets staying in the buffer to the demax unit.

  For example, if the input rate indicating the number of bytes per unit time of the TTS packet input to the buffer and the output rate indicating the number of bytes per unit time of the TTS packet output from the buffer to the demux unit are the same, the buffer stays in the buffer. The actual accumulated amount indicating the number of bytes of the TTS packet to be changed does not change.

  Further, for example, when the input rate is higher than the output rate, the actual accumulation amount increases. For example, when the input rate is smaller than the output rate, the actual accumulation amount decreases.

  The input rate is determined by the station TTS clock in the station apparatus. The output rate is determined by the local TTS clock in the stream processing device 14. Therefore, the stream processing apparatus 14 can determine whether the frequency of the local TTS clock is higher, lower, or appropriate than the frequency of the local TTS clock by monitoring the actual accumulation amount.

  However, in the above operation, when the stream processing apparatus 14 receives a new stream, the magnitude relationship between the input rate and the output rate cannot be determined unless the TTS packet is retained in the buffer to some extent. For this reason, after the first TTS packet is input to the buffer, it is not output from the buffer until a stream stays in the buffer to some extent. Therefore, since the actual accumulated amount is not evaluated until the TTS packet input to the buffer is output from the buffer, a delay time occurs. In particular, when the input rate is small, there arises a problem that the delay time becomes long. For example, the stream processing device 14 cannot move the received packet to the next process until the received packet comes out of the buffer.

  In order to solve the above-described problem, for example, a stream processing device 15 that implements the second technique is conceivable. The stream receiving device 15 acquires the input time indicating the above timing at the timing when the TTS packet of the received stream is input to the buffer. Then, after the stream processing device 15 retains the TTS packet in the buffer, the stream processing device 15 acquires an output time indicating the timing at a timing when the TTS timestamp of the TTS packet matches the TTS count value, Output to the demax section. The stream processing device 15 calculates the difference between the output time and the input time, thereby acquiring the actual residence time indicating the time that the TTS packet actually stayed in the buffer.

  The stream processing device 15 acquires the actual residence time in a plurality of TTS packets, and the frequency of the local TTS clock in the stream processing device 15 is the frequency of the station TTS clock in the station side device based on the acquired tendency of the actual residence time. To determine whether it is high, low or reasonable.

  More specifically, for example, let T be the time indicated by the local TTS clock. The time indicated by the station TTS clock is t. The relationship between T and t can be generally described as T = a × t + b. When a is 1, the local TTS clock and the station-side TTS clock are synchronized. When a> 1, the frequency of the local TTS clock is higher than the frequency of the station-side TTS clock. When a <1, the frequency of the local TTS clock is lower than the frequency of the station side TTS clock.

  For example, the station side device transmits a TTS packet with a TTS time stamp indicating the time (t1 + L) to the stream processing device 15 at time t1. The stream processing device 15 inputs the TTS packet to the buffer at time T1, assuming that the transmission time can be ignored. Time T1 corresponds to time (a × t1 + b). The stream processing device 15 outputs the packet from the buffer to the demux unit at time (t1 + L). Therefore, D1 indicating the actual residence time is ((1−a) × t1 + L−b), which is the difference between the time (t1 + L) and the time (a × t1 + b).

  Further, for example, the station side device transmits a TTS packet with a TTS time stamp indicating the time (t2 + L) to the stream processing device 15 at time t2 after time t1. Assuming that the transmission time can be ignored, the stream processing device 15 inputs the TTS packet to the buffer at time T2. Time T2 corresponds to time (a × t2 + b). The stream processing device 15 outputs the packet from the buffer to the demux unit at time (t2 + L). Therefore, D2 indicating the actual residence time is ((1−a) × t2 + L−b), which is the difference between the time (t2 + L) and the time (a × t2 + b).

  Further, the difference between D2 and D1 indicating the difference in actual residence time is ((1-a) × (t2-t1)). As a precondition, since the time t2 is a time after the time t1, (t2-t1) is positive. For example, when the frequency of the local TTS clock is higher than the frequency of the station-side TTS clock, that is, when a> 1, (1−a) is negative, so the difference in the actual residence time is negative. This indicates that D2 becomes smaller than D1 as time t2 comes after time t1.

  That is, when the frequency of the local TTS clock is higher than the frequency of the station-side TTS clock, the TTS packet transmitted later by the station-side device is more effective than the TTS packet transmitted earlier by the station-side device. Residence time is shortened. Therefore, the stream processing apparatus 15 determines the local TTS clock when the actual residence time of the TTS packet received later is shorter than the actual residence time of the previously received TTS packet, that is, when the actual residence time tends to decrease. It can be determined that the frequency is higher than the frequency of the station-side TTS clock.

  Further, for example, when the frequency of the local TTS clock is lower than the frequency of the station-side TTS clock, that is, when a <1, (1-a) is positive, so the difference in the actual residence time is positive. This indicates that D2 becomes larger than D1 as time t2 comes after time t1.

  That is, when the frequency of the local TTS clock is lower than the frequency of the station-side TTS clock, TTS packets transmitted later by the station-side device are more effective in the buffer than TTS packets transmitted earlier by the station-side device. Residence time is increased. Therefore, the stream processing apparatus 15 determines that the local residence time of the local TTS clock when the actual residence time of the TTS packet received later is longer than the actual residence time of the previously received TTS packet, that is, when the actual residence time tends to increase. It can be determined that the frequency is lower than the frequency of the station-side TTS clock.

  Further, for example, when the frequency of the local TTS clock is the same as the frequency of the station-side TTS clock, that is, when a = 1, (1−a) is 0, so the difference in the actual residence time is 0. This indicates that the difference in the actual residence time does not depend on the time t1 or the time t2.

  That is, when the frequency of the local TTS clock is the same as the frequency of the station-side TTS clock, the actual residence time in the TTS packet buffer does not change. Therefore, the stream processing device 15 can determine that the local TTS clock and the station-side TTS clock are synchronized when the actual residence time does not change.

  By obtaining the actual residence time, the stream processing device 15 can solve the problem that the delay time changes according to the number of bytes per unit time of the TTS packet input to the buffer.

  However, even with the above-described operation, the TTS packet of the received stream remains in the buffer until the TTS time stamp of the TTS packet matches the TTS count value after being input to the buffer. Therefore, there remains a problem that a delay time until the TTS packet input to the buffer is output from the buffer is generated. Further, for example, the stream processing device 15 cannot move the packet to the next processing until the received packet comes out of the buffer. The stream processing apparatus 20 according to the embodiment of the present invention solves the above problem by the following operation.

[Regeneration processing operation of TTS clock based on residence time]
The stream processing apparatus 10 reads out and executes each step of each sequence diagram shown below from a memory (not shown). This program can be installed externally. The installed program is distributed in a state stored in a recording medium, for example.

FIG. 4 is a flowchart that defines an operation procedure when performing the reproduction process of the TTS clock based on the average virtual residence time calculated using the virtual buffer in the stream processing apparatus according to the embodiment of the present invention.
Referring to FIG. 4, first, stream processing apparatus 20 receives a TTS packet in a stream to be received (step S102).

  Next, when receiving the i-th TTS packet, for example, the time stamp acquisition unit 21 in the stream processing device 20 acquires TS [i] indicating the TTS time stamp from the TTS header in the received TTS packet (step S104).

  Next, the time stamp acquisition unit 21 outputs the acquired TS [i] to the virtual buffer calculation unit 23. In addition, the time stamp acquisition unit 21 outputs the received TTS packet to the TTS gate unit 22.

  Next, when the virtual buffer calculation unit 23 receives TS [0], which is the first TTS time stamp after the user selects a program or an error occurs, for example, from the time stamp acquisition unit 21 (YES in step S106), The TTS count value in the TTS counter unit 24 is initialized to TS [0] (step S108).

  Next, upon receiving TS [i] from the time stamp acquisition unit 21, the virtual buffer calculation unit 23 obtains STC [i] corresponding to the TTS count value at the time ti that received TS [i] from the TTS counter unit 24. Obtain (step S110).

  Next, the virtual buffer calculation unit 23 calculates Δt [i] indicating the difference between TS [i] and STC [i] (step S112).

  More specifically, TS [i] is the time at which the i-th TTS packet received by the stream processing device 20 at time ti is output to the demux unit 27 by the TTS gate unit 22. That is, TS [i] can be regarded as the time when the i-th TTS packet leaves the virtual buffer.

  STC [i] indicates a time ti when the i-th TTS packet is received by the stream processing device 20. That is, STC [i] can be regarded as the time when the i-th TTS packet enters the virtual buffer.

  Therefore, Δt [i] indicates the virtual residence time from when the i-th TTS packet enters the virtual buffer until it leaves the virtual buffer.

  Next, when the virtual buffer calculation unit 23 is set by the user to calculate the moving average based on, for example, p Δt closest to the time ti, Δt [i−p + 1] to Δt [i]. AvgΔt [i] indicating the average virtual residence time corresponding to the moving average of the i-th TTS packet is calculated based on Δt until (step S114).

  Next, the virtual buffer computing unit 23 analyzes the trend of AvgΔt from, for example, AvgΔt [i] and AvgΔt [i−1] calculated before time ti (step S116).

  Next, for example, when AvgΔt [i] is larger than AvgΔt [i−1], the virtual buffer calculation unit 23 determines that AvgΔt, which is the average virtual residence time, tends to increase (YES in step S118), and determines the average virtual The TTS clock adjustment unit 25 is notified that the residence time tends to increase. In response to the notification, the TTS clock adjustment unit 25 increases the frequency of the TTS clock generation unit 26 (step S120).

  For example, when AvgΔt [i] is smaller than AvgΔt [i−1], the virtual buffer calculation unit 23 determines that AvgΔt, which is the average virtual residence time, is not increasing (NO in step S118) and is decreasing. (YES in step S122), the TTS clock adjustment unit 25 is notified that the average virtual residence time tends to decrease. In response to the notification, the TTS clock adjustment unit 25 decreases the frequency of the TTS clock generation unit 26 (step S124).

  For example, when AvgΔt [i] is the same as AvgΔt [i−1], the virtual buffer calculation unit 23 determines that the average virtual residence time AvgΔt is neither increasing nor decreasing (NO in step S122). ), A notification for adjusting the frequency of the TTS clock generator 26 is not output to the TTS clock adjuster 25.

  With the above operation, the stream processing device 20 performs the reproduction process of the TTS clock based on the average virtual residence time calculated using the virtual buffer.

  In the second technique, the actual residence time is calculated at the timing when the TTS packet leaves the buffer. On the other hand, the virtual buffer calculation unit 23 can calculate the virtual residence time equivalent to the actual residence time at time ti indicating the time when the i-th TTS packet enters the virtual buffer in step S112.

  That is, the virtual buffer calculation unit 23 does not need to wait until TS [i], which is the time when the i-th TTS packet leaves the virtual buffer, in order to calculate the virtual residence time. Therefore, the stream processing device 20 can reproduce the TTS clock based on the virtual residence time calculated at the time ti indicating the time when the i-th TTS packet enters the virtual buffer. Further, the stream processing device 20 can move the received packet to the next processing without staying in the buffer.

  Moreover, although the virtual buffer calculating part 23 calculated the moving average in said step S114, it is not limited to this. The virtual buffer calculation unit 23 may analyze the tendency of the virtual residence time based on the raw virtual residence time.

  For example, in a situation in which the stream processing device 20 can regularly receive TTS packets transmitted from the station-side device 10 at regular time intervals, at regular time intervals, the virtual buffer calculation unit 23 sets Δt [i] to Δt When larger than [i-1], it can be determined that the virtual residence time tends to increase, and when Δt [i] is smaller than Δt [i-1], it can be determined that the virtual residence time tends to decrease. . In addition, when Δt [i] is approximately equal to Δt [i−1], the virtual buffer calculation unit 23 can determine that the virtual residence time is neither increasing nor decreasing. That is, the stream processing device 20 can synchronize the local TTS clock with the station-side TTS clock by adjusting the frequency of the TTS clock generation unit 26 so that Δt becomes a predetermined value.

  When the station side device 10 retransmits the MPEG2-TS and creates the MPEG2-TTS by deleting the null packet or the like, the MPEG2-TTS is as shown in FIG. It includes TTS packets that are discontinuous in sequence. Further, since the transmission time of the TTS packet varies depending on the TTS packet, the time interval at which the stream processing apparatus 20 receives each TTS packet is not constant. At this time, Δt calculated by the virtual buffer calculation unit 23 in step S112 varies for each received TTS packet.

  When the variation in Δt increases, the frequency of the TTS clock fluctuates. This fluctuation in the frequency of the TTS clock causes an even larger variation in Δt, which may amplify the fluctuation in the frequency of the TTS clock.

  Even in this case, the virtual buffer calculation unit 23 obtains AvgΔt obtained by smoothing Δt by calculating a moving average of Δt that varies in step S114. As a result, AvgΔt can be regarded as Δt that has passed through the low-pass filter, so that fluctuations in the frequency of the TTS clock due to excessive feedback processing can be suppressed.

  Therefore, when determining the tendency of the residence time based on AvgΔt in step S116, the virtual buffer calculation unit 23 makes an appropriate determination even when the time at which the stream processing device 20 receives the TTS packet is irregular. It can be carried out.

  Further, the above operations from step S102 to step S124 need not be performed for all packets received by the stream processing device 20. For example, the stream processing device 20 may perform the above operation every predetermined number of packets.

[Calculation processing operation of virtual accumulation amount in virtual buffer]
In the reproduction process of the TTS clock shown in FIG. 4, the virtual residence time calculated using the virtual buffer is used. However, the reproduction process of the TTS clock may be performed according to the virtual accumulation amount calculated using the virtual buffer. Hereinafter, an operation procedure when performing a reproduction process of the TTS clock according to the average virtual accumulation amount calculated using the virtual buffer will be described.

FIG. 5 is a flowchart that defines an operation procedure when performing calculation processing of the virtual accumulation amount in the virtual buffer in the stream processing apparatus according to the embodiment of the present invention.
Referring to FIG. 5, first, stream processing apparatus 20 receives a TTS packet in a stream to be received (step S202).

  Next, when receiving the i-th TTS packet, for example, the time stamp acquisition unit 21 in the stream processing device 20 acquires TS [i] indicating the TTS time stamp from the TTS header in the received TTS packet (step S204).

  Next, the time stamp acquisition unit 21 outputs the acquired TS [i] to the virtual buffer calculation unit 23. In addition, the time stamp acquisition unit 21 outputs the received TTS packet to the TTS gate unit 22.

  Next, when the virtual buffer calculation unit 23 receives TS [0], which is the first TTS time stamp, from the time stamp acquisition unit 21 after the user selects a program or an error occurs, for example (YES in step S206), The TTS count value in the TTS counter unit 24 is initialized to TS [0] (step S208). Further, the virtual buffer calculation unit 23 sets, for example, 0 as an initial value in Sn indicating the virtual accumulation amount in the virtual buffer.

  Next, upon receiving TS [i] from the time stamp acquisition unit 21, the virtual buffer calculation unit 23 adds 192, which is the size of the TTS packet, to Sn, which is the virtual accumulation amount (step S210).

  Next, the virtual buffer calculation unit 23 monitors the TTS count value until TS [i] indicating the time at which the i-th TTS packet leaves the virtual buffer (NO in step S212), and TS [i] is the TTS count. If the values match (YES in step S212), 192, which is the size of the TTS packet, is subtracted from Sn, which is the virtual accumulation amount (step S214).

  Through the above operation, the stream processing apparatus 20 performs the virtual accumulation amount calculation process in the virtual buffer.

  In the first technique, a stream is accumulated in a buffer until a predetermined level is reached, and the number of bytes of the stream accumulated in the buffer, that is, a change in the accumulation amount is monitored. At this time, the stream processing apparatus cannot start the reproduction process of the TTS clock until the accumulated amount of the stream reaches a predetermined level. Further, since the stream processing apparatus cannot move the stream in the buffer to the next processing, a delay time occurs.

  On the other hand, the virtual buffer calculation unit 23 in the stream processing device 20 adds 192 to Sn which is the virtual accumulation amount at the time when the TTS packet enters the virtual buffer in step S210. In step S212, the virtual buffer calculation unit 23 subtracts 192 from Sn, which is the virtual accumulation amount, at the time when the TTS packet leaves the virtual buffer. That is, the capacity of the TTS packet is included in the virtual storage amount Sn in the virtual buffer only during the time when the TTS packet enters and exits the virtual buffer.

  With the above operation, the processing for accumulating the TTS packet in the virtual buffer can be completed only by performing the processing at the time ti indicating the time when the i-th TTS packet enters the virtual buffer.

  Thereby, the TTS packet can be accumulated in the virtual buffer from the time when the TTS packet enters the virtual buffer to the time when the virtual buffer exits.

  In addition, the virtual buffer calculation unit 23 does not need to accumulate streams until the virtual buffer reaches a predetermined level in order to calculate the virtual accumulation amount. Therefore, the stream processing apparatus 20 can move to the next process without accumulating the stream in the actual buffer.

  Further, the operations from step S202 to step S214 need not be performed for all packets received by the stream processing device 20. For example, the stream processing device 20 may perform the above operation every predetermined number of packets.

[Reproduction processing operation of TTS clock based on accumulated amount]
FIG. 6 is a flowchart that defines an operation procedure when performing the reproduction process of the TTS clock based on the average virtual accumulation amount calculated using the virtual buffer in the stream processing apparatus according to the embodiment of the present invention.

  The stream processing apparatus 20 performs the following operation when performing TTS clock regeneration processing according to the average virtual accumulation amount calculated using the virtual buffer, and the virtual accumulation amount from step S210 to step S216 shown in FIG. You may perform in parallel with the operation | movement for calculating.

  Referring to FIG. 6, the stream processing device 20 starts the operation by receiving the first TTS packet from the station side device 10 (step S308).

  Next, for example, when the time interval for evaluating the virtual accumulation amount Sn is set by the user, the virtual buffer calculation unit 23 monitors the TTS count value in the TTS counter unit 24 (NO in step S310). When the TTS count value for a predetermined time corresponding to the time interval increases in the TTS count value (YES in step S310), Sn [j] that is a virtual accumulation amount at time tj when the predetermined time has elapsed is acquired (step S312). ).

  Next, when the virtual buffer calculation unit 23 is set by the user to calculate a moving average based on, for example, r Sn nearest to time tj, Sn [j−r + 1] to Sn [j] AvgSn [j], which is the average virtual accumulation amount corresponding to the moving average of the jth Sn, is calculated based on the previous Sn (step S314).

  Next, the virtual buffer calculation unit 23 analyzes the trend of AvgSn from, for example, AvgSn [j] and AvgSn [j−1] calculated before time tj (step S316).

  Next, for example, when AvgSn [i] is larger than AvgSn [i−1], the virtual buffer calculation unit 23 determines that the average virtual accumulation amount AvgSn tends to increase (YES in Step S318). The TTS clock adjustment unit 25 is notified that the accumulation amount tends to increase. In response to the notification, the TTS clock adjustment unit 25 increases the frequency of the TTS clock generation unit 26 (step S320).

  For example, when AvgSn [i] is smaller than AvgSn [i−1], the virtual buffer calculation unit 23 determines that the average virtual accumulation amount AvgSn is not increasing (NO in step S318) and is decreasing. (YES in step S322), the TTS clock adjustment unit 25 is notified that the average virtual accumulation amount tends to decrease. The TTS clock adjustment unit 25 lowers the frequency of the TTS clock generation unit 26 in response to the notification (step S324).

  Further, for example, when AvgSn [i] is the same as AvgSn [i−1], the virtual buffer calculation unit 23 determines that the average virtual accumulation amount AvgSn is neither increasing nor decreasing (NO in Step S322). ), A notification for adjusting the frequency of the TTS clock generator 26 is not output to the TTS clock adjuster 25.

  Through the above operation, the stream processing apparatus 20 performs the reproduction process of the TTS clock based on the average virtual accumulation amount calculated using the virtual buffer.

  Next, while receiving the stream (NO in step S326), the virtual buffer calculation unit 23 returns to step S310 and acquires Sn at every predetermined time. In addition, when the reception of the stream ends (YES in step S326), the virtual buffer calculation unit 23 ends the above operation.

  The virtual buffer calculation unit 23 evaluates Sn, which is a virtual accumulation amount, at the time interval set by the user in step S310, but is not limited thereto. For example, the virtual buffer calculation unit 23 may evaluate the virtual storage amount Sn in synchronization with the timing of receiving the TTS time stamp from the time stamp acquisition unit 21. However, since the virtual accumulation amount Sn changes with time, it is preferable to obtain Sn at predetermined time intervals. As a result, sampling bias can be prevented.

  In addition, the virtual buffer calculation unit 23 calculates the moving average in step S314 described above, but the present invention is not limited to this, and the trend of the virtual accumulation amount may be analyzed based on the raw virtual accumulation amount. .

  For example, in a situation where the stream processing apparatus 20 can regularly receive TTS packets transmitted from the station-side apparatus 10 regularly at a constant time interval, the virtual buffer calculation unit 23 can set Sn [j] to Sn When it is larger than [j−1], it can be determined that the virtual accumulation amount tends to increase, and when Sn [j] is smaller than Sn [j−1], it can be determined that the virtual accumulation amount tends to decrease. . Further, when Sn [j] is approximately the same as Sn [j−1], the virtual buffer calculator 23 can determine that the virtual accumulation amount is neither increasing nor decreasing. That is, the stream processing device 20 can synchronize the local TTS clock with the station-side TTS clock by adjusting the frequency of the TTS clock generation unit 26 so that Sn becomes a predetermined value.

  When the station side device 10 retransmits the MPEG2-TS and creates the MPEG2-TTS by deleting the null packet or the like, the MPEG2-TTS is as shown in FIG. It includes TTS packets that are discontinuous in sequence. Further, since the transmission time of the TTS packet varies depending on the TTS packet, the time interval at which the stream processing apparatus 20 receives each TTS packet is not constant. At this time, the Sn acquired by the virtual buffer calculation unit 23 in step S312 greatly varies depending on the acquisition timing.

  When the variation in Sn increases, the frequency of the TTS clock fluctuates. This fluctuation in the frequency of the TTS clock causes a larger variation in Sn, which may amplify the fluctuation in the frequency of the TTS clock.

  Also in this case, the virtual buffer calculation unit 23 obtains AvgSn obtained by smoothing Sn by calculating a moving average of Sn that has been varied in the above-described step S314. As a result, AvgSn can be regarded as Sn that has passed through the low-pass filter, so that fluctuations in the frequency of the TTS clock due to excessive feedback processing can be suppressed.

  Accordingly, when the virtual buffer calculation unit 23 determines the tendency of the virtual accumulation amount based on AvgSn in step S316, even if the time at which the stream processing device 20 receives the TTS packet is irregular, an appropriate determination is made. It can be performed.

[Processing of received TTS packet]
FIG. 7 is a flowchart defining an operation procedure for processing a received TTS packet in the stream processing apparatus according to the embodiment of the present invention.

  The time stamp acquisition unit 21 acquires the TTS time stamp from the i-th TTS packet in step S104 shown in FIG. 4, step S204 shown in FIG. 5, and step S304 shown in FIG. Output to.

  The stream processing apparatus 20 performs the following operations after the TTS packet is output to the TTS gate unit 22, the operations from step S110 to step S124 shown in FIG. 4, and the steps S210 to S216 shown in FIG. The operations up to or the operations from step S310 to step S324 shown in FIG. 6 may be performed in parallel.

  Referring to FIG. 7, the stream processing apparatus 20 performs the operation after the operation of receiving the TTS packet in the received stream (step S402 to step S408), the operation in the flowchart shown in FIG. 4 (step S102 to step S108), and Since this is the same as the operation in the flowchart shown in FIG. 5 (steps S202 to S208), detailed description will not be repeated here.

  Next, when receiving the i-th TTS packet from the time stamp acquisition unit 21, the TTS gate unit 22 checks whether the TTS time stamp and the PCR in the received stream are synchronized (step S410).

  Next, when the TTS time stamp and the PCR in the received stream are synchronized (YES in step S410), the TTS gate unit 22 does not perform the gate operation because the TTS clock can be used instead of the PCR clock. That is, the TTS gate unit 22 creates the i-th TS packet by removing the TTS header from the i-th TTS packet received from the time stamp acquisition unit 21, and outputs the created i-th TS packet to the demax unit 27. To do.

  Next, upon receiving the i-th TS packet from the TTS gate unit 22, the demax unit 27 demultiplexes the received TS packet into a video TS packet, an audio TS packet, a PCR packet, or the like, and the demultiplexed TS packet is converted into a TS. The header and the payload are separated (step S412).

  Next, for example, when the i-th TS packet is a video TS packet, the demax unit 27 outputs a payload including video data to the video ES buffer. For example, when the i-th TS packet is an audio TS packet, the demax unit 27 outputs a payload including audio data to the audio ES buffer 31.

  Further, when the i-th TS packet is a PCR packet including the PCR, the demax unit 27 uses the TTS clock instead of the PCR clock, so that the PCR included in the adaptation field does not have to be output to the PCR recovery unit 28. Also good.

  Next, the video decoder unit 30 performs video data decoding processing according to the timing of the reference clock in the TTS clock generation unit 26 (step S414). Next, the video decoder unit 30 outputs the program image created by the decoding process to, for example, the display device 12.

  In addition, the audio decoder unit 30 performs audio data decoding processing in accordance with the timing of the reference clock in the TTS clock generation unit 26 (step S416). Next, the audio decoder unit 30 outputs the audio information created by the decoding process to the display device 12, for example.

  In addition, when the TTS time stamp and the PCR in the received stream are not synchronized (NO in step S410), the TTS gate unit 22 performs a gate operation to reproduce the local PCR clock (step S418).

  That is, when the TTS count value received from the TTS counter unit 24 matches TS [i] corresponding to the i-th TTS packet received from the time stamp acquisition unit 21, the TTS gate unit 22 creates the TTS header by removing the TTS header. The i th TS packet is output to the demux unit 27.

  Next, upon receiving the i-th TS packet from the TTS gate unit 22, the demax unit 27 demultiplexes the received TS packet into a video TS packet, an audio TS packet, a PCR packet, or the like, and the demultiplexed TS packet is converted into a TS. Separated into a header and a payload (step S420).

  Next, for example, when the i-th TS packet is a video TS packet, the demax unit 27 outputs a payload including video data to the video ES buffer. For example, when the i-th TS packet is an audio TS packet, the demax unit 27 outputs a payload including audio data to the audio ES buffer.

  Further, when the i-th TS packet is a PCR packet including PCR, the demax unit 27 analyzes the PCR from the adaptation field and outputs the analyzed PCR to the PCR recovery unit 28.

  Next, when receiving the PCR from the demax unit 27, the PCR recovery unit 28 regenerates the PCR clock based on the received PCR (step S422). Specifically, when the received PCR is ahead of the local PCR clock, the PCR recovery unit 28 increases the frequency of the local PCR clock. The PCR recovery unit 28 lowers the frequency of the local PCR clock when the received PCR is behind the local PCR clock.

  As a result, the stream processing apparatus 30 can process the received stream in synchronization with the local PCR clock synchronized with the station-side PCR clock.

  In addition, the video decoder unit 30 performs video data decoding processing according to the local PCR clock timing (step S424). Next, the video decoder unit 30 outputs the program image created by the decoding process to, for example, the display device 12.

  Also, the audio decoder unit 30 performs audio data decoding processing according to the local PCR clock timing (step S426). Next, the audio decoder unit 30 outputs the audio information created by the decoding process to the display device 12, for example.

  Through the above operation, the stream processing device 20 processes the TTS packet received by the stream processing device.

  Information included in the payload of the video TS packet or the audio TS packet stays in the video ES buffer 29 or the audio ES buffer 31. For this reason, the video TS packet and the audio TS packet may be output from the TTS gate unit 22 to the demax unit 27 at any timing.

  On the other hand, since the PCR packet is used to synchronize the local PCR clock with the local PCR clock, it is necessary to output it to the PCR recovery unit 28 at a predetermined timing.

  Therefore, in step S418, the TTS gate unit 22 may perform the gate operation for only the PCR packet, and output TS packets other than the PCR packet to the demux unit 27 as they are.

  Further, the TTS gate unit 22 performs a gate operation using a local TTS clock synchronized with the station-side TTS clock in order to regenerate the local PCR clock. That is, the TTS gate unit 22 outputs the PCR packet to the demax unit 27 at a timing corresponding to the TTS time stamp included in the PCR packet based on the local TTS clock synchronized with the station-side TTS clock. As a result, the PCR recovery unit 28 receives the PCR packet from the TTS gate unit 22 via the demax unit 27 at the relative decode timing.

  In this way, by reproducing the local PCR clock based on the PCR received by the PCR recovery unit 28 at the relative decode timing, the stream processing device 20 can synchronize the local PCR clock with the local PCR clock.

  If the station TTS clock and the station PCR clock are synchronized, the local TTS clock is synchronized with the station TTS clock by the local TTS clock regeneration process shown in FIG. 4 or FIG. Automatically synchronizes with the station side PCR clock. Therefore, the stream processing device 20 does not need to synchronize the local PCR clock with the station side PCR using PCR, and can use the local TTS clock instead of the local PCR clock.

  As a result, when the TTS time stamp and the PCR are synchronized in the received stream, the stream processing device 20 can use the TTS clock as the reference clock when performing the processing after the demax unit 27.

  Further, when the TTS time stamp and the PCR are synchronized in the received stream, it is not necessary to regenerate the PCR clock, so that it is not necessary to input the PCR packet to the PCR recovery unit 28 at the relative decoding timing. Therefore, the TTS gate unit 22 does not need to perform the gate operation, and the TTS packet of the received stream is transferred from the time stamp acquisition unit 21 to the processing after the demax unit 27 without staying.

  That is, the stream processing apparatus 20 can move the TTS packet of the received stream to the processing after the demax unit 27 without a delay time.

  By the way, in order to adjust the frequency of the reference clock, it is necessary to store packets in a buffer up to a predetermined specified range. For example, the received packets are sequentially accumulated in the buffer, and after the packets are accumulated to a predetermined specified range, the first received packet is taken out from the buffer and moved to the next processing. Therefore, since a residence time from when the packet enters the buffer until it leaves the buffer, that is, a delay time occurs, even if the packet is received, the packet cannot be processed until the delay time elapses.

  On the other hand, in the stream processing apparatus according to the embodiment of the present invention, the TTS packet includes a TTS header, a TS header, and a payload, at least a part of the TS headers of the plurality of TTS packets includes a PCR, and the plurality of TTSs Each TTS header of the packet includes a TTS time stamp, a TTS clock generation unit 26 for generating a local TTS clock, a TTS counter unit 24 for updating a TTS count value based on the local TTS clock, and a TTS count A virtual buffer operation unit 23 and a TTS clock adjustment unit 25 for adjusting the frequency of the local TTS clock by controlling the TTS clock generation unit 26 based on the value and the TTS time stamp are provided.

  Thus, since the TTS count value and the TTS timestamp value can be obtained from the time when the stream processing device 20 receives the TTS packet, the process for controlling the TTS clock generation unit 26 and the process for decoding are performed at the time. And the buffer in the TTS gate unit 22 becomes unnecessary, and the video decoding process can be started immediately. Therefore, it is possible to suppress an increase in delay time from the time when the stream processing device 20 receives the TTS packet until the TTS packet is decoded, for example.

  Further, by using the TTS count value based on the local TTS clock and the TTS time stamp value based on the TTS clock in the station side device 10 such as a VOD server, the local TTS clock is converted into the TTS clock in the station side device 10. Can be synchronized.

  Thus, since the TTS count value is synchronized with the TTS clock in the station-side device 10, the stream processing device 20 is created based on the TTS clock in the station-side device 10 by performing processing based on the local TTS clock. Thus, each packet of the stream can be processed with the same timing relationship as the processing timing of each packet of the stream in the station-side device 10.

  In the stream processing device according to the embodiment of the present invention, the virtual buffer calculation unit 23 and the TTS clock adjustment unit 25 evaluate the virtual buffer evaluation value for the TTS packet based on the TTS count value and the TTS time stamp. And the frequency of the local TTS clock is adjusted by controlling the TTS clock generation unit 26 according to the calculated evaluation value.

  With such a configuration, an evaluation value of a virtual buffer can be calculated without actually providing a buffer, so that the local TTS clock can be appropriately controlled.

  Since the stream processing apparatus 20 does not mount a buffer, the received TTS packet can be moved to the next process without being retained in the buffer.

  In the stream processing apparatus according to the embodiment of the present invention, the virtual buffer calculation unit 23 and the TTS clock adjustment unit 25 adjust the frequency of the local TTS clock based on the difference between the TTS count value and the TTS time stamp.

  Thus, by calculating the difference between the TTS count value based on the TTS clock and the TTS timestamp value based on the TTS clock in the station side device 10, the operation timing in the stream processing device 20 and the station side device 10 Since it is possible to analyze the change over time in the shift in the operation timing, the local TTS clock can be adjusted accurately.

  Further, since the TTS count value and the TTS time stamp value necessary for calculating the difference are acquired at the time when the TTS packet is received, the process for calculating the difference can be started from the time. . Therefore, the stream processing device 20 can reflect the difference in the adjustment of the frequency of the TTS clock at the time when the TTS packet is received.

  In the stream processing apparatus according to the embodiment of the present invention, the virtual buffer calculation unit 23 and the TTS clock adjustment unit 25 select one or more TTS packets based on the magnitude relationship between the TTS count value and the TTS time stamp. The frequency of the local TTS clock is adjusted based on the total data amount of the selected TTS packet.

  With such a configuration, it is possible to virtually determine the amount of TTS packets stored in the buffer. Therefore, by monitoring the amount of storage over time, the operation timing in the stream processing device 20 and the operation in the station-side device 10 are monitored. It is possible to analyze a change in timing with time. As a result, the local TTS clock can be adjusted accurately.

  In the stream processing apparatus according to the embodiment of the present invention, the virtual buffer calculation unit 23 and the TTS clock adjustment unit 25 perform the frequency of the local TTS clock so that the difference between the TTS count value and the TTS time stamp becomes a predetermined value. Adjust.

  When the local TTS clock and the TTS clock in the station side device 10 are synchronized, the difference between the TTS count value based on the local TTS clock and the TTS time stamp value based on the TTS clock in the station side device 10 becomes constant. By adjusting the frequency of the local TTS clock so that the difference becomes constant, the local TTS clock can be synchronized with the TTS clock in the station side device 10.

  Further, in the stream processing device according to the embodiment of the present invention, the virtual buffer calculation unit 23 and the TTS clock adjustment unit 25 are based on the difference between the TTS time stamps of a part of the plurality of TTS packets and the TTS count value. Adjust the frequency of the local TTS clock.

  With such a configuration, not all TTS packets received by the stream processing device 20, but a TTS count value based on the local TTS clock from some TTS packets, and a TTS time stamp based on the TTS clock in the station side device 10 Since the difference in values is calculated, the calculation burden on the stream processing apparatus 20 can be reduced.

  Further, in the stream processing apparatus according to the embodiment of the present invention, the virtual buffer calculation unit 23 and the TTS clock adjustment unit 25 determine the frequency of the local TTS clock so that the total data amount of the selected TTS packet becomes a predetermined value. Adjust.

  Thus, in the TTS packet received by the stream processing device 20, the number of bytes of the TTS packet is totaled from the timing when the TTS packet is received until the timing when the TTS count value becomes the value indicated by the TTS timestamp. Include in When the local TTS clock and the TTS clock in the station-side device 10 are synchronized, the total is constant. Therefore, the local TTS clock is adjusted by adjusting the frequency of the local TTS clock so that the total is constant. It can be synchronized with the TTS clock in the side device 10.

  Further, in the stream processing apparatus according to the embodiment of the present invention, the virtual buffer calculation unit 23 and the TTS clock adjustment unit 25 calculate the difference between the TTS count value and the TTS time stamp, and the virtual residence time of the TTS packet in the virtual buffer. And the frequency of the local TTS clock is adjusted based on the calculated virtual residence time.

  With such a configuration, the virtual dwell time can be calculated as a virtual buffer evaluation value, so the local TTS clock can be accurately adjusted.

  In the stream processing apparatus according to the embodiment of the present invention, the virtual buffer calculation unit 23 and the TTS clock adjustment unit 25 calculate the total data amount of one or a plurality of TTS packets whose TTS time stamp is equal to or less than the TTS count value. The virtual accumulation amount of the TTS packet in the virtual buffer is calculated, and the frequency of the local TTS clock is adjusted based on the calculated virtual accumulation amount.

  With such a configuration, since the virtual accumulation amount can be calculated as the evaluation value of the virtual buffer, the local TTS clock can be adjusted accurately.

  In the stream processing apparatus according to the embodiment of the present invention, the stream processing apparatus 20 further includes a video decoder unit 30 and an audio decoder unit 32 for performing decoding processing on the payload of the TTS packet. When the PCR and the TTS time stamp are synchronized, the unit 30 and the audio decoder unit 32 perform a decoding process according to the timing of the local TTS clock.

  As described above, when the PCR and TTS time stamps are synchronized, the stream processing apparatus 20 does not need to perform a gate operation on the received TTS packet to reproduce the PCR clock, so that it is promptly decoded into the TSS packet. Processing can be performed.

  In addition, since the operation for reproducing the PCR clock is unnecessary, the processing burden on the stream processing apparatus 20 can be reduced, and the TTS gate unit 22 including a buffer and the like is also unnecessary.

  The TTS counter unit 24 increments the TTS count value every time a clock is received from the TTS clock generation unit 26, but the present invention is not limited to this. The TTS counter unit 24 may decrement the TTS count value each time a clock is received from the TTS clock generation unit 26, for example. Therefore, the TTS counter unit 24 decreases the count value in synchronization with the TTS clock as time elapses. Also by this, the count value can be used as the current time, and the time-series context can be obtained from the magnitude relationship of the count value.

  Moreover, although the stream processing apparatus according to the embodiment of the present invention is configured to perform stream processing in accordance with the MPEG2-TS standard, the present invention is not limited to this. It may be configured to process a stream according to another standard.

  The above embodiment should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

DESCRIPTION OF SYMBOLS 10 Station side apparatus 11 Network 12 Display apparatus 13 Stream processing system 20 Stream processing apparatus 21 Time stamp acquisition part 22 TTS gate part 23 Virtual buffer calculating part (clock adjustment part)
24 TTS counter section 25 TTS clock adjustment section 26 TTS clock generation section 27 Demax section 28 PCR recovery section 29 Video ES buffer 30 Video decoder section (payload processing section)
31 Audio ES buffer 32 Audio decoder part (payload processing part)

Claims (14)

A stream processing apparatus for processing a stream obtained by dividing information of one or a plurality of programs into a plurality of packets,
The packet includes a header and a payload; at least some headers of the plurality of packets include clock recovery information; each header of the plurality of packets includes time information;
A clock generator for generating a reference clock;
A counter for updating the count value based on the reference clock;
By controlling the clock generator based on the count value and the time information, Bei example a clock adjusting portion for adjusting the frequency of the reference clock,
The stream processing device further includes:
A payload processing unit for performing predetermined processing on the payload of the packet;
The payload processing unit performs the predetermined processing according to the timing of the reference clock when the clock reproduction information and the time information are synchronized . A stream processing apparatus for processing a stream obtained by dividing information of one or a plurality of programs into a plurality of packets,
The packet includes a header and a payload; at least some headers of the plurality of packets include clock recovery information; each header of the plurality of packets includes time information;
A clock generator for generating a reference clock;
A counter for updating the count value based on the reference clock;
By controlling the clock generator based on the count value and the time information, Bei example a clock adjusting portion for adjusting the frequency of the reference clock,
The clock adjustment unit calculates an evaluation value of the virtual buffer for the packet based on the count value and the time information, and controls the clock generation unit according to the calculated evaluation value, A stream processing apparatus for adjusting a frequency of the reference clock .   The clock adjustment unit calculates an evaluation value of the virtual buffer for the packet based on the count value and the time information, and controls the clock generation unit according to the calculated evaluation value, The stream processing apparatus according to claim 1, wherein the frequency of the reference clock is adjusted. Said clock adjustment unit, based on the difference between the count value and the time information to adjust the frequency of the reference clock, the stream processing apparatus according to any one of claims 1 to 3. The clock adjustment unit selects one or a plurality of the packets based on the magnitude relationship between the count value and the time information, and adjusts the frequency of the reference clock based on the total data amount of the selected packets. The stream processing apparatus according to any one of claims 1 to 3. The stream processing apparatus according to claim 4 , wherein the clock adjustment unit adjusts the frequency of the reference clock so that the difference becomes a predetermined value. Said clock adjustment unit, based on a difference between the count value and a portion of the time information of the plurality of packets, to adjust the frequency of the reference clock, the stream processing apparatus according to claim 4 or claim 6 . The stream processing apparatus according to claim 5 , wherein the clock adjustment unit adjusts a frequency of the reference clock so that the total becomes a predetermined value. The clock adjustment unit calculates a difference between the count value and the time information as a retention time of the packet in a virtual buffer, and adjusts a frequency of the reference clock based on the calculated retention time. 4. The stream processing apparatus according to claim 6 or claim 7 . The clock adjustment unit calculates a sum of data amounts of one or a plurality of the packets whose time information is equal to or less than the count value as an accumulation amount of the packets in a virtual buffer, and based on the calculated accumulation amount adjusting the frequency of the reference clock Te, stream processing apparatus according to claim 5 or claim 8. A stream processing method in a stream processing apparatus for processing a stream obtained by dividing information of one or more programs into a plurality of packets,
The packet includes a header and a payload; at least some headers of the plurality of packets include clock recovery information; each header of the plurality of packets includes time information;
Generating a reference clock; and
Updating the count value based on the reference clock;
Based on the count value and the time information, viewing including the step of adjusting the frequency of the reference clock,
The stream processing method further includes:
Performing predetermined processing on the payload of the packet,
In the step of performing the predetermined processing, when the clock reproduction information and the time information are synchronized, the predetermined processing is performed according to the timing of the reference clock . A stream processing method in a stream processing apparatus for processing a stream obtained by dividing information of one or more programs into a plurality of packets,
The packet includes a header and a payload; at least some headers of the plurality of packets include clock recovery information; each header of the plurality of packets includes time information;
Generating a reference clock; and
Updating the count value based on the reference clock;
Based on the count value and the time information, viewing including the step of adjusting the frequency of the reference clock,
In the step of adjusting the frequency, an evaluation value of the virtual buffer for the packet is calculated based on the count value and the time information, and the frequency of the reference clock is adjusted according to the calculated evaluation value to, stream processing method. A stream processing program used in a stream processing apparatus for processing a stream obtained by dividing information of one or more programs into a plurality of packets,
The packet includes a header and a payload; at least some headers of the plurality of packets include clock recovery information; each header of the plurality of packets includes time information;
On the computer,
Generating a reference clock; and
Updating the count value based on the reference clock;
Adjusting the frequency of the reference clock based on the count value and the time information ;
A program for executing a predetermined process on the payload of the packet,
A stream processing program that, in the step of performing the predetermined process, performs the predetermined process according to the timing of the reference clock when the clock reproduction information and the time information are synchronized . A stream processing program used in a stream processing apparatus for processing a stream obtained by dividing information of one or more programs into a plurality of packets,
The packet includes a header and a payload; at least some headers of the plurality of packets include clock recovery information; each header of the plurality of packets includes time information;
On the computer,
Generating a reference clock; and
Updating the count value based on the reference clock;
Adjusting the frequency of the reference clock based on the count value and the time information ,
In the step of adjusting the frequency, an evaluation value of the virtual buffer for the packet is calculated based on the count value and the time information, and the frequency of the reference clock is adjusted according to the calculated evaluation value A stream processing program.

Images