JP2011077678A - Data stream processor, video device, and data stream processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To determine presentation timing of data broadcast and captions even when no PCR is obtained from one or more decoders for AV information when the decoders are used. <P>SOLUTION: This data stream processor is used together with the one or more decoders 221, 222, 223, ..., 22n which can decode the AV information from a data stream DS including the AV information, the PCR, the data broadcast etc. The data stream processor is provided with: a PCR information separation module 12 which takes out the PCR from the data stream DS irrespective of operations of the one or more decoders; and a service information module 16 which takes out the content of the data broadcast etc. from the data stream DS. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a processing device, a video device, and a processing method for a data stream including reference clock information and the like.

  One of the terrestrial digital broadcasting or BS / CS digital broadcasting services in Japan is a data broadcasting and caption service. These services use reference clock information called PCR (Program Clock Reference) included in the stream to determine the presentation timing. This PCR is also used to determine the timing for decoding AV information such as video and audio. For this reason, usually, an AV information decoder is operated, and at the time of decoding AV information, PCR information is provided from the decoder at the same time for presentation of data broadcasting, captions, and the like (see Patent Document 1).

JP 2007-81475 A

  On the other hand, digital broadcasting has entered the multi-channel era, and more and more devices include a plurality of decoders. In these devices, a decoder specialized in the AV information decoding function that does not include the data broadcasting and caption functions as described above is desired from the viewpoint of rationalization of the entire system and product cost.

  When trying to build a system with only a decoder specialized in the decoding function of AV information, a dedicated module for data broadcasting and subtitles is installed and used in combination with the decoder output. In such a case, PCR information cannot be acquired, and there arises a problem that the presentation timing of data broadcasting, captions, etc. cannot be accurately determined for the decoded AV information.

  One of the problems of the present invention is that when one or more decoders specialized for AV information decoding functions are used, the presentation timing of data broadcasts, subtitles, etc. can be determined without obtaining PCR information from the decoders. Is to do so.

A data stream processing apparatus according to an embodiment of the present invention includes a data stream (DS: digital broadcasting) including video information and / or audio information (AV information), reference clock information (PCR), and service information (data broadcasting or the like). A device (12, 16, 30, etc.) used in combination with one or more decoders (22) capable of decoding the video information and / or the audio information from the MPEG-2 stream. (To connect this device to the decoder, wiring inside the device or a digital interface such as HDMI is used.)
The data stream processing device includes a reference clock information separation module (12) that extracts the reference clock information (PCR) from the data stream regardless of the operation of the one or more decoders (without performing simultaneous decoding); A service information module (16) is provided for extracting the contents of the service information (data broadcast text, video, audio; subtitle broadcast subtitles, etc.) from the data stream.

  Then, the data stream processing device is configured to perform timing of the video information and / or the audio information decoded by the one or more decoders (22) based on the extracted reference clock information (PCR or PCR-S). And the timing of the contents of the service information retrieved by the service information module (16) (PCR can be used for rough timing alignment, and PCR-S can be used for precise timing alignment). .

  According to the present invention, when one or more decoders specialized for the AV information decoding function are used, the presentation timing of data broadcasting, subtitles, etc. can be determined without obtaining PCR information from the decoder.

The figure explaining the structural example of the data stream processing apparatus which concerns on one embodiment of this invention. The flowchart figure explaining the operation example of the data stream processing apparatus which concerns on one embodiment of this invention.

  Hereinafter, various embodiments of the present invention will be described with reference to the drawings. Note that the term “service information” used in the following description is used in a general sense and is not limited to program arrangement information (electronic program guide information) used as service information of MPEG-2 TS. The “service information” used here includes various information. Specifically, data broadcasting including text data / still image data groups (text, logos, menus, etc.) and BML (Broadcast Markup Language) documents that control the arrangement and operation of text / still images is also here. It shall be included in the “service information” used in. The “service information” used here may include audio information, video (moving image) information, computer software, and the like. When dealing with MPEG-2 TS of digital broadcasting, “service information” used in the following description may be read as data broadcasting.

  FIG. 1 is a diagram for explaining a data stream processing apparatus according to an embodiment of the present invention. This apparatus configuration can be implemented as a function of video equipment such as a digital TV receiver, STB (Set Top Box), digital video recorder, and AV personal computer. The data stream DS handled in this apparatus configuration is mainly assumed to be MPEG-2 TS of digital broadcasting, but this stream DS may be an Internet broadcasting stream or a playback stream of a digital recorder / player.

  1 is a tuner (single tuner or multi-tuner) 10 that receives one or more digital broadcasts, and PCR information separation that separates PCR information from a broadcast station data stream DS (MPEG-2 TS) received by the tuner 10. Module 12, PCR offset module 14 that outputs the second PCR information (PCR-S) by appropriately adding an offset to the PCR value separated by the PCR information separation module 12, PCR or PCR offset output from the PCR information separation module 12 Based on the PCR-S output from the module 14, a service information module (or data broadcast module) 16 for extracting service information (data broadcast, caption, etc.) from the data stream DS of the broadcast station received by the tuner 10, and a service Affection And a attribute information conversion module (e.g. sampling rate converter SCR) 18 for converting module attribute of service information retrieved by the (data broadcast module) 16 (e.g., voice information of the data broadcast) (e.g. sampling frequency) as appropriate.

Here, the PCR offset module 14 adds an offset value that cancels the time lag until the AV information decoded by the decoder 22 described later is transferred to the mixer 30 to the PCR from the PCR information separation module 12. , Generate PCR-S. For example, if it takes t seconds for the AV information decoded by the decoder 22 based on the PCR to be transferred to the mixer 30, the PCR-S may be obtained by adding a numerical value equivalent to t seconds to the PCR. it can. (As a numerical value corresponding to t seconds, for example, it can be a numerical value obtained by counting an MPEG-2 reference clock (27 MHz) for t seconds.)
The apparatus of FIG. 1 further includes a decoder 22 including one or more decoder units 221 to 22n. The one or more decoder units 221 to 22n have a function of decoding one or more AV information included in one or more data streams DS (MPEG-2 TS) received by the tuner 10. Alternatively, the number of decoder units 221 to 22n corresponding to the number of types of formats may be provided so that any of a plurality of types of data formats that may be included in one data stream DS can be decoded. You can also.

  Some or all of the decoder units 221 to 22n may have a function of separating and extracting the PCR from the input data stream DS, but it is not necessary to output the PCR to the outside of the decoder 22. When none of the decoder units 221 to 22n has a PCR separation and extraction function, the function can be provided to an external device (for example, the tuner 10) of the decoder 22. When the tuner 10 has a PCR separation / extraction function, the tuner 10 sends PCR information to the decoder units 221 to 22n.

  The decoding functions of the decoder units 221 to 22n constituting the decoder 22 can be realized by one or more hardware decoders, or can be realized by a software decoder using a high-speed MPU or a high-speed DSP. The selection function of the selector 24 can also be implemented using a high-speed MPU or a high-speed DSP. When the decoder units 221 to 22n are implemented by software using a high-speed MPU or a high-speed DSP, the software can be appropriately updated by, for example, a new version of software transmitted from a broadcasting station. The updated software is stored in a non-illustrated non-volatile memory and thereafter used for operation control of the high-speed MPU or high-speed DSP. In this update, it is not necessary to consider the PCR separation and extraction function of each decoder unit.

  In this way, when the decoder 22 is configured by an updatable software decoder, for example, even if the product including the configuration of FIG. (Format D is a newly added decoding function). Alternatively, for example, even if only three software decoder units are installed at the beginning of product shipment, four or more software decoder units can be installed after the update.

  The one or more AV information decoded by the one or more decoder units 221 to 22n is input to the selector 24. The selector 24 selects one or more of one or more decoded AV information according to a selection instruction from the user, and supplies the selected AV information to the transfer processing unit 26. The transfer processing unit 26 sends the selected AV information to the attribute conversion module 28 via a wire in the apparatus or a transmission path such as HDMI (High-Definition Multimedia Interface) or IEEE 802.3 standard. The attribute conversion module 28 appropriately converts the attribute (for example, sampling frequency) of the received AV information, and outputs it to the mixer 30.

  In the mixer 30, service information (data broadcast) whose attribute is appropriately converted by the attribute conversion module 18 is synthesized or mixed with the AV information whose attribute is appropriately converted by the attribute conversion module 28. At this time, the attribute conversion module 18 and / or the attribute conversion module 28 causes the AV information attribute (for example, audio sampling frequency and the number of quantization bits) from the decoder 22 and the service information module (data broadcasting module) 16 to operate. The information attributes (for example, audio sampling frequency and quantization bit number) are converted to the same.

  The timing at which the contents of the service information (data broadcast) are presented with respect to the AV information decoded by the decoder 22 regarding the information synthesized or mixed by the mixer 30 is determined from the PCR information separation module 12. This is determined based on the output PCR or the PCR-S output from the PCR offset module 14.

  The AV information synthesized or mixed by the mixer 30 is taken into the AV capture module 32. Of the AV information captured by the AV capture module 32, the video information part (AV information video decoded by the decoder 22 is appropriately added with service information / data broadcast images and subtitles based on PCR or PCR-S. Are sent to the video processing unit 34. The video processing unit 34 appropriately performs image processing (noise reduction processing, image sharpening processing, etc.) on the received video information, and sends the video information to the display unit 36 using a high-definition flat display panel or the like.

  On the other hand, the AV capture module 32 appropriately adds the audio information part (the audio of the AV information decoded by the decoder 22) to the audio of the service information / data broadcasting based on PCR or PCR-S in the captured AV information. To the audio processing unit 38. The audio processing unit 38 amplifies the transmitted audio information and drives the speaker 40.

  In the apparatus of FIG. 1, when superimposing data broadcasting graphics or operation sounds on the AV information (video and audio) decoded by the decoder 22, for example, the following processing is performed. First, the video, audio, and PCR data acquired by the tuner 10 are separated and passed to the decoder units 221 to 22n of the decoder 22 as information necessary for decoding. On the other hand, the service information module (data broadcasting module) 16 obtains a PCR value (PCR or PCR-S) from the PCR offset module 14, determines the presentation timing of the contents of the service information (data broadcasting), and decodes the contents. It is sent out for superimposition on the video and audio that are the decoding results of No. 22.

  In the configuration of FIG. 1, a service information module (data broadcasting module) 16 connected to the PCR information separation module 12 that separates and extracts only PCR is provided independently from the decoder 22. Therefore, independent of the operation of the decoder 22 (regardless of whether or not the decoder is operating), the service information module (data broadcasting module) 16 can acquire appropriate presentation timing based on PCR.

  FIG. 2 is a flowchart for explaining an operation example of the data stream processing apparatus according to the embodiment of the present invention. In the configuration of FIG. 1, the tuner 10 receives a data stream DS including AV information, reference clock information (PCR information), and service information (such as data broadcasting) as appropriate (ST10). This data stream DS is passed to the decoder 22, the PCR information separation module 12 and the service information module (data broadcasting module) 16 separated from the decoder 22.

  The PCR information separation module 12 separates the PCR information regardless of the operation of the decoder 22 (221 to 22n) (for example, even when AV information and data broadcasting are not simultaneously decoded) (ST12). The value of the separated PCR information is appropriately given an offset in order to make the presentation timing of information synthesized or mixed in the mixer 30 more accurate (ST14). By applying this offset, the PCR information (PCR) from the PCR information separation module 12 is converted into second PCR information (PCR-S). If there is no timing problem even if this offset is not applied, PCR-S = PCR can be established. In addition, it is possible to perform a rough timing adjustment by PCR and use PCR-S when a more precise timing adjustment is required.

  In the service information module (data broadcasting module) 16, information such as BML related to service information (data broadcasting) is acquired from the data stream DS, and a PCR value (PCR or PCR-S) is acquired. In the configuration of FIG. 1, this PCR value is obtained from the PCR information separation module 12 or the PCR offset module 14 provided outside the service information module (data broadcasting module) 16. However, instead of providing the modules 12 and 14 outside the module 16, a PCR information separating and supplying module (not shown) corresponding to the module 12 + 14 may be built inside the module 16.

  In the service information module (data broadcasting module) 16, only the PCR value is reflected in a reference clock STC (System Time Clock) (not shown) in the video equipment (TV or the like). Since the PCR value is reflected in the STC, the contents of the service information (data broadcast) in the video device are extracted using the PCR value (PCR or PCR-S) (ST16). In other words, the second reference clock corresponding to the reference clock information (PCR) extracted from the module 12 (PCR-S: where PCR = PCR-S is also included as one of the correspondences) The content of the service information (data broadcast text, video, audio, etc.) is extracted from the data stream (ST16). The extracted information is appropriately converted in attributes (sampling rate, etc.) (ST18) and sent to the mixer 30.

  On the other hand, apart from the service information (data broadcasting) processing (ST12 to ST16), AV information decoding processing is also performed. That is, AV information using PCR is individually decoded in one or more of the decoders 221 to 22n (ST22). (In ST22, a plurality of decoders may cooperate to decode the same AV information. For example, when decoding a 2k * 4k class ultra-high-resolution next-generation high-definition video, one decoder cannot handle it. May operate a plurality of decoders in parallel to cope with the video decoding.) When at least one of the decoded AV information is selected (ST24), the selected AV information is selected. Is transferred to the mixer 30 (ST26). At that time, the attributes (sampling rate, etc.) of the AV information transferred on the decoder 22 side are appropriately converted (ST28).

  The content of the AV information whose attribute is appropriately converted and the service information (data broadcast or the like) whose attribute is appropriately converted is synthesized (superimposed or mixed) at the timing in the mixer 30 (ST30). This timing adjustment can be performed based on the above-described PCR or PCR-S. The AV information + service information (data broadcast, etc.) thus timed is output to an AV capture device (digital TV, digital recorder, AV personal computer, etc.) (ST32).

  Here, since the compressed sound is used for the operation sound (sound effect) of the data broadcasting, the sound decoding function on the decoder 22 side is usually used for decoding the operation sound (sound effect). In that case, in order to instruct the superimposition (synthesis or mixing) timing of the operation sound (sound effect) from the service information module (data broadcasting module) 16 to the decoder 22 side, an embodiment in which a dedicated decoder for operation sound is provided on the module 16 side. Can be considered. Also, the superimposition (synthesizing or mixing) timing is fed back from the module 16 to the decoder 22, and the audio information in which the operation sound (sound effect) of the service broadcast is superimposed on the audio from the decoder 22 at an appropriate timing is sent to the module 16 side. An embodiment that is dedicated to the operation sound is also conceivable.

  When superimposing (synthesizing or mixing) audio is performed by the service information module (data broadcasting module) 16, the output from the decoder 22 may be PCM audio, and the transmission may be HDMI transmission or normal PCM capture. . In this case, a system configuration is required in which sound effects are superimposed (synthesized or mixed) on PCM sound that has entered via HDMI. Since it is conceivable that the sampling frequency of PCM audio via HDMI varies depending on the input source, a mode in which SRC (Sampling Rate Converter) is mounted on the operation sound and / or the input PCM audio can be considered. In the configuration of FIG. 1, the attribute conversion modules 18 and 28 are responsible for this SRC function.

  It is also effective to present the PCR value with an offset in consideration of the time lag for transferring audio and video. In the configuration of FIG. 1, the PCR offset module 14 has a function of this PCR offset.

  By adopting the configuration as described above, the processing function of the service information module (data broadcasting module) 16 can be separated from the decoder 22 which can include a large number of decoder functions, and the degree of freedom of the system configuration is increased.

<Summary of Embodiment>
1. In a system using a reference clock such as PCR (a system including modules 12 to 16), the PCR value is acquired without simultaneously decoding AV information.

  2. The operation results (text, image / video, audio, etc.) of the system (12-16) operated using the acquired PCR are superimposed (synthesized or mixed) on the audio or video decoded by the other decoder 22.

  3. In order to realize the above, a configuration is adopted in which the data stream DS is supplied to the modules (12, 16) using the PCR separately from the decoder 22.

<Modification>
The embodiment of the present invention can be used in a system separated from the decoder 22 (a system such as a PCR that does not normally depend on a reference clock generated and supplied by the decoder 22). For example, it can also be used in a caption broadcast processing device that uses a reference clock (PCR) independently of the operation of the AV information decoder 22.

<Effect of Embodiment>
a. When one or more AV decoders (221 to 22n) specialized for the AV information decoding function are used, even if PCR information is not obtained from these decoders or regardless of the operation of these decoders, The presentation timing of subtitles can be determined.

  b. When synthesizing the decoding result of the AV decoder and data broadcasting, etc., even when the synthesis timing cannot be accurately adjusted only by PCR, by using PCR-S having an appropriate offset in PCR, the AV decoder It is possible to accurately match the decoding result and the synthesis timing of data broadcasting.

  c. Since service functions such as data broadcasting are separated from one or more AV decoders (221 to 22n), the configuration of each decoder is simplified, and the addition of AV decoders is facilitated. In particular, when a plurality of AV decoders are realized by software using a high-speed MPU or a high-speed DSP, it is easy to change the function of the AV decoder and / or increase or decrease the number of AV decoders by updating the software.

<Correspondence Example between Embodiment and Invention>
<1> The data stream processing apparatus according to an embodiment of the present invention is capable of decoding AV information from a data stream (DS) including AV information, reference clock information (PCR), and service information (data broadcast, etc.). The apparatus (or system) used in combination with the decoders (221 to 22n). The apparatus includes a reference clock information separation module (12) that extracts the reference clock information (PCR) from the data stream, regardless of the operation of one or more decoders (even without simultaneous decoding), and the data stream. A service information module (16) is provided for extracting the contents of the service information (text, video, audio of data broadcast; subtitles of subtitle broadcast, etc.). Then, based on the extracted reference clock information (PCR or PCR-S), the timing of the AV information decoded by the one or more decoders, and the service information extracted by the service information module (16) It is configured to match the timing of the contents (rough timing alignment is possible with PCR, and precise timing alignment is possible with PCR-S).

  <2> A video device according to an embodiment of the present invention includes a receiving module (10) that receives a digital stream (DS) including AV information, reference clock information (PCR), and service information (such as data broadcasting); One or more decoders (221 to 22n) for decoding AV information from the digital stream are provided. The video apparatus further includes a reference clock information separation module (12) that extracts the reference clock information (PCR) from the digital stream regardless of the operation of the one or more decoders (without performing simultaneous decoding); The service information module (16) for extracting the contents of the service information (character, video, audio, etc.) from the digital stream, and the AV information and the timing that are timed based on the reference clock information (PCR or PCR-S) A mixer (30) that synthesizes the contents of service information, and a video / audio capture module (32) that takes in and uses the contents of the AV information and the service information synthesized by the mixer.

  <3> The data stream processing method according to an embodiment of the present invention is capable of decoding AV information from a data stream (DS) including AV information, reference clock information (PCR), and service information (data broadcast, etc.). This method uses the decoders (221 to 22n). In this method, the reference clock information (PCR) is extracted from the data stream (ST12) regardless of the operation of one or more decoders (without simultaneous decoding), and the extracted reference clock information (PCR) The content of the service information (data broadcast text, video, audio, etc.) is extracted from the data stream based on the reference clock (PCR-S) corresponding to (ST16). Then, the timing of the AV information decoded by the one or more decoders and the timing of the contents of the service information are matched based on the reference clock (PCR-S) (ST30).

  In addition, this invention is not limited to embodiment mentioned above, In the implementation stage, it can change variously in the range which does not deviate from the summary. In addition, the embodiments may be appropriately combined as much as possible, and in that case, the combined effect can be obtained. Further, the above embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, the problem described in the column of the problem to be solved by the invention can be solved, and the effect described in the column of the effect of the invention Can be obtained as an invention.

  DESCRIPTION OF SYMBOLS 10 ... Tuner (reception module), 12 ... PCR information separation module (reference clock information separation module), 14 ... PCR offset module, 16 ... Service information module (data broadcasting module), 18 ... Attribute conversion module, 22 ... Decoder, 24 Selector, 26 ... Transfer processing unit (internal wiring or external wiring such as HDMI), 28 ... Attribute conversion module, 30 ... Mixer, 32 ... AV capture module (video / audio capture module), 34 ... Video processing unit, 36 ... Display unit 38 ... Audio processing unit, 40 ... Speaker.

Claims (9)

An apparatus used in combination with one or more decoders capable of decoding the video information and / or the audio information from a data stream including video information and / or audio information, reference clock information, and service information,
A reference clock information separation module that extracts the reference clock information from the data stream regardless of the operation of the one or more decoders;
A service information module that extracts the contents of the service information from the data stream;
Based on the extracted reference clock information, the timing of the video information and / or the audio information decoded by the one or more decoders is matched with the timing of the content of the service information extracted by the service information module. A data stream processing device configured as described above.   2. The apparatus according to claim 1, wherein the service information corresponds to data broadcasting, and includes a mixer that synthesizes the content of the service information and the video information and / or the audio information that are timed based on the reference clock information. .   The one or more decoders include one or more decoders that do not extract the content of the service information, and the video information and / or the audio information decoded by one of these decoders and the content of the service information are the mixer The device of claim 2, which is synthesized by:   The apparatus according to claim 1, configured to generate a value obtained by adding an offset to the content of the reference clock information.   At least one attribute conversion among a decoder side attribute conversion module that changes an attribute of information decoded by the one or more decoders, and a service information processing side attribute conversion module that changes an attribute of service information extracted from the service information module The apparatus of claim 1 comprising a module. A receiving module for receiving a digital stream including video information and / or audio information, reference clock information, and service information;
One or more decoders for decoding the video information and / or the audio information from the digital stream;
A reference clock information separation module that extracts the reference clock information from the digital stream regardless of the operation of the one or more decoders;
A service information module for extracting the contents of the service information from the digital stream;
A mixer for synthesizing the contents of the service information and the video information and / or the audio information, the timing of which is adjusted based on the reference clock information;
A video apparatus comprising a video / audio capture module that captures and uses the contents of the video information and / or the audio information and the service information synthesized by the mixer.   The video apparatus according to claim 6, wherein the video / audio capture module is connected to a video processing unit that displays the video information and characters and / or video included in the service information.   The video apparatus according to claim 6, wherein the video / audio capture module is connected to an audio processing unit that presents the audio information and a sound included in the service information. A method using one or more decoders capable of decoding the video information and / or the audio information from a data stream including video information and / or audio information, reference clock information and service information,
Regardless of the operation of the one or more decoders, the reference clock information is extracted from the data stream;
Based on the reference clock corresponding to the extracted reference clock information, the content of the service information is extracted from the data stream,
A data stream processing method for matching the timing of the video information and / or the audio information decoded by the one or more decoders with the timing of the content of the service information based on the reference clock.

Images