JP6953238B2 - Receiver, receiving method and program - Google Patents

Description

The present invention relates to a receiving device, a receiving method and a program.

The MMT (MPEG Media Transport) method adopted in Japan's advanced broadband satellite broadcasting is a multiplexing method for transmitting video, audio, multimedia data, etc. regardless of the transmission path such as broadcasting and communication. The MMT method is more convenient for cooperation between broadcasting and communication than the conventional MPEG-2 system (Moving Picture Expert Group-2 Systems). By adopting the MMT method, it has become possible to provide more convenient services for viewers. For example, the multimedia service can be multiplexed and provided as HTML5 (HyperText Modeling Language) content. In the multimedia service, the viewer can display the content in a desired form and can select various forms of the content from more options via communication.

At present, the MMT method is in the process of becoming widespread, and the development of products and systems compatible with MMT has just begun. In particular, there are few LSIs (Large Scale Integrated Circuits) that form the core of the configuration system of a television receiver, and those that have the ability to analyze MMT from received signals and extract and process multimedia data required for various services. Such an LSI is called a SoC (System on Chip). In the currently popular SoC, the SoC corresponding to the MPEG-2 TS (Transport Stream) system defined by the MPEG-2 system is the mainstream.

In addition, existing broadcasting services provided using the conventional MPEG-2 TS system have been decided to continue for the time being. Therefore, the SoC provided in the receiving device is required to support the MPEG-2 TS system. In view of such a situation, for example, a receiving device that converts the multiplexed data multiplexed by the MMT method into the multiplexed data multiplexed by the MPEG-2 TS method has been proposed (Patent Document 1). ..

JP-A-2017-73761

However, when recording the multiplexed data multiplexed by the conventional MPEG-2 TS method or playing back the content, the compatibility of the data multiplexed with the video or audio may not be ensured. For example, the character code compatibility before and after conversion of subtitle data separated from MMT-type multiplexed data into MPEG-2 TS-type subtitle ES (Emergent Stream) data is not perfect. Therefore, the subtitles may not be reproduced by the conventional method.

On the other hand, in the MPEG-2 TS system, a means for multiplexing and transmitting subtitle data described in TTML (Timed Text Markup Language) is standardized. However, when recording and reproducing content from multiplexed data multiplexed by the MPEG-2 TS method, the method of synchronizing the subtitle data with the video data, audio data, and the like is not clearly defined. For example, in the multiplexed data multiplexed by the MPEG-2 TS method, it is not possible to synchronize the time based on NTP (Network Time Protocol). Therefore, it may not be possible to synchronize playback with other data at the display start time described in the TTML subtitle data.

The present invention has been made to solve the above problems, and one aspect of the present invention is from the first multiplexing data multiplexed by the first multiplexing method to audio data which is a component of a broadcasting service. A separation unit that separates the video data, the audio data, and additional content data added to the video data, and the audio data and the video data are multiplexed by a second multiplexing method different from the first multiplexing method. The multiplexing unit that generates the second multiplexing data, the output unit that outputs the second multiplexing data and the additional content data, respectively, and the second multiplexing data in a separation method corresponding to the second multiplexing method. wherein the voice data is separated video data, and a reception processing unit for superimposing one of the speech indicated by a picture or the audio data and the additional content based on the added contents data indicating at least the video data from the The separation unit further separates the reference time information indicating the reference time from the first multiplexed data, the reception processing unit detects the time difference between the system time and the reference time, and sets the presentation time of the additional content as described above. It is a receiving device that presents the additional content at a time corrected by a time difference.

According to the embodiment of the present invention, even a device adopting the conventional second multiplexing method can provide various broadcasting services based on the received signal multiplexed by the first multiplexing method.

It is a schematic block diagram which shows the functional structure example of the receiving apparatus which concerns on 1st Embodiment. It is a schematic block diagram which shows the functional structure example of the reception processing part which concerns on 1st Embodiment. It is explanatory drawing for demonstrating the example of the synchronization process which concerns on 2nd Embodiment. It is explanatory drawing for demonstrating the example of the synchronization process which concerns on 3rd Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
First, an outline of the receiving device 1 according to the first embodiment of the present invention will be described.
FIG. 1 is a schematic block diagram showing a functional configuration example of the receiving device 1 according to the present embodiment.
The receiving device 1 includes a broadcast receiving unit 12, a demodulation unit 14, a conversion processing unit 20, and a receiving processing unit 30.

The broadcast receiving unit 12 receives a broadcast signal transmitted from a transmitting device (not shown) via a broadcast transmission line as a 儒 reception signal. A broadcast transmission line is a transmission line that unilaterally and simultaneously transmits broadcast signals to an unspecified number of receiving devices. The broadcast transmission line is a broadcast wave in a predetermined frequency band. The broadcast transmission line may include a wired or wireless communication line such as a network or a dedicated line as a part thereof. The broadcast receiving unit 12 receives a broadcast signal in the frequency band corresponding to the broadcast channel indicated by the control signal input from the operation control unit (not shown). The operation control unit outputs, for example, an operation signal input from the remote control device (remote controller) to the broadcast receiving unit 12 as a control signal in response to the user's operation. The operation control unit may be provided in either the conversion processing unit 20 or the reception processing unit 30, or may be provided independently of any of them.
The received signal may be an IF (Intermediate Frequency) signal. The broadcast receiving unit 12 is, for example, a tuner. The broadcast receiving unit 12 down-converts the received broadcast signal into a broadcast signal in the baseband, and outputs a broadcast signal obtained by down-converting the received broadcast signal to the demodulation unit 14.

The demodulation unit 14 demodulates the broadcast signal input from the broadcast receiving unit 12 using a predetermined demodulation method, and converts it into multiplexed data multiplexed by a predetermined multiplexing method. The demodulation method may be any demodulation method corresponding to the modulation method used in the transmission device (for example, 8PSK: 8 Phase Shift Keying (8-phase phase modulation), APSK: Amplitude and Phase Shift Keying (amplitude phase modulation)). .. The multiplexed data is, for example, a TLV (Type-length-value) stream. The TLV stream is a data stream composed of a series of TLV packets multiplexed by the TLV / MMT method. The multiplexing process by the TLV / MMT method includes the multiplexing by the MMT method and the multiplexing by the TLV method. The TLV method and the MMT method may be regarded as a transport method (transmission method). The demodulation unit 14 outputs the generated multiplexed data to the conversion processing unit 20. The demodulation unit 14 is, for example, a demodulator.

The conversion processing unit 20 separates the multiplexed data input from the demodulation unit 14 into audio data, video data, additional content data, and additional data related to broadcasting, which are components of the broadcasting service. The additional content data is data related to the additional content added to the audio and video provided by the broadcasting service. The additional content is content other than audio and video. The additional content data includes, for example, character data indicating characters to be superimposed on the video, multimedia data, or a combination thereof. The additional data includes control information related to reception processing and reference time information indicating a reference time. The conversion processing unit 20 is separated by a second multiplexing method (for example, MPEG-2 TS method), which is a multiplexing method separate from the first multiplexing method used for multiplexing the input multiplexed data. The second multiplexed data is generated by multiplexing the audio data and the video data.

In the following description, the multiplexed data input from the demodulation unit 14 to the conversion processing unit 20 is referred to as the first multiplexed data, and is distinguished from the second multiplexed data. The conversion processing unit 20 outputs the generated second multiplexed data and additional content data to the reception processing unit 30.
The conversion processing unit 20 may be configured to include a general-purpose or dedicated arithmetic device such as a CPU (Central Processing Unit). The arithmetic device may realize its function by executing the process instructed by the instruction described in the predetermined control program.

The reception processing unit 30 separates the audio data and the video data from the second multiplexing data input from the conversion processing unit 20 by a separation method corresponding to the second multiplexing method. The reception processing unit 30 decodes the separated voice data into a voice signal by a predetermined voice decoding method (for example, MPEG-4 AUDIO). The reception processing unit 30 decodes the separated video data into a video signal by a predetermined video decoding method (for example, HEVC: High Efficiency Video Coding).

Further, the reception processing unit 30 configures the additional content based on the additional content data input from the conversion processing unit 20. When the constituent additional content includes visual information, the reception processing unit 30 synthesizes the visual information with the video data and outputs the combined video data to a display unit (not shown). The visual information includes information that makes it visible, such as moving images, still images, characters, symbols, and figures. When the constituent additional content includes auditory information, the reception processing unit 30 synthesizes the auditory information into audio data and outputs the synthesized audio data to a reproduction unit (not shown). Auditory information includes information that can be heard, such as spoken voice, operating sound, sound effect, and music.
The reception processing unit 30 is, for example, an integrated circuit including a dedicated arithmetic device, a so-called SoC.

Next, the functional configuration of the conversion processing unit 20 will be described.
The conversion processing unit 20 includes an input unit 202, a decryption unit 204, a descramble unit 206, a local encryption unit 208, an HTML storage unit 210, a local decryption unit 212, a conversion unit 214, and a separation unit 220. , The analysis unit 222, the multiplexing unit 230, the SI management unit 232, the NTP management unit 234, the NTP management unit 236, the HTML management unit 238, the output unit 240, and the input / output unit 242. It is composed.

The first multiplexing data is input to the input unit 202 from the demodulation unit 14, and the input first multiplexing data is output to the decoding unit 204. The input unit 202 is, for example, a data input interface.

The first multiplexed data is input to the decoding unit 204 from the input unit 202. The multiplexed data input from the input unit 202 is encrypted by a predetermined encryption method (for example, AES: Advanced Encryption Standard), and an error correction code is further added. The error correction code is, for example, a BCH (Bose Chain Hocquengem) code and an LDPC (Low Density Parity Check) code. The decoding unit 204 performs error correction processing on the first multiplexed data encrypted using the added error correction code. The decoding unit 204 performs decoding processing on the first multiplexed data obtained by performing error correction processing, and converts it into unencrypted first multiplexing data. The decryption method used for the decryption process may be any method corresponding to the encryption method used for the encryption. The decoding unit 204 outputs the converted first multiplexed data to the descramble unit 206.

The first multiplexed data is input from the decoding unit 204 to the descramble unit 206. The first multiplexed data input to the descramble unit 206 is scrambled multiplexed data. The descramble unit 206 descrambles the first multiplexed data using the key information for descramble (descrambling), and outputs the descrambled multiplexed data to the local encryption unit 208 or the conversion unit 214. .. More specifically, the descramble unit 206 decrypts the individual information (EMM: Enterprise Management Message) using the preset identification information, and the contract information and common information (ECM: Enterprise) for each subscriber are decrypted. Generates a work key for decrypting (decrypting) the Control Message). The descramble unit 206 decrypts the common information (ECM) included in the additional data constituting the first multiplexed data by using the generated work key, and extracts the decrypted descramble key. do. The descramble unit 206 descrambles the multiplexed data using the extracted descramble key. The descramble unit 206 outputs the descrambled multiplexed data to the conversion unit 214.

When recording is instructed by a control signal input from an operation control unit (not shown) that controls the operation of the receiving device 1, the descramble unit 206 locally scrambles the descrambled first multiplexed data. It may be output to the encryption unit 208.

The local encryption unit 208 encrypts the first multiplexed data input from the descramble unit 206 by a predetermined encryption method to generate the encrypted first multiplexed data, and the generated first multiplexed data. Is stored in the TLV storage unit 210.
Therefore, the encrypted TLV stream is stored in the TLV storage unit 210. The TLV storage unit 210 includes, for example, a storage medium such as an HDD (Hard Disk Drive).

The local decoding unit 212 sequentially reads the TLV stream in the section indicated by the control signal input from the operation control unit (not shown), and decodes the first multiplexed data forming the read TLV stream. The decryption method used by the local decryption unit 212 may be any decryption method corresponding to the encryption method used by the local encryption unit 208 for encryption. The local decryption unit 212 outputs the decrypted encrypted data to the conversion unit 214.

The conversion unit 214 converts the format of the first multiplexed data input from the descramble unit 206 or the local decoding unit 212 from the TLV stream into an MMTP packet, and outputs the converted first multiplexed data to the separation unit 220. do. Here, the conversion unit 214 extracts an IP (Internet Protocol) packet from each TLV packet constituting the TLV stream. The conversion unit 214 further extracts an MMTP (MMT Protocol) packet and an IP packet (hereinafter, NTP packet) storing time information in the NTP (Network Time Protocol) format from the extracted IP packet. The MMTP packet stores various subdivided data multiplexed by the MMT method. The NTP packet stores reference time information, which is time information indicating a reference time. As the reference time, for example, a transmission time for each predetermined time based on Coordinated Universal Time (UTC) is used. In the NTP packet, the reference time information is represented by the time stamp of that time. The conversion unit 214 sequentially outputs the extracted MMTP packet and the NTP packet to the separation unit 220.

The separation unit 220 includes audio data indicating audio, video data indicating video, additional content data indicating additional content, and control information that are components (assets) of the broadcasting service from the first multiplexing data input from the conversion unit 214. To separate. The additional content is content other than audio and video. The additional content data includes, for example, character data indicating characters to be superimposed on the video, multimedia data, or a combination thereof. The character data may be data (subtitles) indicating a character string indicating the content related to the audio or video provided by the broadcasting service, or indicates a character string indicating the content independent of the audio or video. It may be data (character super, asynchronous subtitles).

The character data is, for example, data described in TTML (Timed Text Markup Language). TTML is one of the markup languages, and is characterized in that the display timing and display position of text can be specified. The multimedia data is data for presenting a set of expression media (multimedia) composed of a plurality of expression media. The individual expression media are, for example, moving images, still images, figures, sounds, characters, and the like. The multimedia data does not necessarily have to include data indicating a part or all of the expression media. The multimedia data may include information indicating the location of the expression media (such as an address on the network). The multimedia data is provided, for example, as an application described in HTML5 (HyperText Markup Language). HTML5 is one of the markup languages, and is characterized in that it is possible to specify playback, stop, and the like of video and audio.

As additional data, control information regarding broadcast reception and reference time information are included. The control information includes MMT-SI (System Information) related to MMT, which is a media transport method. The MMT-SI includes various tables and descriptors as its element information. In addition to the above-mentioned EMM and ECM, the MMT-SI includes MPT (MMT Package Table, MMT Package Table), MH-EIT (MH-Event Information Table, MH-Event Information Table, etc.) as tables related to the present embodiment. MH-SDT (MH-Service Description Table, MH-Service Description Table), MH-TOT (MH-Time Office Table, MH-Time Offset Table), MH-AIT (MH-Application Information Table, etc.) Included. MPT indicates configuration information that constitutes a package such as a list of assets. MH-EIT indicates program information related to each program such as a program (event) name, broadcast time, title, performers, and genre. MH-SDT indicates channel information related to each channel such as the name of the organization channel and the name of the broadcaster. MH-TOT indicates the current date and time information indicating the current time. However, it is indicated by MH-TOT. The current time is a time (JST time) based on Japanese standard time, and is used for a setting screen, a guidance screen, and various other displays from the receiving device 1. In the following, the time information indicated by MH-TOT is used as a system. Time information and that time are called system time, and are distinguished from NTP format reference time information and reference time. MH-AIT provides control information for executing and stopping applications and additional information (application control information) required for execution. If application control information exists, the information indicating its existence is described in the MPT.

Further, the MMT-SI includes an MH-data coding method descriptor (MH-Data_Component_Descriptor) as a descriptor related to the present embodiment. Additional identification information (additional_data_component_info) is described in the MH-data encoding method descriptor. The reference start time (reference_start_time) is described in the additional identification information. The reference start time is the presentation start time of the character information and is the time specified in the NTP format. Specifically, the reference start time is specified by the UTC time, which is the starting point of the time code in the TTML document in which the subtitles and the character super are described in TTML.

The separation unit 220 outputs the separated audio data and video data to the multiplexing unit 230, and outputs MMT-SI as control information to the analysis unit 222. Further, the separation unit 220 outputs the reference time information to the NTP management unit 234, the character data to the TTML management unit 236, and the multimedia data to the HTML management unit 238.

The analysis unit 222 analyzes the control information input from the separation unit 220, and analyzes the MPT indicating the configuration information, the MH-EIT indicating the program information, the MH-SDT indicating the channel information, the MH-TOT indicating the system time information, and the application. MH-AIT showing control information is extracted. Further, the analysis unit 222 extracts additional identification information indicating the reference start time related to the presentation of the character data from the control information. The analysis unit 222 stores the extracted various information in the SI management unit 232. Hereinafter, the information extracted from MMT-SI is collectively referred to as SI information.

The analysis unit 222 generates a clock signal (STC: System Time Clock) having a predetermined clock frequency (for example, 27 MHz) synchronized with the reference time indicated by the reference time information input from the separation unit. The clock signal is used to synchronize the processing between each component of the conversion processing unit 20 (clock synchronization). The analysis unit 222 divides the generated clock signal at a predetermined frequency division rate (for example, 1/2700) and reconstructs the reference time information indicating the reference time for each predetermined time (for example, 0.1 ms). do. The analysis unit 222 outputs the reconstructed reference time information to the multiplexing unit 230 as PCR (Program Lock Reference) defined by the MPEG-2 TS method. The analysis unit 222 analyzes the MPT indicating the component information and converts the audio data indicating the broadcast audio and the video data indicating the video into a PMT (Program Map Table) indicating the components. The PMT is an information table showing configuration information in the MPEG-2 TS format. The analysis unit 222 converts the MH-TOT indicating the system time information into the TOT indicating the system time information in the MPEG-2 TS format. The analysis unit 222 outputs the PMT indicating the configuration information and the TOT indicating the system time information to the multiplexing unit 230.

Of the audio data and video data input from the separation unit 220, the multiplexing unit 230 identifies the audio data and video data designated by the PMT indicating the configuration information input from the analysis unit 222. The multiplexing unit 230 converts the specified audio data format and video data format into an elementary stream (ES: Elementary Stream), which is a format defined by the MPEG-2 TS system. Further, the multiplexing unit 230 constitutes control information including a PMT indicating the configuration information input from the analysis unit 222 and a TOT indicating the system time information. The data format of the control information to be configured is the format specified by the MPEG-2 TS method. The multiplexing unit 230 multiplexes the converted audio data, video data, PCR and control signals input from the analysis unit 222, and generates the second multiplexing data in the format specified in the MPEG-2 TS system. .. The PCR is included in the TS forming the second multiplexed data at predetermined time (eg, 0.1 ms) intervals. The multiplexing unit 230 outputs the generated second multiplexing data to the output unit 240.

The SI management unit 232 sequentially stores SI information input from the analysis unit 222. The SI management unit 232 may store the SI information in association with the reference time information indicating the transmission time from the transmission device. When the information request signal is input from the reception processing unit 30 via the input / output unit 242, the SI management unit 232 inputs the SI information specified by the information request signal into the reception processing unit 30 as a response. It may be output via the output unit 242.

The NTP management unit 234 sequentially stores the reference time information input from the separation unit 220. The NTP management unit 234 outputs the newly input reference time information to the reception processing unit 30 via the input / output unit 242.

The TTML management unit 236 sequentially stores the character data input from the separation unit 220. The TTML management unit 236 outputs the newly input character data to the input / output unit 242. When the information request signal is input from the reception processing unit 30 via the input / output unit 242, the TTML management unit 236 inputs the character data specified by the information request signal into the reception processing unit 30 as a response. It may be output via the output unit 242.

The HTML management unit 238 sequentially stores multimedia data input from the separation unit 220. The HTML management unit 238 outputs the newly input multimedia data to the input / output unit 242. When the information request signal is input from the reception processing unit 30 via the input / output unit 242, the HTML management unit 238 sends the multimedia data specified by the information request signal to the reception processing unit 30 as a response. It may be output via the input / output unit 242.
The SI management unit 232, the NTP management unit 234, the TTML management unit 236, and the HTML management unit 238 may be configured as relational databases, respectively.

The output unit 240 outputs the multiplexing data input from the multiplexing unit 230 to the reception processing unit 30. The output unit 240 is, for example, an output interface.
The input / output unit 242 outputs various data input from the SI management unit 232, the NTP management unit 234, the TTML management unit 236, and the HTML management unit 238 to the reception processing unit 30. Further, when the information request signal is input from the reception processing unit 30, the input / output unit 242 outputs the information request signal input to the output destination component unit (for example, the TTML management unit 236). The input / output unit 242 is, for example, an input / output interface.

Next, the functional configuration of the reception processing unit 30 will be described with reference to FIG.
FIG. 2 is a schematic block diagram showing a functional configuration example of the reception processing unit 30 according to the present embodiment. FIG. 2 shows the conversion processing unit 20. However, the illustration of the input unit 202, the decryption unit 204, the descramble unit 206, the local encryption unit 208, the TLV storage unit 210, the local decryption unit 212, and the conversion unit 214 is omitted.

The reception processing unit 30 includes an input unit 320, an input / output unit 322, a communication unit 324, a separation unit 330, a video decoding unit 332, a video compositing unit 334, an audio decoding unit 336, and an audio mixing unit 338. , A synchronization processing unit 340, and an additional content processing unit 342.

The input unit 320 outputs the second multiplexed data input from the conversion processing unit 20 to the separation unit 330. The input unit 320 is, for example, an input interface.
The input / output unit 322 outputs various data input from the conversion processing unit 20 to the synchronization processing unit 340 or the additional content processing unit 342. Further, the input / output unit 322 outputs the information request signal input from the additional content processing unit 342 to the conversion processing unit 20. The input / output unit 322 is, for example, an input / output interface.
The communication unit 324 transmits and receives various data between the additional content processing unit 342 and the device connected to the network. The network may be wired or wireless. The communication unit 324 is, for example, a communication interface.

The separation unit 330 separates audio data, video data, and control information from the second multiplexed data input from the input unit 320. The separation unit 330 outputs the separated audio data and video data to the audio decoding unit 336 and the video decoding unit 332, respectively. The separation unit 330 extracts system time information from the separated control information, and outputs a TOT indicating the extracted system time information to the synchronization processing unit 340. The reception processing unit 30 presents the content to be broadcast based on the system time represented by the TOT and various related information related to the content. The related information includes, for example, reservation information related to program reservation, a current timetable showing the current time, and the like.

The video decoding unit 332 decodes the video data input from the separation unit 330 using a predetermined video decoding method. The video data input to the video decoding unit 332 is encoded by a video coding method corresponding to the video decoding method. The video decoding unit 332 outputs the decoded video data to the video compositing unit 334.

The video synthesizing unit 334 synthesizes the visual information input from the additional content processing unit 342 with the video data input from the video decoding unit 332, and generates the video data on which the visual information is superimposed. The video compositing unit 334 outputs the generated video data to a display unit (not shown).

The voice decoding unit 336 decodes the voice data input from the separation unit 330 using a predetermined voice decoding method. The voice data input to the voice decoding unit 336 is encoded by a voice coding method corresponding to the voice decoding method. The audio decoding unit 336 outputs the decoded audio data to the audio mixing unit 338.

The audio mixing unit 338 synthesizes the auditory information input from the additional content processing unit 342 with the audio data input from the audio decoding unit 336, and generates audio data on which the auditory information is superimposed. The audio mixing unit 338 outputs the generated audio data to a reproduction unit (not shown).

The synchronization processing unit 340 corrects the presentation control time of the additional content by the time difference between the reference time indicated by the reference time information input from the conversion processing unit 20 and the system time indicated by the system time information input from the separation unit 330. The presentation control time is a general term for the presentation start time and the presentation end time. More specifically, the synchronization processing unit 340 calculates the time difference of the NTP-based reference time from the TOT-based system time, and outputs the time difference information indicating the calculated time difference to the additional content processing unit 342. The function of the synchronization processing unit 340 is realized by the above-mentioned arithmetic device executing the processing instructed by the instruction described in a predetermined script language (for example, ECMA-262: European Computer Manufacturers Association-262). ..

The additional content processing unit 342 executes a predetermined application program (hereinafter, application) to acquire the additional content. When the acquired additional content includes visual information, the additional content processing unit 342 outputs the visual information to the video compositing unit 334. When the acquired additional content includes auditory information, the additional content processing unit 342 outputs the auditory information to the audio mixing unit 338. The application is described in, for example, HTML5, and various commands related to the presentation of additional contents and other information are described. To execute an application means to perform a process instructed by an instruction described in the application. The additional content processing unit 342 is, for example, a browser.

The predetermined application executed by the additional content processing unit 342 may be an application stored in advance in a storage unit (not shown) of the reception processing unit 30, or a broadcasting application transmitted by a broadcasting signal. good. As the preset application, for example, there are a subtitle display application, a program guide display application, and the like. Broadcast applications include multimedia data stored in the HTML management unit 238.

Here, the execution of the subtitle display application will be described. The time when the execution of the subtitle display application is started is, for example, the time when the additional content processing unit 342 starts receiving the component information including the character data as the component of the service. The configuration information is included in the latest SI information transmitted from the SI management unit 232 as described above. When executing the subtitle display application, the additional content processing unit 342 performs the following processing. The additional content processing unit 342 generates an information request signal indicating the request for the latest character data at that time, and outputs the generated information request signal to the TTML management unit 236. The additional content processing unit 342 receives character data from the TTML management unit 236 as a response to the information request signal. The additional content processing unit 342 specifies the reference start time (reference_start_time) described in the MH-data coding method descriptor from the received SI information. Then, the additional content processing unit 342 adds the time difference indicated by the time difference information input from the synchronization processing unit 340 to the specified reference start time, and calculates the time shifted by the time difference from the reference start time as the correction reference start time. .. The calculated correction reference start time corresponds to the display time of the subtitles according to the TOT-based time scale. The additional content processing unit 342 outputs a subtitle composed of a character string indicated by the character data to the video synthesis unit 334 when the current time reaches the correction reference start time. Here, the correction reference start time based on the reference time corresponds to the reference start time based on the system time. Therefore, the subtitles and the broadcast video are displayed synchronously on the display unit at the reference start time based on the system time.

The process of calculating the correction reference start time may not be performed by the additional content processing unit 342, but may be executed by the conversion processing unit 20. In that case, the analysis unit 222 of the conversion processing unit 20 has a time difference between the system time of the TOT reference indicated by the MH-TOT separated from the MMT-SI and the reference time of the NTP reference indicated by the reference time information input from the separation unit 220. Is calculated. The analysis unit 222 calculates the correction reference start time by adding the reference start time (reference_start_time) extracted from the MPT to the calculated time difference. The analysis unit 222 outputs the correction reference start time information indicating the calculated correction reference start time to the TTML management unit 236 in association with the character data to which the reference start time is applied. The analysis unit 222 can refer to the MPT and specify the character data to be applied to the asset corresponding to the asset descriptor area including the MH-data encoding method descriptor in which the reference start time is described. ..
When the subtitle display application is executed, the additional content processing unit 342 generates an information request signal indicating a request for the latest character data at that time, and outputs the generated information request signal to the TTML management unit 236. The additional content processing unit 342 receives character data from the TTML management unit 236 in association with the correction reference start time information as a response to the information request signal. When the current time of the TOT reference reaches the correction reference start time indicated by the correction reference start time information, the additional content processing unit 342 outputs a subtitle composed of the character string indicated by the received character data to the video synthesis unit 334. Therefore, even with this method, the subtitles and the broadcast video are displayed in synchronization on the display unit at the reference start time based on the system time.

Next, the execution of the program guide display application will be described. The time when the execution of the program guide display application is started is, for example, when a control signal instructing the program guide display is input from the operation control unit (not shown) to the additional content processing unit 342. When executing the program guide display application, the additional content processing unit 342 performs the following processing. The additional content processing unit 342 generates an information request signal indicating the latest channel information and program information at that time, and outputs the generated information request signal to the SI management unit 232. The additional content processing unit 342 receives channel information and program information from the SI management unit 232 as a response to the information request signal. The additional content processing unit 342 generates program guide information by arranging the broadcaster name, the channel number, and the program information for each program in the order of the broadcast time for each channel indicated by the channel information. The additional content processing unit 342 outputs the generated program guide information to the video compositing unit 334. As a result, the program guide information is displayed on the display unit. When any program information whose current time is within the broadcast time is instructed, the operation control unit may select a channel of the program. More specifically, the operation control unit outputs an operation control signal indicating the broadcast channel on which the program is broadcast to the broadcast receiving unit 12. Further, when the recording of a program related to any of the program information is instructed, the operation control unit may make a recording reservation for the program. More specifically, the descramble unit 206 outputs the first multiplexed data to the local encryption unit 208 at the program start time of the program in response to the instruction of the operation control signal from the operation control unit, and stores the TLV. Record in section 210. Further, the descramble unit 206 stops the output of the first multiplexed data to the local encryption unit 208 at the program end time, and stops the recording in the TLV storage unit 210. Since recording / playback using the first multiplexed data recorded in the TLV storage unit 210 is executed with reference to the reference time, in the present embodiment, the program start time related to the recording start and the program end time related to the recording end are corrected. It does not have to be.

Next, the execution of the broadcasting application will be described. The additional content processing unit 342 acquires multimedia data stored in the HTML management unit 238 as a broadcasting application based on the latest application control information received from the SI management unit 232. Therefore, the additional content processing unit 342 identifies the HTML management unit 238 as the acquisition destination information described in the application control information, and outputs an information request signal indicating the multimedia data to the HTML management unit 238. The multimedia data output from the HTML management unit 238 is input to the additional content processing unit 342 as a response. Therefore, when multimedia data is specified as the application to be executed in the application control information, the analysis unit 222 describes the address of the HTML management unit 238 in the application control information as information indicating the location of the multimedia data. Then, the analysis unit 222 outputs the application control information to the SI management unit 232.

The additional content processing unit 342 controls the execution of the acquired application based on various instructions described in the application control information. Such an instruction includes information such as a start instruction and an end. These instructions may specify an execution time for executing the instruction (that is, an application start time, end time, content presentation time, and the like). When the specified execution time is a time based on the reference time, the additional content processing unit 342 adds the above-mentioned time difference to the execution time, and calculates the time shifted by the time difference from the execution time as the correction execution time. .. When the correction execution time comes, the additional content processing unit 342 executes the process instructed by the command. For example, the additional content processing unit 342 extracts the element content included in the application or acquires the element content associated with the application as the process instructed by the instruction described in the application. Then, the additional content processing unit 342 calculates the corrected presentation time by adding the time difference to the presentation time. When the acquired element content includes visual information, the additional content processing unit 342 outputs the visual information to the video composition unit 334 when the current time reaches the correction presentation time. Therefore, the visual information and the broadcast image are displayed synchronously on the display unit at the presentation time based on the system time. When the acquired additional content includes auditory information, the additional content processing unit 342 outputs the auditory information to the audio mixing unit 338 when the current time reaches the correction presentation time. Therefore, on the display unit, the auditory information and the broadcast sound are reproduced in synchronization with the presentation time based on the system time.

In addition, element content that is a component of additional content presented by multimedia data may be stored in a device (for example, a web server) connected to a network. In that case, the address of the device is described in the multimedia data as the acquisition destination information of the element content. The additional content processing unit 342 transmits an information request signal indicating the element content to the device specified by the acquisition destination information described in the multimedia data via the communication unit 324. The additional content processing unit 342 receives the element content transmitted from the device specified in the acquisition destination information as a response to the information request signal.

As described above, the receiving device 1 according to the present embodiment is from the first multiplexed data multiplexed by the first multiplexing method (for example, the TLV / MMT method) to the audio data which is a component of the broadcasting service. , The separation unit 220 for separating the video data, the audio data, and the additional content data added to the video data. Further, the receiving device 1 is a multiplexing unit 230 that generates the second multiplexing data by multiplexing the audio data and the video data by a second multiplexing method (for example, MPEG-2 TS method) as the first multiplexing method. The output unit 240 and the input / output unit 242 are provided as output units for outputting the second multiplexed data and the additional content data, respectively.
Further, the receiving device 1 separates the audio data and the video data from the second multiplexing data by a separation method corresponding to the second multiplexing method, and the additional content based on the additional content data and at least the video or audio data indicated by the video data. A reception processing unit 30 for superimposing any of the voices indicated by is provided.
Here, the additional content is, for example, characters, multimedia content related to a broadcasting service provided by multimedia data, and the like.

With this configuration, the additional content data separated from the first multiplexing data is provided independently of the second multiplexing data based on the second multiplexing method. Therefore, in addition to the function that the reception processing unit provides audio and video from the second multiplexing data based on the second multiplexing method, additional content that does not support the second multiplexing method independently of the second multiplexing data. Can be provided. Therefore, it is possible to provide various broadcasting services other than audio and video.

Further, the separation unit 220 separates the reference time information indicating the reference time from the first multiplexed data, and the reception processing unit 30 detects the time difference between the system time and the reference time and detects the presentation time of the additional content. The additional content is output at the correction presentation time, which is the time corrected by the time difference.
With this configuration, the presentation time of the additional content based on the reference time in the first multiplexed data is corrected to the presentation time based on the system time as the correction presentation time. Therefore, the timing at which the additional content is presented can be synchronized with the timing of the audio or video presented with reference to the system time.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. In the following description, the differences of the first embodiment will be mainly used. The same components as those in the first embodiment are designated by the same reference numerals and the description thereof will be incorporated.

First, the schematic configuration of the receiving device 1 according to the present embodiment will be described with reference to FIG. In the receiving device 1, the analysis unit 222 specifies the reference start time indicated by the additional identification information in addition to the system time information and the reference time information from the control information separated from the first multiplexed data. The analysis unit 222 calculates the time difference of the system time indicated by the system time information from the reference time indicated by the reference time information, and adds the calculated time difference to the reference start time to calculate the correction reference start time 1. The analysis unit 222 generates configuration information including information indicating audio data and video data as a component of the service, and presentation time information indicating a correction reference start time 1 as a presentation time of character data. The analysis unit 222 outputs the generated configuration information to the multiplexing unit 230. Therefore, the presentation time information is also multiplexed in the second multiplexing data generated by the multiplexing unit 230. The correction reference start time 1 is a reference start time based on the system time at the time of reception. The receiving device 1 includes a TS storage unit 250 (FIG. 3) that stores (records) the second multiplexing data in the MPEG-2 TS format generated by the multiplexing unit 230. The stored second multiplexed data includes system time information indicating the system time (recording time) at that time.

When an operation control signal instructing reproduction is input from the operation control unit (not shown), the TS storage unit 250 reads out the second multiplexed data instructed by the operation control signal and outputs it to the separation unit 330. The separation unit 330 separates the presentation time information in addition to the system time information from the second multiplexed data, and outputs the separated system time information and the presentation time information to the synchronization processing unit 340.
The synchronization processing unit 340 calculates the time difference of the recording time indicated by the system time information input from the separation unit 330 from the current time based on the system time at that time. The synchronization processing unit 340 adds the calculated time difference to the correction reference start time 1 indicated by the presentation time information input from the separation unit 330, and further calculates the corrected correction reference start time 2. The correction reference start time 2 is a reference start time based on the system time at that time. The synchronization processing unit 340 outputs the presentation time information indicating the correction reference start time 2 to the additional content processing unit 342.

The additional content processing unit 342 reads the character data stored at the same time as the second multiplexed data acquired by the separation unit 330 from the TTML management unit 236. Here, the additional content processing unit 342 outputs an information request signal instructing the character data to the TTML management unit 236, and the character data instructed as a response is input. The additional content processing unit 342 sets the subtitle data consisting of the character string indicated by the input character data to the correction reference start time 2 indicated by the presentation time information input from the synchronization processing unit 340 with the current time based on the reference time. When it becomes, it is output to the image composition unit 334. Therefore, the recorded / reproduced video and the subtitles are reproduced in synchronization with respect to the system time.

Next, an example of the synchronous processing according to the present embodiment will be described.
FIG. 3 is an explanatory diagram for explaining an example of the synchronization process according to the present embodiment.
The conversion processing unit 20 (FIG. 2) includes a clock generation unit 224, and the reception processing unit 30 (FIG. 2) includes a clock generation unit 344.

In the conversion processing unit 20, the analysis unit 222 separates the MH-TOT indicating the system time information and the MPT indicating the configuration information from the MMT-SI indicating the control information separated from the first multiplexed data. (R02: MMT analysis) Further, the reference time information in the NTP format separated from the separation unit 220 is input to the analysis unit 222.
The analysis unit 222 extracts the reference start time (reference_start_time) from the additional identification information described in the MH-data coding method descriptor from the MPT separated from the first multiplexed data. (R04)

The analysis unit 222 measures the system time as the current time based on the separated system time information MH-TOT, and generates system time information indicating the timed system time (R06).
The analysis unit 222 calculates the time difference of the system time from the reference time which is the current time indicated by the reference time information. (R08)
The analysis unit 222 adds the calculated time difference to the reference start time to calculate the correction reference start time 1. (R10)
The analysis unit 222 generates a PMT indicating the configuration information in which the calculated correction reference start time 1 is multiplexed as the reference start time (reference_start_time), and outputs the generated PMT to the multiplexing unit 230 (FIG. 2). (R12)

The analysis unit 222 multiplies the frequency (for example, 1 Hz) of the reference time information in the NTP format and converts it into the reference time information indicating the reference time at a predetermined frequency (for example, 10 Hz). (R20: Frequency conversion)
The analysis unit 222 reconstructs the converted reference time information as PCR indicating the reference time information in the format specified in the MPEG-2 TS method. The analysis unit 222 outputs the reconstructed PCR to the multiplexing unit 230 (FIG. 2) and the clock generation unit 224. (R22)

The multiplexing unit 230 (FIG. 2) multiplexes control information including audio data, video data, PCR and PMT input from the separating unit 220 (FIG. 2) by the multiplexing method specified by MPEG-2 TS. Generate the second multiplexed data. (R24: TS conversion)
The multiplexing unit 230 stores the generated second multiplexing data in the TS storage unit 250 when an operation control signal instructing recording is input from the operation control unit (not shown). When recording is not instructed, the multiplexing unit 230 outputs the generated second multiplexing data to the separation unit 330 of the reception processing unit 30. When an operation control signal instructing reproduction is input from the operation control unit (not shown), the TS storage unit 250 receives the second multiplexed data of the section instructed by the operation control signal in the operation processing unit 30. Output to the separation unit 330.

The clock generator 224 includes a comparator, an oscillator, and an STC counter. The comparator, oscillator and STC counter form a PLL (Phase Locked Loop) circuit.
The comparator latches the counter pulse input from the STC counter at the timing when the PCR is input from the analysis unit 222, and compares the phases of the PCR and the counter pulse. The comparator outputs a comparison error signal indicating the phase difference to the oscillator. (R26)

The oscillator generates a clock signal having a predetermined clock frequency (for example, 27 MHz) based on the comparison error signal input from the comparator. The clock frequency is controlled to be constant by the comparison error signal. The oscillator outputs the generated clock signal to the STC counter. (R28)
The STC counter divides the clock signal input from the oscillator by a predetermined frequency division factor to generate a clock pulse. The STC counter outputs the generated clock pulse to the comparator. (R30)

In the reception processing unit 30, the separation unit 330 analyzes the TS forming the second multiplexed data input from the conversion processing unit 20 or the TS storage unit 250, and performs PCR and control showing audio data, video data, and reference time information. Separate into information. The separation unit 330 outputs the separated PCR and control information to the synchronization processing unit 340. (P02: TS analysis)
The synchronization processing unit 340 extracts the reference start time (reference_start_time) indicating the correction reference start time 1 from the PMT from the control information input from the separation unit 330. (P04)

The switch of the synchronization processing unit 340 does not input the operation control signal instructing recording / playback, and has a TOT indicating the system time information included in the control information input from the separation unit 330 during normal playback (live). It is supplied to the timer provided in the unit. The time indicated by the system time information supplied to the timer is the current time based on the system time. On the other hand, the switch of the synchronization processing unit 340 itself has a TOT indicating system time information included in the control information input from the separation unit 330 when an operation control signal instructing recording / playback is input (during recording / playback). It is supplied to the subtractor provided in the unit. The time indicated by the system time information supplied to the subtractor is the recording time based on the system time. (P06)
The timer sequentially clocks the current time based on the time indicated by the input system time information, and outputs the system time information indicating the measured current time to the subtraction unit. (P08) System time information may be input to the timer at least once. The timer continues to measure and output the current time even after the system time information is no longer input, such as during recording.

When the system time information indicating the recording time and the system time information indicating the current time are input from the switch, the subtractor of the synchronization processing unit 340 subtracts the current time from the recording time to record the recording time. The time difference of the current time is calculated from, and the time difference information indicating the calculated time difference is output to the adder. (P10) However, the subtractor of the synchronization processing unit 340 does not output the time difference information when the system time information is not input from the timer.
The adder of the synchronization processing unit 340 calculates the correction reference start time 2 by adding the time difference indicated by the time difference information input from the subtractor to the correction reference start time 1 described as the reference start time (reference_start_time). The adder outputs the reference start time (reference_start_time) describing the calculated correction reference start time 2 to the additional content processing unit 342 (FIG. 2) as the presentation time information. (P12) When the time difference information is not input from the subtractor, the adder outputs the reference start time (reference_start_time) input to the adder as it is to the additional content processing unit 342 as the presentation time information.

The clock generation unit 344 of the reception processing unit 30 includes a comparator, an oscillator, and an STC counter, and forms a PLL circuit, similarly to the clock generation unit 224 of the conversion processing unit 20. With this configuration, the clock generation unit 344 generates a clock signal synchronized with the PCR input from the separation unit 330. The clock signal generated by the clock generation unit 344 is used to synchronize the processing between the components of the reception processing unit 30 (clock synchronization). Therefore, the conversion processing unit 20 and the reception processing unit 30 synchronize their mutual operations with reference to the reference time.

Therefore, the receiving device 1 according to the present embodiment can display the subtitles at the time when the presentation time instructed at the time of recording is corrected as the time based on the system time at that time during recording / playback. In addition, the receiving device 1 can display subtitles at a time corrected by using the presentation time instructed at the time of reception as a time based on the system time at that time during normal playback. In both cases, the video provided with reference to the system time in the second multiplexed data multiplexed by the MPEG-2 TS method and the second multiplexed data are provided in parallel with reference to the reference time. Synchronize with the subtitles. Further, since the presentation time information indicating the reference start time related to the display of the subtitle is in a format common to the MMT method and the MPEG-2 TS method, the processing amount related to the change of the presentation time information and the processing related to the presentation of the additional content. The amount does not become excessive.

As described above, the receiving device 1 according to the present embodiment has audio data, video data, audio data, and video which are components of the broadcasting service from the first multiplexing data multiplexed by the first multiplexing method. A separation unit 220 is provided for separating additional content data relating to additional content added to the data, reference time information indicating a reference time, system time information indicating the system time, and presentation time information indicating the presentation time of the additional content. Further, the receiving device 1 includes an analysis unit 222, a multiplexing unit 230, an output unit 240 as an output unit, and an input / output unit 242. The analysis unit 222 calculates the corrected presentation time obtained by correcting the presentation time by the time difference between the reference time and the system time. The multiplexing unit 230 multiplexes the audio data, the video data, and the correction presentation time information indicating the correction presentation time by a second multiplexing method different from the first multiplexing method to generate the second multiplexing data. The output unit 240 outputs the second multiplexed data, and the input / output unit 242 outputs the additional content data.
Further, the receiving device 1 separates the audio data, the video data, and the correction presentation time information from the second multiplexing data by a separation method corresponding to the second multiplexing method, and adds content to the correction presentation time indicated by the correction presentation time information. A reception processing unit 30 for outputting additional contents related to data is provided.

According to this configuration, for the additional content whose presentation time can be specified in the control information of the first multiplexing method and the second multiplexing method, the presentation time based on the reference time is the corrected presentation time based on the system time. Is specified in the control information of the second multiplexing method. Therefore, the corrected presentation time, which is the time for presenting the additional content, can be synchronized with the timing based on the system time at which the audio or video is presented. Further, since it is not necessary for the reception processing unit 30 to correct the presentation time information, it is possible to reduce changes in the specifications and settings of the reception processing unit 30.

Further, the receiving device 1 includes a TS storage unit 250 that stores the second multiplexed data generated by the multiplexing unit 230, and a TTML management unit 236 and an HTML management unit 238 as storage units that store additional content data. When the second multiplexed data and the additional content data are input from the storage unit, the reception processing unit 30 calculates the second time difference, which is the time difference between the time when the second multiplexed data is stored and the current time, and presents the correction. The second correction presentation time is calculated by setting the time by the second time difference. Then, the reception processing unit 30 outputs the additional content related to the additional content data input from the storage unit at the second correction presentation time.
According to this configuration, when recording and playing back the audio or video related to the second multiplexed data and the additional content related to the additional content data, when recording at the correction presentation time based on the system time at the time of storage (recording). The second correction presentation time corrected by the time difference between the time and the current time is calculated. The second correction presentation time corresponds to the current time based on the system time at that time corresponding to the presentation time based on the system time at the time of recording / playback. Therefore, the second correction presentation time, which is the time for presenting the additional content to be recorded and reproduced, can be synchronized with the timing at which the audio or video related to the recording and reproduction is presented.

[Third Embodiment]
Next, a third embodiment of the present invention will be described. In the following description, the differences of the second embodiment will be mainly used. The same components as those in the above-described embodiment are designated by the same reference numerals and the description thereof will be incorporated.

In the second embodiment, the analysis unit 222 adds the time difference of the system time from the reference time to the reference start time to calculate the correction reference start time 1, and secondly multiplexes the presentation time information indicating the correction reference start time 1. It was multiplexed into the converted data.
On the other hand, in the present embodiment, the analysis unit 222 generates the second multiplexed data in which the reference start time extracted from the control information of the first multiplexed data is multiplexed as it is. The receiving device 1 further includes a time difference information storage unit 252. When the operation control signal instructing recording is input, the analysis unit 222 stores the time difference information indicating the time difference between the reference time and the system time in the time difference information storage unit 252 in association with the second multiplexed data. Further, when the normal reproduction is set, the analysis unit 222 outputs the time difference information to the reception processing unit 30 in association with the second multiplexed data.

Further, in the second embodiment, the synchronization processing unit 340 calculates the time difference of the recording time from the current time, and calculates it at the correction reference start time 1 indicated by the presentation time information input from the separation unit 330 (FIG. 2). The presentation time information indicating the correction reference start time 2 obtained by adding the time difference was generated.
On the other hand, in the present embodiment, the synchronization processing unit 340 calculates the time difference of the recording time from the current time, and adds the calculated time difference to the reference start time indicated by the presentation time information input from the separation unit 330. The correction reference start time 3 is calculated. On the other hand, when the operation control signal instructing recording / playback is input, the synchronization processing unit 340 stores the time difference information stored in association with the second multiplexed data of the section instructed by the operation control signal in the time difference information storage unit. Remember from 252. Further, when the operation control signal instructing the recording / playback is not input and the normal playback is set, the time difference information is input to the synchronization processing unit 340 in association with the second multiplexed data input to the separation unit 330. NS. The synchronization processing unit 340 adds the time difference indicated by the acquired time difference information to the correction reference start time 3, and calculates the correction reference start time 4, which is the time when the time difference is further shifted from the correction reference start time 3. The synchronization processing unit 340 outputs the presentation time information indicating the correction reference start time 4 to the additional content processing unit 342.

The additional content processing unit 342 acquires character data stored at the same time as the second multiplexed data input to the separation unit 330, and uses the subtitle data consisting of the character string indicated by the acquired character data as a reference time. When the current time to be determined reaches the correction reference start time 4 indicated by the presentation time information input from the synchronization processing unit 340, the data is output to the image composition unit 334. Even with such a configuration, the recorded / reproduced video and the subtitles are reproduced in synchronization with respect to the system time.

Next, an example of the synchronous processing according to the present embodiment will be described.
FIG. 4 is an explanatory diagram for explaining an example of the synchronization process according to the present embodiment.
Also in this embodiment, the conversion processing unit 20 (FIG. 2) includes a clock generation unit 224, and the reception processing unit 30 (FIG. 2) includes a clock generation unit 344.

However, among the processing blocks R04, R06, R08, R10, R12, R20 and R22 executed by the analysis unit 222, the processing blocks R10 and R12 are omitted.
Here, the analysis unit 222 multiplexes the PMT as it is without correcting the time indicated by the presentation time information indicating the reference start time (reference_start_time) separated from the first multiplexed data. Then, the multiplexing unit 230 multiplexes the PMT to generate the second multiplexing data. (R24: TS conversion)

When the operation control signal instructing recording is input, the analysis unit 222 associates the time difference information (NPT_Diff) indicating the time difference calculated by the processing block R08 with the second multiplexed data, and the time difference information storage unit 252. Remember in. When the operation control signal is not input and normal playback is set, the analysis unit 222 receives the time difference information (NPT_Diff) indicating the time difference calculated by the processing block R08 in association with the second multiplexed data. Output to unit 30.

Further, as the processing block executed by the synchronous processing unit 340, the processing block P14 is added in addition to the processing blocks P04, P06, P08, P10 and P12.
When the operation control signal instructing the recording / reproduction is input, the synchronization processing unit 340 reads the time difference information corresponding to the second multiplexed data of the section instructed by the operation control signal from the time difference information storage unit 252.
When the operation control signal is not input and normal playback is set, the time difference information corresponding to the second multiplexed data input to the separation unit 330 (FIG. 2) is input to the synchronization processing unit 340. ..
The synchronization processing unit 340 calculates the correction reference start time 4 by adding the time difference indicated by the acquired time difference information to the correction reference start time 3 described as the reference start time output from the processing block P12. (P14) The synchronization processing unit 340 outputs the reference start time describing the referenced correction reference start time 4 to the additional content processing unit 342 as presentation time information.

Therefore, the receiving device 1 according to the present embodiment displays the subtitles at the correction reference start time 4 in which the presentation time instructed at the time of recording is corrected to the time based on the system time at that time during recording / playback. Can be done. Further, during normal reproduction, the receiving device 1 can display subtitles at the correction reference start time 4 in which the presentation time instructed at the time of reception is corrected to a time based on the system time. Further, in the present embodiment, the conversion processing unit 20 does not need to change the presentation time information corresponding to the reference start time related to the display of the subtitle each time. Therefore, the amount of processing related to the change of the presentation time information does not become excessive.

As described above, the receiving device 1 according to the present embodiment includes audio data, video data, audio data, and description, which are components of the broadcasting service from the first multiplexing data multiplexed by the first multiplexing method. The separation unit 330 separates the additional content data related to the additional content added to the video data, the reference time information indicating the reference time, the system time information indicating the system time, and the presentation time information indicating the presentation time of the additional content. Further, the receiving device 1 includes an analysis unit 222, a multiplexing unit 230, an output unit 240 as an output unit, an input / output unit 242 or a TS storage unit 250, and a time difference information storage unit 252.
Further, the receiving device 1 separates the audio data, the video data, and the presentation time information from the second multiplexing data by a separation method corresponding to the second multiplexing method, and the time difference information indicates the presentation time indicated by the presentation time information. A reception processing unit 30 is provided which calculates the correction presentation time by correcting with the above and outputs the additional content related to the additional content data at the correction presentation time.

According to this configuration, for the additional content whose presentation time can be specified in the control information of the first multiplexing method and the second multiplexing method, the reference time transmitted by the control information of the second multiplexing method is set as the reference time. The correction presentation time corrected by the time difference between the time and the system time can be obtained. The reception processing unit 30 can have the additional content presented at the corrected presentation time based on the system time. Therefore, the corrected presentation time, which is the time when the additional content is presented, can be synchronized with the timing based on the system time when the audio or video is presented. Further, since it is not necessary to correct the presentation time information included in the control information of the first multiplexing method, it is possible to avoid an increase or setting of the processing amount related to the correction.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to the above-described embodiments, and includes designs and the like within a range that does not deviate from the gist of the present invention. The configurations described in the above embodiments can be arbitrarily combined.

For example, in the second and third embodiments, the case where the additional content data is mainly character data and the time corresponding to the presentation time information is the reference start time is taken as an example, but the present invention is not limited to this. The additional content data may be multimedia data. In that case, various execution times (for example, start time, end time, presentation time, etc.) described in the application control information or the application may be applied as the time corresponding to the presentation time information. However, the broadcast time related to the program guide display application does not have to be corrected as the above-mentioned presentation time information.
Further, among the character data, the character super is not required to be synchronized with the broadcast video or audio. Therefore, when the reference start time indicates the presentation time of the character super, it is subject to correction as the above-mentioned presentation time information. It does not have to be.
The additional content is not limited to visual information such as characters and images, but may be auditory information such as voice, music, and sound effects, or a combination of visual information and auditory information.

Further, a part of the configuration of the receiving device 1 may be omitted, or another configuration may be added. For example, in the receiving device 1, any one of the broadcast receiving unit 12, the demodulating unit 14, the display unit (not shown) and the reproducing unit (not shown), or any combination thereof may be omitted. The receiving device 1 may be configured as a conversion device by omitting the receiving processing unit 30, the display unit and the reproducing unit. The broadcast receiving unit 12 and the demodulating unit 14 may be further omitted from this conversion device.
Further, in the conversion processing unit 20, the local encryption unit 208, the TLV storage unit 210, and the local decryption unit 212 may be omitted.

In the second and third embodiments, the TS storage unit 250 may have a configuration independent of the conversion processing unit 20 and the reception processing unit 30, or may be a part of the conversion processing unit 20 or the reception processing unit 30. It may be configured.
Further, in the third embodiment, the time difference information storage unit 252 may have a configuration independent of the conversion processing unit 20 and the reception processing unit 30, or may be a part of the conversion processing unit 20 or the reception processing unit 30. It may be configured. The TS storage unit 250 may be integrally configured with the time difference information storage unit 252, or may be a separate body.

The invention described above can also be carried out in the following aspects.
(1) Separation unit that separates audio data, video data, and additional content data added to the audio data and video data, which are components of the broadcasting service, from the first multiplexing data multiplexed by the first multiplexing method. A multiplexing unit that generates the second multiplexing data by multiplexing the audio data and the video data by a second multiplexing method different from the first multiplexing method, the second multiplexing data, and the above. A receiving device including an output unit for outputting additional content data.

(2) The audio data and the video data are separated from the second multiplexing data by a separation method corresponding to the second multiplexing method, and the additional content based on the additional content data and at least the video indicated by the video data or The receiving device of (1) including a receiving processing unit that superimposes any of the voices indicated by the voice data.

(3) The separation unit further separates the reference time information indicating the reference time from the first multiplexed data, and the reception processing unit detects the time difference between the system time and the reference time to obtain the additional content. The receiving device of (2) that presents the additional content at a time obtained by correcting the presentation time by the time difference.

(4) The separation unit further separates the channel information related to the broadcast channel and the program information related to the program broadcast on the broadcast channel from the first multiplexing data, and the reception processing unit further separates the program for each broadcast channel. The receiving device according to (2) or (3), which synthesizes the video with the program guide information indicating the information.

(5) The receiving device according to any one of (1) to (4), wherein the additional content data includes multimedia data for reproducing content related to the broadcasting service.

(6) The receiving device according to any one of (1) to (5), wherein the additional content data includes character data indicating characters.

(7) The first multiplexing method is a Type-length-value / MPEG Media Transport method, and the second multiplexing method is an MPEG-2 Transport Stream method, which is any one of (1) to (6). Receiver.

(8) A receiving method in a receiving device, wherein the receiving device uses the first multiplexing data multiplexed by the first multiplexing method to obtain audio data, video data, and the audio data which are components of a broadcasting service. A separation step for separating the additional content data added to the video data, and multiplexing of the audio data and the video data by a second multiplexing method different from the first multiplexing method to generate the second multiplexing data. A receiving method including a multiplexing step for outputting the second multiplexing data and an output step for outputting the additional content data, respectively.

(9) Audio data, video data, and additional content added to the audio data and video data, which are components of the broadcasting service, from the first multiplexing data multiplexed by the first multiplexing method on the computer of the receiving device. Separation procedure for separating data, multiplexing procedure for generating second multiplexing data by multiplexing the audio data and the video data by a second multiplexing method different from the first multiplexing method, the second multiplexing. A program for executing an output procedure for outputting the conversion data and the additional content data, respectively.

Further, a program for realizing the functions of the conversion processing unit 20 and the reception processing unit 30 described above is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into the computer system and executed. As a result, the conversion processing unit 20 and the reception processing unit 30 may be realized. Although the program is a program separate from the application program described in the above-described embodiment, some functions thereof may be realized based on the application program. Here, "loading and executing a program recorded on a recording medium into a computer system" includes installing the program in the computer system. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. Further, the "computer system" may include a plurality of computer devices connected via a network including a communication line such as the Internet, WAN, LAN, and a dedicated line.

Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, or a storage device such as a hard disk built in a computer system. As described above, the recording medium in which the program is stored may be a non-transient recording medium such as a CD-ROM. The recording medium also includes an internal or external recording medium that can be accessed from the distribution server to distribute the program. The code of the program stored in the recording medium of the distribution server may be different from the code of the program in a format that can be executed by the terminal device. That is, the format stored in the distribution server does not matter as long as it can be downloaded from the distribution server and installed in a form that can be executed by the terminal device.

The program may be divided into a plurality of parts, downloaded at different timings, and then integrated with the terminal device, or the distribution server for distributing each of the divided programs may be different. Furthermore, a "computer-readable recording medium" is a volatile memory (for example, RAM) inside a computer system that serves as a server or client when a program is transmitted via a network, and holds the program for a certain period of time. It shall also include those that are. Further, the above program may be for realizing a part of the above-mentioned functions. Further, it may be a so-called difference file (difference program) that can realize the above-mentioned function in combination with a program already recorded in the computer system.

1 ... Receiver, 12 ... Broadcast receiver, 14 ... Demodulation unit, 20 ... Conversion processing unit, 30 ... Reception processing unit, 202 ... Input unit, 204 ... Decryption unit, 206 ... Desk rumble unit, 208 ... Local encryption unit , 210 ... TLV storage unit, 212 ... local decoding unit, 214 ... conversion unit, 220 ... separation unit, 222 ... analysis unit, 230 ... multiplexing unit, 232 ... SI management unit, 234 ... NTP management unit, 236 ... TTML management Unit, 238 ... HTML management unit, 240 ... output unit, 242 ... input / output unit, 250 ... TS storage unit, 252 ... time difference information storage unit, 320 ... input unit, 322 ... input / output unit, 324 ... communication unit, 330 ... Separation unit, 332 ... Video decoding unit, 334 ... Video synthesis unit, 336 ... Audio decoding unit, 338 ... Audio mixing unit, 340 ... Synchronous processing unit, 342 ... Additional content processing unit

Claims (7)

A separation unit that separates audio data, video data, and additional content data added to the audio data and video data, which are components of the broadcasting service, from the first multiplexing data multiplexed by the first multiplexing method.
A multiplexing unit that generates the second multiplexing data by multiplexing the audio data and the video data by a second multiplexing method different from the first multiplexing method.
An output unit that outputs the second multiplexed data and the additional content data, respectively.
The audio data and the video data are separated from the second multiplexing data by a separation method corresponding to the second multiplexing method.
A reception processing unit that superimposes the additional content based on the additional content data and at least either the video indicated by the video data or the audio indicated by the audio data.
Equipped with a,
The separation part
The reference time information indicating the reference time is further separated from the first multiplexed data, and the reference time information is further separated.
The reception processing unit
Detects the time difference between the system time and the reference time,
The additional content is presented at a time obtained by correcting the presentation time of the additional content by the time difference.
RECEIVER. The separation part
The channel information related to the broadcast channel and the program information related to the program broadcast on the broadcast channel are further separated from the first multiplexed data.
The reception processing unit
The receiving device according to claim 1 , wherein the program guide information indicating the program information for each broadcast channel and the video are combined. The receiving device according to claim 1 or 2 , wherein the additional content data includes multimedia data for reproducing content related to the broadcasting service. The receiving device according to any one of claims 1 to 3 , wherein the additional content data includes character data indicating characters. The first multiplexing method is a Type-length-value / MPEG Media Transport method.
The receiving device according to any one of claims 1 to 4 , wherein the second multiplexing method is an MPEG-2 Transport Stream method. It is a receiving method in the receiving device.
The receiving device
A separation step for separating audio data, video data, and additional content data added to the audio data and video data, which are components of the broadcasting service, from the first multiplexing data multiplexed by the first multiplexing method.
A multiplexing step of generating the second multiplexing data by multiplexing the audio data and the video data by a second multiplexing method different from the first multiplexing method.
An output step for outputting the second multiplexed data and the additional content data, respectively.
The audio data and the video data are separated from the second multiplexing data by a separation method corresponding to the second multiplexing method.
A reception processing step of superimposing the additional content based on the additional content data and at least either the video indicated by the video data or the audio indicated by the audio data.
The execution,
The separation step
The reference time information indicating the reference time is further separated from the first multiplexed data, and the reference time information is further separated.
The reception processing step
Detects the time difference between the system time and the reference time,
A receiving method for presenting the additional content at a time obtained by correcting the presentation time of the additional content by the time difference. To the computer of the receiving device
A separation procedure for separating audio data, video data, and additional content data added to the audio data and video data, which are components of the broadcasting service, from the first multiplexing data multiplexed by the first multiplexing method.
A multiplexing procedure for generating second multiplexing data by multiplexing the audio data and the video data by a second multiplexing method different from the first multiplexing method.
Output procedure for outputting the second multiplexed data and the additional content data, respectively.
The audio data and the video data are separated from the second multiplexing data by a separation method corresponding to the second multiplexing method.
A reception processing procedure that superimposes the additional content based on the additional content data and at least either the video indicated by the video data or the audio indicated by the audio data.
A program for executing,
The separation procedure
The reference time information indicating the reference time is further separated from the first multiplexed data, and the reference time information is further separated.
The reception processing procedure is
Detects the time difference between the system time and the reference time,
A program for presenting the additional content at a time obtained by correcting the presentation time of the additional content by the time difference.

Images