JP2010258785A - Multiplexed stream data reproducing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a multiplexed stream data reproducing device which can select the stream data serving as the reference of reproduction from among a plurality of stream data multiplexed as the multiplexed stream data. <P>SOLUTION: A reference clock request receiving unit 1 receives a designation of stream data which serves as the reference clock of reproduction. A demultiplexer 2 extracts the reference time information which is outputted to a reference clock reproduction unit 3 from the specified stream data based on the designation from the reference clock request receiving unit 1. The reference clock reproduction unit 3 reproduces the reference clock from the reference time information outputted from the demultiplexer 2. Decoders 4-1 to 4-n perform decoding based on the reference clock outputted from the reference clock reproduction unit 3. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a multiplexed stream data reproducing apparatus that reproduces multiplexed stream data.

Multiplexing of stream data is to combine a plurality of stream data into one stream data that can be synchronously reproduced. Familiar examples include video and audio multiplexing on a DVD (Digital Versatile Disk) and video / audio / data multiplexing on digital television broadcasts. Conventional methods for multiplexing such stream data include the following.
(1) MPEG system type (2) AVI / ASF type MPEG: Motion Pictures Experts Group
AVI: Audio Video Interface
ASF: Advanced Systems Format

  The method (1) is based on an MPEG system that is internationally standardized as a multimedia data multiplexing method (see, for example, Patent Document 1 and Non-Patent Document 1). In this method, each stream data to be multiplexed is finely divided in units of PES (Packetized Elementary Stream), and these are arranged in order of time, thereby forming multiplexed stream data. In the multiplexed stream data, reference time information (PCR: Program Clock Reference and SCR: System Clock Reference) is included, and on the playback device side, a high-accuracy reference clock is reproduced in the playback device. Can do. When the multiplexed stream data is reproduced, the reference clock is compared with a time stamp (PTS: Presentation Time Stamp and DTS: Decoding Time Stamp) included in the PES to determine the reproduction timing of each PES.

  On the other hand, the method (2) is a file format developed by Microsoft Corporation, and is a method frequently used for multiplexing stream data such as video data, audio data, and metadata in a personal computer or the like. (For example, see Non-Patent Document 2). In this system, multiplexed stream data is configured by finely dividing each stream data in units called chunks and arranging them in time order. There is header information in the multiplexed stream data, and the update frequency (reproduction frequency) for each stream data is described therein. When the multiplexed stream data is reproduced, the chunk is processed according to the reproduction frequency to reproduce the multiplexed stream data.

Japanese Patent No. 3849943

ISO / IEC 13818-1, ISO / IEC 14496-1 Specification, Revision 01.20.02, Microsoft Corporation, June, 2004, [Search on March 31, 2008], Internet URL <http://download.microsoft.com/download/e/0/6/e06db390-1e2a- 4978-82bb-311810d8a28d / ASF_Specification.doc>

  In the method (1), reproduction is performed using the reference clock as a pacemaker as described above. However, since the reference clock is attached to the entire multiplexed stream data, reproduction based on this clock can focus on specific stream data even if the entire stream data can be reproduced synchronously. The synchronized playback cannot be applied. For example, when stream data that exists only in intermittent time is included in the multiplexed stream data, the method (1) can only perform synchronous reproduction based on the reference clock attached to the entire multiplexed stream. Only the section in which intermittent stream data exists cannot be reproduced in synchronization with other stream data.

  In the case of the method (2), each stream data is reproduced based on a preset reproduction frequency. Therefore, it is necessary to arrange the chunks of the stream data in the multiplexed stream data so that they are reproduced according to the reproduction frequency. Therefore, unlike this method (1), this method cannot construct multiplexed stream data in which some stream data is missing on the way or the playback frequency such as the frame rate changes on the way. .

  As described above, in the reproduction of the conventional multiplexed stream data, the reproduction timing of the pattern and data that has already been reproduced is determined at the time when the multiplexed stream data is created. There was a problem that the playback timing could not be changed on the machine side. For this reason, synchronized playback cannot always be performed with a playback pattern intended by the application.

  The present invention has been made to solve the above-described problems, and a multiplexed stream that can select stream data to be a reference for reproduction from among a plurality of stream data multiplexed into the multiplexed stream data. An object is to obtain a data reproducing apparatus.

  The multiplexed stream data reproducing apparatus according to the present invention separates multiplexed stream data obtained by multiplexing a plurality of stream data into each stream data, and a reference time for synchronously reproducing each separated stream data A demultiplexer unit that outputs information, a reference clock reproduction unit that reproduces a reference clock from reference time information output from the demultiplexer unit, and each stream data separated by the demultiplexer unit is output from the reference clock reproduction unit A plurality of decoders for decoding based on a reference clock, a reference clock request receiving unit that receives designation of stream data to be a reference clock for reproduction, and transmits the designated stream data to the demultiplexer unit. Output to reference clock reproduction unit That the reference time information, is obtained so as to extract from the stream data transmitted from the reference clock request receiving unit.

  In the multiplexed stream data reproduction device of the monitoring control apparatus according to the present invention, the reference time information output from the demultiplexer unit to the reference clock reproduction unit is extracted from the designated stream data. It is possible to select stream data as a reference for reproduction from among a plurality of stream data multiplexed in the above.

It is a block diagram of the monitoring system using the multiplexed stream data reproducing | regenerating apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the timeline of the multiplexed stream data in the multiplexed stream data reproducing | regenerating apparatus by Embodiment 1 of this invention. It is a block diagram which shows the multiplexed stream data reproducing | regenerating apparatus by Embodiment 1 of this invention. It is a block diagram which shows the multiplexed stream data reproducing | regenerating apparatus by Embodiment 2 of this invention. It is explanatory drawing which shows the timeline of the multiplexed stream data in the multiplexed stream data reproduction apparatus by Embodiment 2 of this invention.

Embodiment 1 FIG.
FIG. 1 is a block diagram of a monitoring system using a multiplexed stream data reproducing apparatus according to Embodiment 1 of the present invention.
In the industrial monitoring field, multi-log recorders that simultaneously record data other than video in recorders that record camera video, such as CCTV systems, are beginning to be put into practical use. In FIG. 1, a multi-log recorder 104 that simultaneously records video data from the camera 101, audio data from the microphone 102, and measurement data from the sensor 103, and a playback device 105 and a playback device 105 that play back the recorded data. A system including a connected display device 106 and a speaker 107 is shown.

  In the multi-log recorder 104, the data such as the video data, the audio data, and the measurement data are input as time-series stream data, and are multiplexed and recorded so as to be synchronously reproduced by the reproducing apparatus 105. The multi-log recorder 104 and the playback device 105 are connected by a network or the like, and the playback device 105 can capture multiplexed stream data recorded by the multi-log recorder. The multiplexed stream data captured by the playback device 105 is decoded within the device, and is output to the display device 106 if it is video data or measurement data, and to the speaker 107 if it is audio data.

  However, compared to audio data and measurement data, video data has a large amount of data, so it is not always recorded. For example, video data is recorded only when there is movement, or it is recorded only during the day and not at night. Are often used. FIG. 2 shows a multi-log recorder that records video data 201, audio data 202, and measurement data 203. Audio data 202 is always recorded, video data is recorded only when there is movement, and measurement data is reacted by a sensor. This is an example in which multiplexed stream data is recorded on the timeline when only recording is performed. As shown in the figure, video data 201, audio data 202, and measurement data 203 are all recorded from time t1 to time t2. Only video data 201 and audio data 202 are recorded between times t2 and t3. Only the audio data 202 is recorded between the times t3 and t4, and the audio data 202 and the measurement data 203 are recorded between the times t4 and t5. When such a stream data group is multiplexed by the conventional MPEG system type, since the audio data 202 always exists between the times t1 and t5, the multiplexed stream data has a time from time t1 to t5. It becomes data with intervals. When this multiplexed stream data is reproduced, it takes a time of (t5-t1).

  On the other hand, consider a case where an application whose main purpose is to display sensor measurement data is built using such data. In such an application, the measurement data is mainly used, and it is natural that only the section in which the measurement data exists is reproduced. In addition, there are many requests to simultaneously confirm the video and audio at that time.

  However, in a playback device such as a conventional MPEG system, there is no option of playing back the multiplexed stream data as described above based on the measurement data, and the reference time information existing in the multiplexed stream data is also included. At the same time, the reference clock is reproduced, and reproduction is performed as the clock progresses. In the above case, the audio data is reproduced during (t5-t1) as a result, although it is desired to reproduce only the time (t2-t1) + (t5-t4). In the present invention, for such multiplexed stream data, stream data serving as a time reference is designated in advance, and other multiplexed stream data is also reproduced as the time of the stream data advances. The porpose is to do. For example, in FIG. 2, when the time of measurement data is designated as a reference, only a section with measurement data is reproduced, and a section without measurement data is reproduced with skipping. At this time, other multiplexed stream data is also reproduced in accordance with the progress of the time, or is reproduced by skipping. On the other hand, when audio data or a standard time (STC) in the MPEG system is designated as the time standard, the reproduction is performed over a period of (t5-t1) as in the normal MPEG system.

FIG. 3 is a configuration diagram of the multiplexed stream data reproducing apparatus according to the first embodiment.
The multiplexed stream data reproducing apparatus includes a reference clock request receiving unit 1, a demultiplexer (DMUX) unit 2, a reference clock reproducing unit 3, decoders 4-1 to 4-n, output units 5-1 to 5-n, and an interface. 6-1 to 6-n.
The reference clock request accepting unit 1 accepts designation of a reference clock from an application and transmits it to the DMUX unit 2. The DMUX unit 2 separates the input multiplexed stream data and transmits time information corresponding to the reference clock transmitted from the reference clock request receiving unit 1 to the reference clock reproduction unit 3. The reference clock reproduction unit 3 reproduces the reference clock from the reference time information output from the DMUX unit 2. The decoders 4-1 to 4-n correspond to the N stream data separated by the DMUX unit 2, respectively, and decode each stream data based on the reference clock from the reference clock reproduction unit 3. The output units 5-1 to 5-n output the decoded stream data output from the respective decoders 4-1 to 4-n to the interfaces 6-1 to 6-n corresponding to the respective stream data. . Further, the interfaces 6-1 to 6-n connect the respective stream data output from the output units 5-1 to 5-n to, for example, the display device 106 and the speaker 107 illustrated in FIG.

  Next, the operation of the present embodiment will be specifically described taking the multiplexed stream data described in FIG. 2 as an example. Here, it is assumed that the application is intended to reproduce measurement data as a subject. In such a case, the application instructs the reference clock request receiving unit 1 to use the measurement data as a time reference. The designated information is transmitted from the reference clock request accepting unit 1 to the DMUX unit 2, and the DMUX unit 2 extracts DTS information or PTS information in the PES including the measurement data and transmits it to the reference clock reproduction unit 3. The reference clock reproduction unit 3 reproduces the reference clock from the DTS information or PTS information. Each of the video data, audio data, and measurement data decoders 4-1 to 4-n receives the PES data output from the DMUX unit 2, receives the DTS information or PTS information contained in the PES data, and the reference clock reproduction unit 3. Are compared with the reference clock value reproduced in step (1), and the PES data having the same clock is decoded.

  At this time, the measurement data is not always included, but is included intermittently, and thus the DTS / PTS information included in the measurement data may be skipped during playback. In this case, the reference time is also skipped. For example, in the audio data decoder or the like, there is data that does not match the time information, but the data is discarded and is not decoded. Only the decoded stream data is output to each interface 6-1 to 6-n. As a result, video data, audio data, and measurement data are synchronously reproduced only at the time when the measurement data exists. For video data, audio data, and the like, a random access point may be determined depending on the encoding method. In this case, the reproduction may be performed from a previous random access point or from the next random access point.

  As described above, in the present embodiment, stream data serving as a reference for time can be selected, so that it is possible to support reproduction of patterns required by various applications for one multiplexed stream data. This has the effect of becoming

  In the present embodiment, it is needless to say that a large buffer is required between the DMUX unit 2 and the decoders 4-1 to 4-n when the measurement data does not readily appear. .

  In this embodiment, video data, audio data, and measurement data have been described as examples. However, the stream data to be multiplexed is not limited to these, and the present invention may include various stream data. Needless to say, this is possible.

  Furthermore, in the present embodiment, as an example, the measurement data is used as a time reference, but other stream data or a system time STC using conventional PCR / SCR can be used as a time reference. In addition, a unique time axis can be included in the multiplexed stream as one stream data. When such data is designated as a time reference from the reference clock request receiving unit 1, the data becomes a pacemaker and can be reproduced on its own time axis. For example, in a video surveillance system, nighttime data is not so important, and it may be convenient to have a function of fast-forward playback for the nighttime part. In such a case, since normal PCR / SCR is played back in real time, stream data having a PTS / DTS time that flows early at night is inserted, and a reference clock is set for this stream data. Specify to reproduce. By doing so, it becomes possible for time to flow early in the night portion and to be played back in real time in the daytime.

  In addition, it is possible to include the result of image processing as one stream data. When such data is designated as a time reference, reproduction following the result of image processing becomes possible. By doing so, it is possible to record the video itself at all times and reproduce only the time interval (for example, the interval in which the motion has occurred) in response to the image processing.

  As described above, according to the multiplexed stream data reproducing apparatus of the first embodiment, the multiplexed stream data obtained by multiplexing a plurality of stream data is separated into each stream data, and each separated stream A demultiplexer unit that outputs reference time information for synchronously reproducing data, a reference clock reproduction unit that reproduces a reference clock from the reference time information output from the demultiplexer unit, and a demultiplexer unit Accepts a plurality of decoders that decode stream data based on a reference clock output from the reference clock reproduction unit, and accepts a designation of stream data that is a reproduction reference clock, and accepts a reference clock request that conveys the designated stream data to the demultiplexer unit The demultiplexer unit is a reference Since the reference time information to be output to the lock reproduction unit is extracted from the stream data transmitted from the reference clock request receiving unit, reproduction from a plurality of stream data multiplexed into the multiplexed stream data is performed. The reference stream data can be selected.

Embodiment 2. FIG.
Embodiment 2 is an example in which stream data serving as a time reference can be selected while selecting a stream data type to be reproduced from stream data included in multiplexed stream data. For example, when the multiplexing method is an MPEG system, stream data can be selected by specifying a stream ID, so that stream data can be uniquely specified. By specifying a plurality of stream data, a stream data group to be synchronously reproduced is specified. be able to.

FIG. 4 is a configuration diagram of the multiplexed stream data reproducing apparatus according to the second embodiment.
The multiplexed stream data reproducing apparatus according to the second embodiment includes a reference clock request receiving unit 1, a DMUX unit 2a, a reference clock reproducing unit 3, decoders 4-1 to 4-n, output units 5-1 to 5-n, and an interface. 6-1 to 6-n and a stream data request receiving unit 7. Here, since the reference clock request accepting unit 1, the reference clock reproducing unit 3 to the interfaces 6-1 to 6-n are the same as the configuration of the first embodiment, the corresponding parts are denoted by the same reference numerals and the description thereof will be given. Is omitted. The stream data request receiving unit 7 has a function of receiving a stream data group to be reproduced from an application and transmitting it to the DMUX unit 2a. Further, in addition to the function of the DMUX unit 2 of the first embodiment, the DMUX unit 2a decodes only the stream data to be reproduced based on the stream data group to be reproduced instructed from the stream data request accepting unit 7-4. Has a function of outputting to 4-n.

Next, the operation of the multiplexed stream data reproducing apparatus according to the second embodiment will be described.
FIG. 5 shows an example in which recorded data is viewed on a timeline in a multi-log recorder that records video data (# 1) 301, video data (# 2) 302, audio data 303, and measurement data 304 in the present embodiment. is there. Video data (# 1) 301 is always recorded, and video data (# 2) 302 is intermittently recorded.

Here, the application in the present embodiment reproduces only the video data (# 1) 301, the video data (# 2) 302, and the audio data 303, and the video data (# 2) 302 is reproduced based on the time. I want to do it. In this case, the measurement data is unnecessary, so there is no need for reproduction. Therefore, the application requests the stream data request receiving unit 7 to reproduce only the video data (# 1) 301, the video data (# 2) 302, and the audio data 303. In addition, the application requests the reference clock request receiving unit 1 to reproduce the video data (# 2) 302 based on the time.
As a result, the DMUX unit 2a extracts the DTS information or PTS information in the PES including the audio data 303 and transmits it to the reference clock reproduction unit 3 as in the first embodiment. The reference clock reproduction unit 3 reproduces the reference clock from the DTS information or PTS information. In addition, the DMUX unit 2a sends only video data (# 1) 301, video data (# 2) 302 and audio data 303 separated from the multiplexed stream data to the corresponding decoders 4-1 to 4-n.

  The decoders 4-1 to 4-n corresponding to the video data (# 1) 301, the video data (# 2) 302, and the audio data 303 receive the PES data output from the DMUX unit 2a and are included in the PES data. The DTS information or PTS information to be checked and the reference clock value reproduced by the reference clock reproduction unit 3 are collated, and the PES data having the same clock is decoded. Thus, the video data (# 1) 301, the video data (# 2) 302, and the audio data 303 are output from the output units 5-1 to 5-n in synchronization with the audio data 303.

In the second embodiment, the data to be selected is video data and audio data, and the measurement data is not selected. Needless to say, the effect of the present invention does not depend on the stream data to be selected.
In the second embodiment, only stream data to be synchronously reproduced is transmitted from the DMUX unit 2a to the decoders 4-1 to 4-n. However, all stream data is transmitted from the DMUX unit 2a to the decoders 4-1 to 4-4. The stream data to be sent to -n and separately decoded may be designated.

  As described above, in the multiplexed stream data reproducing apparatus according to the second embodiment, a stream data request that receives designation of stream data to be reproduced from a plurality of multiplexed stream data and transmits this designation to the demultiplexer unit. In addition to the effects of the first embodiment, the demultiplexer unit outputs only the stream data transmitted from the stream data request reception unit to the decoder. There is an effect that it is possible to cope with the reproduction of patterns required by various applications with a small decoding load.

  1 reference clock request reception unit, 2 DMUX (demultiplexer) unit, 3 reference clock reproduction unit, 4-1 to 4-n decoder, 5-1 to 5-n output unit, 6-1 to 6-n interface, 7 Stream data request reception unit, 101 camera, 102 microphone, 103 sensor, 104 multi-log recorder, 105 playback device, 106 display device, 107 speaker.

Claims (2)

A demultiplexer unit that separates multiplexed stream data obtained by multiplexing a plurality of stream data into each stream data, and outputs reference time information for synchronously reproducing each separated stream data;
A reference clock reproduction unit that reproduces a reference clock from the reference time information output from the demultiplexer unit;
A plurality of decoders for decoding each stream data separated by the demultiplexer unit based on a reference clock output from the reference clock reproduction unit;
A reference clock request receiving unit that receives designation of stream data to be a reference clock for reproduction, and transmits the designated stream data to the demultiplexer unit;
The multiplexed stream data reproducing apparatus, wherein the demultiplexer unit extracts reference time information output to the reference clock reproduction unit from stream data transmitted from the reference clock request receiving unit. A stream data request receiving unit that receives designation of stream data to be reproduced from a plurality of multiplexed stream data and transmits the designation to the demultiplexer unit,
2. The multiplexed stream data reproducing apparatus according to claim 1, wherein the demultiplexer unit outputs only the stream data transmitted from the stream data request receiving unit to the decoder.

Images