JP4403403B2 - Stream multiplexing apparatus, stream recording apparatus, and stream multiplexing method - Google Patents

Description

  The present invention relates to a stream multiplexing apparatus that multiplexes a plurality of encoded streams, a stream recording apparatus including the same, and a stream multiplexing method, and in particular, operates independently of an encoder that generates an encoded stream. The present invention relates to a stream multiplexing apparatus capable of performing the above, a stream recording apparatus including the same, and a stream multiplexing method.

  In recent years, with the progress of image compression technology such as the MPEG (Moving Picture Experts Group) system, it has become common to digitize and handle image signals. For example, digital video recorders that record moving images as digital data on various recording media such as optical disks are widely used.

  In the MPEG method, compression-coded video and audio data (ES: Elementary Stream) is time-division multiplexed to generate one stream (PS: Program Stream). The PS has a hierarchical structure of packs and packets. The video ES and audio ES data is divided into video packets and audio packets, respectively, and a pack is constituted by the plurality of packets. In addition, time management information such as a decoding time and a reproduction time is recorded in the header of the packet, so that a stream can be synchronously reproduced at an intended timing on the encoder side.

FIG. 6 is a diagram showing a main configuration of a conventional stream generation apparatus using the MPEG method.
As shown in FIG. 6, the stream generation device includes a video encoder 101, an audio encoder 102, a video ES buffer 103, a video Info buffer 104, an audio ES buffer 105, an audio Info buffer 106, and a MUX (Multiplexer) 107.

  The video encoder 101 and the audio encoder 102 respectively compress and encode video data and audio data, and sequentially store the encoded ES data in the video ES buffer 103 and the audio ES buffer 105 in units of access units (AU). One access unit is composed of one frame or one field picture for video and an audio frame for audio.

  In addition, the video encoder 101 and the audio encoder 102 store the output information (AU_Info) relating to the AU in the video Info buffer 104 and the audio Info buffer 106, respectively. As AU_Info, position information on each AU Info buffer, data size, PTS (Presentation Time Stamp) that is management information of reproduction output time, DTS (Decoding Time Stamp) that is management information of decoding time, and the like are recorded. .

  The MUX 107 reads out data from the video ES buffer 103 and the audio ES buffer 105 while referring to the information of each Info buffer, stores the data in a video packet and an audio packet, respectively, multiplexes these packets into a pack, and PS. Complete and output. At this time, time management information (DTS, PTS) is recorded in the packet header including the head data of the AU, and SCR (System Clock Reference) which is a reference reference value of these times is recorded in the system header of the pack.

  By recording such information in the stream, the playback device refers to the SCR value and sets the reference time intended on the encoder side as a synchronization signal called STC (System Time Clock) inside the decoder. The video and audio can be synchronized and reproduced by controlling the decoding and reproduction output timing of the corresponding AU based on the STC value and the PTS and DTS.

In addition, for ES multiplexing, a standard decoder model called P-STD (Program-System Target Decoder) is defined, and it is necessary for the encoder side to be able to correctly decode the P-STD. There is. Specifically, the MUX 107 multiplexes while managing the code amount so that the P-STD buffer assumed on the encoder side does not overflow or underflow (see, for example, Patent Document 1). Furthermore, in order to absorb the difference in processing time between the scheduling process for buffer management and the data multiplexing process according to the above process, schedule information and additional data to be multiplexed are exchanged between the scheduler and the MUX. In some cases, an intermediate buffer is stored (for example, see Patent Document 2).
JP-A-8-335371 (paragraph numbers [0013] to [0017], FIG. 11) JP-A-10-276164 (paragraph numbers [0057] to [0076], FIG. 1)

  By the way, in a device such as a video recorder provided with the above-described stream generation device, many functions are required, and control processing tends to be gradually complicated. For this reason, for example, it is considered to implement control of individual functions such as an encoder and MUX as tasks on a real-time OS (Operating System) to improve the efficiency of development and verification of each function.

  However, when each control of the encoder and MUX is an individual task, the encoder operates at a timing determined according to input data, whereas the MUX operates at a timing based on the multiplexing rate. For this reason, even when there is no data to be multiplexed in the ES buffer, the MUX continues to operate regardless of this, and an abnormal situation occurs in which an unnecessary packet in which no data is stored is continuously output.

  For example, during the encoding of video data or audio data, input data to the encoder is basically not interrupted. For this reason, when the operation of the MUX is slow, when some trouble occurs in the MUX, or when the capacity of the ES buffer is insufficient, the encoded AU data cannot be stored in the ES buffer, or before the read Failures such as overwriting data occur. Since such a failure is fatal to the system, in order to prevent its occurrence, the capacity of the ES buffer is increased, or data is extracted from the ES buffer at a sufficient speed to generate a multiplexed stream. Thus, it is necessary to design the MUX.

  However, when the MUX reading speed is increased, there is a possibility that a state in which there is no AU to be multiplexed exists in the ES buffer due to a variation in time required for the encoding process for each AU. In the conventional MUX, such a state is detected by communicating with the encoder side and performing a synchronous operation to prevent generation of an unnecessary packet. However, it is not possible to control each part as an individual task. Further, for example, the MUX operates while detecting and managing the input data size in the ES buffer, so that the above situation can be avoided without communication of each unit. In this case, however, the number of accesses to the ES buffer by the MUX This increases the complexity of the abnormality determination process.

  The present invention has been made in view of such points, and even when input of data to be multiplexed is interrupted, it continues to operate normally independently without exchanging information with the encoder. It is an object of the present invention to provide a stream multiplexing apparatus that can perform the above-described process.

  Another object of the present invention is to provide stream multiplexing that can continue to operate normally without exchanging information with the encoder even when input of data to be multiplexed is interrupted. A stream recording apparatus including the converting apparatus.

  Furthermore, another object of the present invention is to provide stream multiplexing that can continue to operate normally without exchanging information with an encoder even when input of data to be multiplexed is interrupted. Is to provide a method.

In the present invention, in order to solve the above problems, an encoded stream buffer in which encoded streams respectively generated by a plurality of encoders are temporarily stored for each access unit as an access unit at the time of decoding, and the encoding Among the unit information read from the unit information buffer in a stream multiplexing device that receives data from a unit information buffer that stores unit information related to the access unit stored in the stream buffer and generates a multiplexed stream Whether or not the data of the access unit sufficient to satisfy the payload area of the data group constituting the multiplexed stream is stored in the encoded stream buffer based on the data size information of the corresponding access unit Judgment When the data size determination means and the data size determination means determine that the condition is satisfied, the necessary access unit data is read from the encoded stream buffer, and one data group is generated and output. have a data group generation unit, the data size determining unit, when the data of the current head of the access unit in the coded stream buffer is determined not to be able to meet the payload area of the data groups And waiting for the unit information corresponding to the next access unit to be stored in the unit information buffer, and when the unit information is stored, the unit information corresponding to the unit information and the first access unit Each of the first and next access units based on Stream multiplexing apparatus is provided which is characterized by determining whether they meet the payload area by over data.

In such a stream multiplexing device, when it is determined by the data size determination means that the data of the access unit that satisfies the payload area of the data group constituting the multiplexed stream is stored in the encoded stream buffer. Only, the data group generation means reads the data of the access unit from the encoded stream buffer and generates one data group. In addition, when it is determined that the payload area of the data group cannot be filled with the data of the current head access unit in the encoded stream buffer, the unit information corresponding to the next access unit is stored in the unit information buffer. Wait to be done. When the unit information corresponding to the next access unit is stored, whether the payload area is filled with each data of the head and the next access unit based on the unit information and the unit information corresponding to the head access unit. Determine whether or not. For this reason, when a sufficient access unit for multiplexing is not generated due to a difference in processing speed with the encoder or a failure in the encoder, an abnormal operation of continuing to output an extra data group is performed. There will be nothing. Since the data size determination means performs the above determination based on the unit information in the unit information buffer, normal multiplexing processing is performed independently of the encoder without exchanging data with the encoder. Will continue to run.

In the present invention, the encoded streams respectively generated by the plurality of encoders are stored in the encoded stream buffer that is temporarily stored for each access unit that is an access unit at the time of decoding, and the encoded stream buffer. In a stream multiplexing method for receiving data from a unit information buffer in which unit information related to the access unit is stored and generating a multiplexed stream, the unit read by the data size determination unit from the unit information buffer Of the information , based on the information on the data size of the corresponding access unit, the data of the access unit sufficient to satisfy the payload area of the data group constituting the multiplexed stream is stored in the encoded stream buffer. Have When it is determined that the condition is satisfied in the data size determination step for determining whether or not and the data size determination step, the data group generation means reads the necessary access unit data from the encoded stream buffer, One of viewing contains a data group generation step of generating and outputting the data set, in the data size determination step, the current first payload area of the data group by the data of the access unit in the coded stream buffer If it is determined that the unit information cannot be satisfied, it waits for the unit information corresponding to the next access unit to be stored in the unit information buffer, and when the unit information is stored, the unit information and the head Based on the unit information corresponding to the access unit , The top and the stream multiplexing method characterized by determining whether they meet the payload area by the data of the next of said access unit is provided.

In such a stream multiplexing method, when it is determined in the data size determination step that access unit data sufficient to satisfy the payload area of the data group constituting the multiplexed stream is stored in the encoded stream buffer. Only the data group generation step is executed, and the data of the access unit is read from the encoded stream buffer to generate one data group. In the data size determination step, when it is determined that the payload area of the data group cannot be filled with the data of the current head access unit in the encoded stream buffer, unit information corresponding to the next access unit is obtained. Wait for storage in the unit information buffer. When the unit information corresponding to the next access unit is stored, whether the payload area is filled with each data of the head and the next access unit based on the unit information and the unit information corresponding to the head access unit. Determine whether or not. For this reason, when a sufficient access unit for multiplexing is not generated due to a difference in processing speed with the encoder or a failure in the encoder, an abnormal operation of continuing to output an extra data group is performed. Nothing will happen. In the data size determination step, since the above determination is performed based on the unit information in the unit information buffer, the multiplexing process is normally performed independently of the encoder without exchanging data with the encoder. Will continue to run.

  According to the present invention, only when it is determined that access unit data sufficient to satisfy the payload area of the data group is prepared, the access unit data is read from the encoded stream buffer and one data group is generated. Therefore, when an access unit sufficient for multiplexing is not generated due to a difference in processing speed with the encoder or a failure in the encoder, an abnormal operation of continuing to output an extra data group is performed. Nothing will happen. Since the above determination is performed based on the unit information in the unit information buffer, the multiplexing process can be normally executed independently of the encoder without exchanging data with the encoder. it can.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
Here, the case where the present invention is applied to a stream recording apparatus that generates an MPEG stream and records it on a recording medium will be described. Examples of such a stream recording apparatus include a digital video recorder and a digital video camera.

FIG. 1 is a block diagram showing a configuration of a stream recording apparatus according to the embodiment.
The stream recording apparatus shown in FIG. 1 includes a video encoder 11 and an audio encoder 12 that compress and encode a video signal and an audio signal, respectively, using an MPEG method, and a video ES buffer 13 that temporarily stores the generated video ES and audio ES. The audio ES buffer 14, the video info buffer 15 and the audio info buffer 16 in which information about the video ES and the audio ES are stored, the MUX 20 for time-division multiplexing the video ES and the audio ES, and the multiplexed MPEG stream A recording processing unit 31 that performs the recording process, a recording medium 32 on which an MPEG stream is recorded, and a system controller 40 that controls these in an integrated manner.

  For example, a video signal and an audio signal from a TV tuner, an external input terminal, a network I / F, and the like are input to the video encoder 11 and the audio encoder 12 as digital data, respectively. Alternatively, a video signal picked up by a solid-state image sensor and an audio signal picked up by a microphone may be input.

  The video encoder 11 compresses and encodes the input video signal, and sequentially stores the encoded AU (picture) of the video ES in the video ES buffer 13. At the same time, the information (AU_Info) relating to the output AU is stored in the video Info buffer 15. The audio encoder 12 compresses and encodes the input audio signal, and sequentially stores the encoded audio ES AU (audio frame) in the audio ES buffer 14. At the same time, AU_Info related to the output AU is stored in the audio Info buffer 16. As AU_Info, position information, data size, PTS, DTS and the like on each AU Info buffer are recorded.

  The MUX 20 reads the video ES and audio ES from the video ES buffer 13 and the audio ES buffer 14 in units of AUs, divides them into packets, stores them, generates a pack with a plurality of packets, multiplexes each ES, and multiplexes them. Stream (PS) is output. At this time, AU_Info in the video Info buffer 15 and the audio Info buffer 16 is read as necessary, and information such as SCR and stream ID is generated and stored in the packet header or pack header.

  The recording processing unit 31 adds an error correction code or the like to the PS from the MUX 20, converts and modulates data into a format that can be recorded on the recording medium 32, and sequentially stores the PS in the recording medium 32. As the recording medium 32, an optical disc such as a DVD (Digital Versatile Disk), an HDD (Hard Disk Drive), a magnetic tape, a semiconductor memory, or the like is applicable.

  The system controller 40 includes a CPU, a memory, and the like, and comprehensively controls each of the above blocks. In the present embodiment, the system controller 40 executes control processing for each block on the real-time OS, and in particular, implements the control functions of the video encoder 11, audio encoder 12, and MUX 20 as individual tasks of the real-time OS. Yes.

  With this configuration, each of the video encoder 11, the audio encoder 12, and the MUX 20 operates without synchronizing with each other. For this reason, for example, when the video encoder 11 or the audio encoder 12 stops outputting the video ES and audio ES to the subsequent buffer due to a difference in processing speed with the MUX 20, the occurrence of a failure, or the transition to recording pause. In addition, a situation may occur in which the MUX 20 continues to generate packets without detecting this.

  In order to prevent such a situation, the MUX 20 includes a data size determination unit 21 and a pack generation unit 22. The data size determination unit 21 accesses the video Info buffer 15 and the audio Info buffer 16, acquires AU_Info such as the data size, and based on the acquired information, AUs that satisfy the pack to be output are stored in the ES buffer. It is determined whether it is prepared.

  When the data size determination unit 21 determines that sufficient AUs are prepared in the ES buffer, the pack generation unit 22 reads the necessary AUs from each ES buffer, and from the data size determination unit 21 to the header. Information to be stored is acquired, a pack is generated, and output to the recording processing unit 31. Conversely, if the data size determination unit 21 determines that sufficient AUs are not prepared in the ES buffer, the pack generation unit 22 enters an “ES wait state” in which pack generation is stopped.

  In this way, before actually generating a pack, the MUX 20 determines whether AU data sufficient to satisfy the next pack is prepared in the ES buffer based on the AU_Info recorded in each Info buffer. To do. As a result, the ES generation stop state can be detected without exchanging data with the video encoder 11 and the audio encoder 12, so that the normal operation is always continued.

Hereinafter, the processing of the MUX 20 will be described in detail.
FIG. 2 is a flowchart showing a flow of the entire multiplexing process by the MUX 20.
[Step S101] The MUX 20 simulates a P-STD buffer model. As a result, if it is determined that a video pack (V pack) can be generated, the processing from step S103 is executed. If determined to be impossible, the process proceeds to step S102.

  [Step S102] It is determined whether or not to end the multiplexing process. For example, it is determined that the process is finished when the video recording is stopped due to a user operation input or the like, and when there is no termination request, the process returns to step S101.

  [Step S103] The data size determination unit 21 determines whether or not the data in the AU has been stored in the pack payload area to the end in the previous multiplexing process (pack generation process). If it is determined that the data has been stored, the process proceeds to step S105. If it is determined that unstored data remains among the AU data, the process proceeds to step S104.

  [Step S104] It is determined whether or not the remaining data size of the AU not stored in the payload is equal to or greater than the payload capacity. If the payload capacity is greater than or equal to the capacity of the payload, a pack satisfying the payload of the new pack can be generated from the remaining AU data, so the process proceeds to step S109 and the pack generation unit 22 generates a new pack. Start processing. If the remaining data size is smaller than the payload, the process proceeds to step S105.

  [Step S105] The data size determination unit 21 determines whether or not AU_Info for the next AU exists in the Info buffer. This means that it is determined whether or not the AU to be multiplexed next exists in the ES buffer. If AU_Info exists, the process proceeds to step S106. On the other hand, if it does not exist, it enters the ES waiting state and proceeds to step S108.

[Step S106] The data size determination unit 21 searches the Info buffer to acquire information such as DTS, PTS, and data size for the next AU.
[Step S107] The data size determination unit 21 determines whether or not the payload is satisfied when the next AU is stored based on the data size acquired in step S106. If it is satisfied, a pack can be generated. Therefore, the process proceeds to step S109, and the pack generation unit 22 starts generating a new pack. If not satisfied, the process returns to step S105, and it is determined again whether AU_Info of the next AU to be stored in the remaining area of the payload exists in the Info buffer.

  [Step S108] In the ES waiting state, the data size determination unit 21 determines whether or not to end the multiplexing process. For example, it is determined that the process is finished when the video recording is stopped due to a user operation input or the like. If there is no termination request, the process returns to step S105 to determine whether or not the AU_Info of the next AU has been stored in the Info buffer, whereby a new AU to be stored in the remaining payload is generated by the encoder. The determination operation is stopped until it is made, and the ES waiting state is continued.

  [Step S109] The pack generation unit 22 reads the AU determined to be stored in the payload in the above step from the ES buffer, generates a pack, and outputs the pack. At this time, the data size determination unit 21 stores necessary information among the information acquired in step S106 in the header. The processing in step S109 will be described in detail with reference to FIG.

[Step S110] The MUX 20 advances the current time (SCR) so as to correspond to the time management information stored in Step S109. Then, the process returns to step S101.
FIG. 3 is a flowchart showing the flow of pack generation processing (corresponding to step S109) by the pack generation unit 22.

  [Step S201] It is determined whether a pack header, a packet header, a system header, or the like is necessary. If necessary, the process proceeds to step S202. If not, the process proceeds to step S203.

  [Step S202] A header is created. For example, necessary data among the data acquired by the data size determination unit 21 in step S106 of FIG. 2 is stored in the header. Further, new data is created and stored as necessary.

[Step S203] The ES buffer is accessed to read out and acquire a new AU.
[Step S204] The acquired AU data is stored in the payload.
[Step S205] It is determined whether the payload is completed (data is filled). If not completed, the process returns to step S203, and the next AU is obtained from the ES buffer. If the payload is completed, the process proceeds to step S206.

[Step S206] The generated pack is output. In this pack, the AU data is always filled in the payload.
Next, a specific example of pack generation in the ES waiting state will be given, and the operation will be described in association with the processing of the above flowchart. In the following example, a fixed-length pack of 2 Kbytes (2048 bytes) conforming to “The System Part of the MPEG-2 Standard” defined by ISO / IEC (International Organization for Standardization / International Electrotechnical Commission) 13818-1 is generated. It shall be. In such a pack, the remaining 2020 bytes excluding the pack header and packet header are used as a payload for storing AU data. For the sake of simplicity, only the generation of a V pack that stores video data will be described here.

FIG. 4 is a diagram showing how a pack is generated when the encoded AU size is sufficiently large.
In FIG. 4, the video ES buffer 13 stores AU (n) which is the nth AU. This AU (n) has a larger data size than the payload in the pack. In multiplexing this AU (n), the MUX 20 first reads the first 2020 bytes of the AU (n), stores it in the first V pack, and outputs it. In this case, if it is determined that a pack can be generated by simulating the P-STD in step S101, the process proceeds to steps S103, S105, S106, and S107. In step S107, it is determined that the payload is satisfied, and in step S109. Create the first V-pack.

  Thereafter, processing for generating the next pack is started. At this time, after the determination in step S101, it is determined in step S103 that the remainder of AU (n) is present in the video ES buffer 13, and step S104 is performed. The process proceeds to step S109 as long as the data size of the remaining AU (n) is 2020 bytes or more. That is, data is read out from the remainder of AU (n) in the video ES buffer 13 by 2020 bytes, and V packs are sequentially filled and output.

  Here, assuming that the last 760 bytes of data of AU (n) remain in the video ES buffer 13, the MUX 20 tries to store the remaining 760 bytes of data in the next V pack. At this time, the process proceeds to steps S101, S103, and S104. In step S104, it is determined that the remaining AU (n) is less than 2020 bytes, and the process proceeds to step S105. If it is determined by the search of the video Info buffer 15 in step S105 that the next AU, AU (n + 1) (not shown), is not stored in the video ES buffer 13, the remaining area of the payload Since the process cannot be satisfied, the process is temporarily stopped in the ES waiting state.

  Thereafter, when the generation of a new AU (n + 1) is confirmed by the search of the video Info buffer 15 in step S105, the process proceeds to step S106 to acquire AU_Info for the AU (n + 1), and only to fill the payload. When it is confirmed that the amount of AU data is stored in the video ES buffer 13 (step S107), pack generation is started using these data (step S109).

FIG. 5 is a diagram showing how a pack is generated when the encoded AU size is small.
In FIG. 5, nth to (n + 3) th AU (n) to AU (n + 3) are stored in the video ES buffer 13, but the total data size is assumed to be smaller than the payload capacity. In multiplexing these, the MUX 20 first determines whether or not the payload can be filled with the head AU (n). That is, the process proceeds to steps S101, S103, S105, and S106, and based on the data size acquired from the video Info buffer 15 in step S106, it is determined that the payload cannot be filled in step S107.

  As a result, the process returns to step S105 to search the video Info buffer 15 again, confirms the presence of the next AU (n + 1), and acquires the corresponding AU_Info in step S106. Thereby, it is determined in step S107 that the payload cannot be filled again, and the process returns to step S105. Thereafter, the same processing is repeated for AU (n + 2) and AU (n + 3), and these data cannot fill the payload. Therefore, the process returns to step S105, and the next AU (n + 4) (not shown) is the video ES buffer. 13 is determined. Here, when there is no next AU (n + 4), the process waits for the ES and temporarily stops.

  Thereafter, when generation of a new AU (n + 4) is confirmed by searching the video Info buffer 15 in step S105, the process proceeds to step S106, and AU_Info for the AU (n + 4) is acquired. When it is determined that the payload is satisfied by AU (n) to AU (n + 4) (step S107), pack generation is started using those data (step S109).

  As described above, in the present embodiment, until it is guaranteed that the payload of the next pack is filled with AU data, that is, AUs that only satisfy the pack payload area are generated by the encoder, and ES Until it is confirmed that the data is stored in the buffer, the AU is not multiplexed and the pack is not output. Therefore, the MUX 20 operates even when the ES encoding timing by the video encoder 11 or the audio encoder 12 is delayed due to a difference in processing speed from the MUX 20 or when the ES encoding is stopped due to a failure or a recording stop command. The situation where an extra pack is output continuously is avoided.

  At the same time, since the data to be multiplexed is recognized in advance, the MUX 20 does not read the same data from the ES buffer many times, and the number of accesses from the MUX 20 to the ES buffer is kept to a minimum. The time required is shortened and the processing is made efficient.

  Also, the MUX 20 acquires the data size of the AU to be multiplexed from the AU_Info of the Info buffer, and determines whether or not the payload is satisfied with the AU. For this reason, the MUX 20 can continue to operate normally without communicating with the video encoder 11 and the audio encoder 12 while avoiding the situation of outputting an extra pack.

  Accordingly, each encoder and the MUX 20 can be operated normally and completely independently, and control of each encoder and the MUX 20 can be implemented as individual tasks on the real-time OS. By making each control an individual task, it becomes possible to develop the functions of the encoder and MUX20 individually, flexibly responding to specification changes, etc., and verifying defects for each function to be controlled to increase verification efficiency You can also.

  In addition, by performing the determination process based on the information in the Info buffer, for example, the process is simplified compared to the case where the determination process is performed by reading the AU on the ES buffer and checking the data size. Transfer time and processing time can be shortened.

  In the above embodiment, the case where the present invention is applied to a stream recording apparatus has been described. However, the present invention is not limited to this. Can be applied.

It is a block diagram which shows the structure of the stream recording apparatus which concerns on embodiment. It is a flowchart which shows the flow of the whole multiplexing process by MUX. It is a flowchart which shows the flow of the pack production | generation process by a pack production | generation part. It is a figure which shows the mode of a pack production | generation when the size of encoded AU is large enough. It is a figure which shows the mode of the pack production | generation when the size of encoded AU is small. It is a figure which shows the principal part structure of the conventional stream production | generation apparatus using an MPEG system.

Explanation of symbols

  11 ... Video encoder, 22 ... Audio encoder, 13 ... Video ES buffer, 14 ... Audio ES buffer, 15 ... Video Info buffer, 16 ... Audio Info buffer, 20 ... MUX, 21 ... Data size Judgment unit, 22 ... pack generation unit, 31 ... recording processing unit, 32 ... recording medium, 40 ... system controller

Claims (4)

An encoded stream buffer in which encoded streams respectively generated by a plurality of encoders are temporarily stored for each access unit that is an access unit at the time of decoding, and a unit related to the access unit stored in the encoded stream buffer In a stream multiplexing apparatus that receives data from a unit information buffer in which information is stored and generates a multiplexed stream,
Of the unit information read from the unit information buffer, based on the data size information of the corresponding access unit, the data of the access unit sufficient to fill the payload area of the data group constituting the multiplexed stream is present. Data size determination means for determining whether or not stored in the encoded stream buffer;
A data group generation unit that reads out the necessary data of the access unit from the encoded stream buffer and generates and outputs one data group when the data size determination unit determines that the condition is satisfied;
I have a,
When the data size determination unit determines that the payload area of the data group cannot be filled with the data of the current head access unit in the encoded stream buffer, the data size determination unit corresponds to the next access unit. Waiting for the unit information to be stored in the unit information buffer, and when the unit information is stored, based on the unit information and the unit information corresponding to the head access unit, the head and the next A stream multiplexing apparatus for determining whether or not the payload area is filled with each data of the access unit . When a part of the data of the access unit cannot be stored in the payload area of the data group by the data group generation unit, the data size determination unit determines that the remaining data that has not been stored is the next data group. It is determined whether or not the payload area is satisfied, and if so, the data size determination means generates the data group using the remaining data, and if not, the data size determination means Waiting for the unit information corresponding to the access unit to be stored in the unit information buffer, and when the unit information is stored, the data of the access unit corresponding to the unit information and the remaining data 2. The stream multiplexing apparatus according to claim 1, wherein it is determined whether or not the payload area is satisfied. In a stream recording apparatus that generates a multiplexed stream and records it on a recording medium,
  A plurality of encoding means for generating individual encoded streams respectively;
  An encoded stream buffer that temporarily stores the encoded stream generated by the encoding means for each access unit that is an access unit at the time of decoding;
  A unit information buffer for temporarily storing unit information related to the access unit stored in the encoded stream buffer received from the encoding means;
  Of the unit information read from the unit information buffer, based on the data size information of the corresponding access unit, the data of the access unit sufficient to fill the payload area of the data group constituting the multiplexed stream is present. Data size determination means for determining whether or not stored in the encoded stream buffer;
  A data group that reads out the necessary data of the access unit from the encoded stream buffer and generates one data group and outputs it as the multiplexed stream when the data size determining unit determines that the condition is satisfied Generating means;
  Have
  When the data size determination unit determines that the payload area of the data group cannot be filled with the data of the current head access unit in the encoded stream buffer, the data size determination unit corresponds to the next access unit. Waiting for the unit information to be stored in the unit information buffer, and when the unit information is stored, based on the unit information and the unit information corresponding to the head access unit, the head and the next A stream recording apparatus for determining whether or not the payload area is filled with each data of the access unit. An encoded stream buffer in which encoded streams respectively generated by a plurality of encoders are temporarily stored for each access unit that is an access unit at the time of decoding, and a unit related to the access unit stored in the encoded stream buffer In a stream multiplexing method for receiving data from a unit information buffer in which information is stored and generating a multiplexed stream,
  The data size determination means only fills the payload area of the data group constituting the multiplexed stream based on the data size information of the corresponding access unit among the unit information read from the unit information buffer. A data size determination step for determining whether data of the access unit is stored in the encoded stream buffer;
  Data determined when the data size determining step determines that the condition is satisfied, data group generation means reads the necessary access unit data from the encoded stream buffer, and generates and outputs one data group A group generation step;
  Including
  In the data size determination step, when it is determined that the payload area of the data group cannot be filled with the data of the current top access unit in the encoded stream buffer, it corresponds to the next access unit. Waiting for the unit information to be stored in the unit information buffer, and when the unit information is stored, based on the unit information and the unit information corresponding to the head access unit, the head and the next A stream multiplexing method comprising determining whether or not the payload area is filled with each data of the access unit.

Images