JP5625776B2 - Program, multiplexing apparatus and multiplexing method - Google Patents

Description

  The present invention relates to a program, a multiplexing device, and a multiplexing method.

  When transmitting data of each medium such as a moving image and sound, these are encoded to generate a bit stream, and each bit stream is multiplexed and transmitted in one stream.

  Conventionally, a decoder model on the receiving side is defined, and a multiplexing process is performed so that a virtual buffer in this decoder model does not fail due to overflow or underflow. For example, the transition of the virtual buffer of the decoder model is verified for each frame, and the encoded data amount of each frame is subtracted from the virtual buffer of the decoder model at each frame decoding time, and the buffer amount corresponding to the set bit rate is incremented. Techniques are known. In this method, the buffer increment is adjusted so that the virtual buffer of the decoder model does not overflow, and the multiplexing ratio is controlled based on the buffer increment.

JP 2001-339718 A JP-A-8-256329 JP-A-9-139720 International Publication No. 98/32252

  However, when multiplexing bit streams of multiple types of media, if an upper limit is set for the bit rate after multiplexing in consideration of the capacity of the transmission path, the upper limit is observed, and the virtual buffer fails. It was difficult to multiplex so as not to occur.

  According to one aspect of the invention, the following program is provided. This program calculates the priority in multiplexing for each of a plurality of bit streams to be multiplexed, and compares the integrated value of the information amount of the plurality of bit streams in a multiplexing section with a predetermined upper limit value. The computer executes a process of selecting a bitstream to be multiplexed from the plurality of bitstreams based on a comparison result between the integrated value and the upper limit value and the priority.

  According to the disclosed program, multiplexing apparatus, and multiplexing method, it is possible to prevent the decoder buffer from failing and to perform multiplexing according to the upper limit of the bit rate.

It is a figure which shows an example of the multiplexing apparatus of this Embodiment. It is a figure which shows an example of operation | movement of a multiplexing apparatus. It is a figure which shows an example of the multiplexing area and access unit of a bit stream. It is a flowchart which shows the flow of an example of a priority calculation process. It is a figure which shows the example of calculation of Bdiffmin. It is a figure which shows an example of the bit stream of a some media, and a multiplexing area. 12 is a flowchart illustrating an example of selection processing and multiplexing processing of a bit stream to be multiplexed (part 1). 12 is a flowchart illustrating an example of selection processing and multiplexing processing of a bit stream to be multiplexed (part 2). It is a figure which shows an example of the relationship between time and the number of remaining transmission bits in data multiplexing. It is a figure which shows an example of the hardware of the computer used for this Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram illustrating an example of a multiplexing apparatus according to the present embodiment.
The multiplexing apparatus 10 includes a multiplexing unit 11, priority calculation units 12-1, 12-2, ..., 12-n, a selection unit 13, and a storage unit 14.

  The multiplexing unit 11 is encoded by the encoding units 20-1, 20-2,..., 20-n and is stored in a plurality of types once stored in the memories 21-1, 21-2,. Multiplex media bitstreams. The media is audio, video, etc.

  The priority calculation units 12-1 to 12-n calculate the priority in multiplexing for each of a plurality of bit streams to be multiplexed. The priority calculation units 12-1 to 12-n, for example, a decoder buffer size, a decoding unit (hereinafter referred to as an access unit) of a decoder model defined by a standard for each medium, or a DTS (Decoding Time Stamp) indicating a decoding start time. ) To calculate the priority. The priority means a degree of risk that a virtual buffer that is an actual decoder buffer may fail in the decoder model.

  The selection unit 13 obtains an integrated value of the information amount of bit streams of a plurality of types of media in a certain multiplexing section, and calculates the integrated value (for example, the unit is the number of bits) and a predetermined upper limit value (for example, the unit is the bit). Number). Then, the selection unit 13 selects a bit stream to be multiplexed from a plurality of bit streams based on the comparison result between the integrated value and the upper limit value and the calculated priority. The upper limit value is, for example, a value obtained by multiplying the upper limit value of the bit rate set in consideration of the capacity of the transmission path through which the multiplexed bit stream flows, and the time of the multiplexing section.

Although details will be described later, for example, the selection unit 13 selects bit streams to be multiplexed in descending order of priority.
In addition, when the integrated value exceeds the upper limit value, the selection unit 13 has the lowest priority among the bitstreams of media in which the virtual buffer does not fail in the next multiplexing section without multiplexing. Processes such as not selecting one. Alternatively, when the above integrated value exceeds the upper limit value, the selection unit 13 selects a bitstream having an access unit straddling the end boundary of the multiplexing section according to the priority, and the access unit of the selected bitstream Is excluded from the multiplexing target. This suppresses the multiplexed bit stream from exceeding the upper limit value and prevents the decoding buffer from failing.

  The storage unit 14 stores the decoder buffer size of each medium, access unit information (number of bits, etc.), standard values such as DTS, the above-described upper limit value, and the like. The storage unit 14 is, for example, a ROM (Read Only Memory). The storage unit 14 may be provided outside the multiplexing apparatus 10.

  In the example of FIG. 1, priority calculation units 12-1 to 12-n are provided for each bitstream. However, one priority calculation unit calculates the priority of each bitstream. Also good.

  Further, multiplexing may be performed in consideration of the information amount of time information and data information (for example, data for subtitle broadcasting) other than media such as audio and moving images.

Hereinafter, the operation of the multiplexing apparatus 10 will be described.
FIG. 2 is a diagram illustrating an example of the operation of the multiplexing apparatus.
Step S1: The priority calculation units 12-1 to 12-n calculate the priority in multiplexing for each bitstream.

Step S2: The selection unit 13 obtains an integrated value of the information amount of bit streams of a plurality of types of media in the multiplexing section, and compares the integrated value with a predetermined upper limit value.
Step S3: The selection unit 13 selects a bitstream to be multiplexed based on the comparison result between the integrated value and the upper limit value and the priority calculated in the process of step S1.

Step S4: The multiplexing unit 11 multiplexes and outputs a plurality of bit streams selected by the selection unit 13.
By performing such multiplexing processing in consideration of the upper limit value and priority, it is possible to prevent the multiplexed bit stream from exceeding the upper limit of the bit rate and to prevent the decoding buffer from failing.

Next, an example of each of the above processes will be described in more detail.
(Example of priority calculation processing)
FIG. 3 is a diagram illustrating an example of bit stream multiplexing sections and access units.

  The horizontal axis represents time. Access units U1, U2, U3, U4, and U5 included in the bitstream are synchronization units such as frames and pictures, and are decoding units as described above.

  The multiplexing section Gc is not necessarily divided by the breaks between the access units U1 to U5. In the example of FIG. 3, the multiplexing section Gc is set in the range from the middle of the access unit U2 to the middle of the access unit U5. .

  In the following, it is assumed that the multiplexing section Gc is set in an appropriate time unit equal to or less than the insertion interval of the PCR (Program Clock Reference) defined by the operation regulations. Further, it is assumed that a DTS is added to each of the access units U1 to U5. For this reason, it is possible to determine which access units U1 to U5 are included in the multiplexing section Gc.

FIG. 4 is a flowchart illustrating an exemplary flow of priority calculation processing.
Step S10: The amount of generated information is not always constant in each access unit. Therefore, the priority calculation units 12-1 to 12-n calculate the virtual buffer amount for each access unit for each bit stream read from the memories 21-1 to 21-n in a certain multiplexing section Gc. . For example, when there are i access units in the multiplexing section Gc in a certain bitstream, the virtual buffer amounts Buff1 to Buffi are calculated.

  At this time, when there are access units that are not targeted for multiplexing in the previous multiplexing section, the priority calculation units 12-1 to 12-n determine the priority access unit as the first access unit in the current multiplexing section. Add information.

Step S11: The priority calculation units 12-1 to 12-n calculate the maximum virtual buffer amount BuffMAX and the minimum virtual buffer amount BuffMIN in the multiplexing section Gc.
In the process of step S11, the priority calculation units 12-1 to 12-n perform calculations for the access units straddling the boundaries of the multiplexed sections, assuming that the virtual buffer amount changes with a simple increase.

Further, assuming that the buffer extraction amount in each access unit is Buffoff1 to Buffoffi, the integrated value BUFFn of the virtual buffer amount at the end of the nth access unit in the multiplexing section Gc is _expressed by the following equation.

BUFF _n = BUFF _n-1 + Buffn-Buffoffn
In the above equation, Buffn is the virtual buffer amount of the nth access unit itself, and Buffoffn is the buffer extraction amount of the nth access unit. That is, a value obtained by adding the nth virtual buffer amount Buffn and subtracting the buffer extraction amount Buffoffn to the virtual buffer amount integrated value BUFF _n−1 calculated up to the (n−1) th access unit is a new one. The integrated value BUFF _n of the virtual buffer amount.

Then, the priority calculation unit 12-1 to 12-n is, BuffMAX = MAX (BuffMAX, BUFF n) by, calculating the BuffMAX. MAX () is a function that outputs the maximum value from the values in parentheses. For each access unit, the larger BuffMAX so far and the integrated value BUFF _n of the virtual buffer amount are calculated as a new BuffMAX.

Further, the priority calculation unit 12-1 to 12-n is, BuffMIN = MIN (BuffMIN, BUFF n) by, calculating the BuffMIN. MIN () is a function that outputs the minimum value from the values in parentheses. For each access unit, and BuffMIN until then, the smaller of the integrated value BUFF _n of the virtual buffer amount is calculated as a new BuffMIN.

  If 0 is allowed as the virtual buffer amount, BuffMIN = BMAX. BMAX is an allowable value (maximum value) of the virtual buffer amount defined by the standard for each medium.

  Step S12: The priority calculation units 12-1 to 12-n compare BMAX-BuffMAX and BuffMIN in the multiplexing section Gc, and set the smaller calculation result as Bdiffmin.

FIG. 5 is a diagram illustrating a calculation example of Bdiffmin.
The horizontal axis represents time, and the vertical axis represents the virtual buffer amount. FIG. 5 shows a case where a multiplexing section is set in access units U10, U11, and U12 in a certain bit stream. When the virtual buffer amount changes as shown in FIG. 5, BuffMIN is smaller than BMAX-BuffMAX. Therefore, in this case, BuffMIN is selected as Bdiffmin. Note that a small BuffMIN indicates that there is a possibility that an underflow of the virtual buffer may occur in this multiplexing section if no multiplexing is performed. On the other hand, when BMAX-BuffMAX is small, in the multiplexing section, if multiplexing is not performed, data is not extracted from the virtual buffer, and overflow may occur.

  Step S13: The priority calculation units 12-1 to 12-n calculate the priority E = Bdiffmin / BMAX in the multiplexing section Gc. The priority calculation units 12-1 to 12-n normalize by dividing Bdiffmin of the bit stream of each medium by the allowable value BMAX of the virtual buffer amount of each medium, and obtain the priority. As a result, it is possible to obtain a unified index indicating the priority of multiplexing bitstreams for each medium.

Next, details of the selection process in the selection unit 13 and the multiplexing process in the multiplexing unit 11 will be described.
(Example of selection process and multiplexing process of bit stream to be multiplexed)
FIG. 6 is a diagram illustrating an example of bit streams of a plurality of media and multiplexing sections.

  The horizontal axis indicates the time direction. FIG. 6 shows a bit stream of media M1, M2,..., Mn and a multiplexing section Gc1. The bit stream of the medium M1 includes access units U20, U21, U22, U23, and U24. The bit stream of the medium M2 includes access units U30, U31, U32, U33, U34, U35, and U36. The bit stream of the media Mn includes an access unit U40.

As shown in FIG. 6, the sizes and positions of the access units of the media M1 to Mn vary depending on the media.
7 and 8 are flowcharts showing an example of selection processing and multiplexing processing of a bit stream to be multiplexed.

Step S20: The selection unit 13 subtracts the number of bits of time information from the maximum number of bits Bmax in a certain multiplexing section.
The maximum bit number Bmax is represented as Bmax = Rmax1 × Gct1, where Gct1 is the time of the multiplexing section Gc1 and Rmax1 is the upper limit of the bit rate in the multiplexing section Gc1. Here, if the maximum number of bits after subtraction of time information is Bmaxs, Bmaxs = Bmax−Btime × (end time of multiplexed section Gc1−STCb) / STCg. Btime is the number of bits required for multiplexing time information, STCb is the time information insertion time in the previous multiplexing section, and STCg is the time information insertion interval.

This prevents the bit rate from exceeding the upper limit when time information is inserted into the bit stream.
Step S21: Next, the selection unit 13 checks whether there is an access unit starting from the middle of the multiplexing section Gc1 or completing in the middle, and calculates the number of bits of those access units by the time ratio. For example, in the example shown in FIG. 6, the access units U21, U24, U36, and U40 straddle the boundary of the multiplexing section Gc1. For such an access unit, the selector 13 assumes that the transmission bit rate is constant and updates the DTS of each access unit with the start time of DTS = DTS−multiplexing section Gc1 to obtain the time. The number of bits is calculated by the ratio.

  Step S22: If there is an access unit that is not transmitting in the previous multiplexing section because the maximum number of bits has been exceeded, the selection unit 13 sets the bit number as a carry-over from the previous multiplexing section. Add to the number of bits of the first access unit in the multiplexing section. Thereby, it is possible to perform multiplexing so as not to exceed the upper limit value (maximum number of bits) in the previous multiplexing section.

  Step S23: The selection unit 13 compares the integrated value of the number of bits of each access unit after the correction in step S22 with the maximum number of bits Bmaxs after subtraction of the number of bits of time information.

  Step S24 is performed when the integrated value of the number of bits of each access unit after correction is equal to or less than Bmaxs. In addition, when the integrated value of the number of bits of each access unit after correction exceeds Bmaxs, the process of step S29 in FIG. 8 is performed.

  Step S24: The selection unit 13 determines the bit amount of data information to be multiplexed according to the difference between Bmaxs and the integrated value of the number of bits of each corrected access unit, and corrects the total bit rate of the multiplexed section. .

  For example, when the total bit rate of all the media in the multiplexing section Gc1 is RG1, RG1 = RG1 + MIN ((Bmaxs1−integrated value of the number of bits of each access unit modified, bit amount of data information)) / Gct1 Correct it.

  According to this equation, if the bit amount of the data information is smaller than the difference between the maximum bit number Bmaxs1 of the multiplexing section Gc1 and the integrated value of the bit number of each modified access unit, the bit amount of the data information The bit rate increases by the amount. Further, when the bit amount of the data information is larger than the difference, the bit rate is increased by the bit amount of the data information corresponding to the difference.

  By determining the bit amount of the data information to be multiplexed according to the difference between Bmaxs and the integrated value of the number of bits of each access unit after correction, the bit rate is prevented from exceeding the upper limit by the data information.

  Step S25: The selection unit 13 determines whether or not to multiplex time information. If the time information is multiplexed, the process of step S26 is performed. If the time information is not multiplexed, the process of step S27 is performed. For example, when the STC that is time information of the current system is STC-STCb> STCg, the selection unit 13 determines to multiplex time information.

Alternatively, the selection unit 13 may determine that multiplexing is performed if (STCb + STCg−STC) / STCg is smaller than a predetermined threshold.
Step S26: The selection unit 13 inserts the time information into the bit stream to be multiplexed.

  Step S27: The selection unit 13 selects the bit streams to be multiplexed in order from the bit stream of the medium with the highest priority. For example, if DTS-STC <Bufferdelay, the selection unit 13 causes the multiplexing unit 11 to perform multiplexing processing.

  Here, Bufferdelay is the time when the first access unit is extracted from the decoder buffer, which is defined by the buffer model for each medium, for example. By performing multiplexing at the timing shown in the above equation, it is possible to prevent the decoder buffer from failing.

When multiplexing is performed, the selection unit 13 updates STC with STC = STC + number of multiplexed bits / RG1.
Step S28: For example, data information used for teletext etc. is preferably sent within a certain time according to operation regulations, although there are no buffer regulations.

FIG. 9 is a diagram illustrating an example of the relationship between time and the number of remaining transmission bits in data multiplexing. The horizontal axis represents time, and the vertical axis represents the number of remaining transmission bits.
FIG. 9 shows an example of the current time STCc, the scheduled transmission completion time STCe, the number of bits Db of all data information, and the number of already transmitted bits Dd. Here, Rg = Db / STCe can be expressed as Rg, which is the average bit rate at which all data information is transmitted at the scheduled transmission completion time STCe. Further, if the average bit rate at which the rest of the data information is transmitted at the scheduled transmission completion time STCe at the current time STCc is Rr, Rr = (Db−Dd) / (STCe−STCc). At this time, for example, when Rr−Rg> Th (predetermined threshold), the selection unit 13 multiplexes the data information.

When the data information is multiplexed, the selection unit 13 updates STC with STC = STC + number of multiplexed bits / RG1. Thereafter, the next multiplexing section is processed.
Step S29: If the integrated value of the number of bits of each access unit after correction exceeds Bmaxs, the selection unit 13 determines whether there is an access unit straddling the end boundary of the multiplexing section. If there is an access unit straddling the end boundary of the multiplexed section, the process of step S30 is performed, and if there is no access unit straddling the end boundary of the multiplexed section, it is allowed to exceed the upper limit of the bit rate. Then, the process of step S24 described above is performed.

For example, in the example shown in FIG. 6, the access units U24, U36, U40 straddle the end boundary of the multiplexing section Gc1.
Step S30: When there is an access unit straddling the end boundary of the multiplexing section, the selection unit 13 selects a candidate to be excluded from the multiplexing target (set the number of bits to 0) from the corresponding access units according to the priority. To select. When there are a plurality of corresponding access units, the selection unit 13 selects, for example, one access unit of the bit stream having the lowest priority calculated by the priority calculation units 12-1 to 12-n.

  Step S31: When the number of bits of the access unit selected in the process of step S30 is set to 0 and the access unit is multiplexed in the next multiplexing section, does the selection unit 13 exceed the maximum number of bits Bmaxs? Determine whether or not.

  When the maximum number of bits is exceeded in the next multiplexing section, the selection unit 13 proceeds to the process of step S34 without correcting the number of bits of the selected access unit, and when the maximum number of bits is not exceeded. The process of step S32 is performed.

  Step S32: The selection unit 13 corrects the number of bits of the access unit selected as a candidate for setting the number of bits to 0. As a result, the selected access unit is not multiplexed in the current multiplexing section, and is multiplexed in the next multiplexing section, for example.

  Step S33: The selection unit 13 compares the integrated value of the number of bits of each access unit after correcting the number of bits in the process of step S32 with the maximum number of bits Bmaxs after subtraction of the number of bits of time information.

  Step S24 is performed when the integrated value of the number of bits of each access unit after correction is equal to or less than Bmaxs. If the integrated value of the number of bits of each access unit after correction still exceeds Bmaxs, the process of step S34 is performed.

  Step S34: The selection unit 13 determines whether there is another candidate for an access unit that has not been selected in the process of Step S30 and whose bit number is 0. If there are other candidates, the processing from step S30 is repeated, and in the processing of step S30, the access unit of the bit stream having the next lowest priority is selected from the access units straddling the end boundary of the multiplexing section. Is done. If there is no other candidate, the correction is not performed and the bit rate is allowed to exceed the upper limit, and the process of step S24 in FIG. 7 is performed.

  As described above, by performing the multiplexing process in consideration of the upper limit value and priority of the bit rate, it is possible to prevent the multiplexed bit stream from exceeding the upper limit value and to prevent the decoding buffer from failing.

Note that the processing flows shown in FIGS. 7 and 8 are merely examples, and for example, the processing steps may be appropriately switched.
For example, in the above description, when the integrated value of the information amount of the access unit in the multiplexing section exceeds Bmaxs, the access unit that is not multiplexed is determined. However, the present invention is not limited to this. For example, the bit rate may be calculated from the integrated value, and if the bit rate exceeds the upper limit of the bit rate in the multiplexing section, an access unit that is not multiplexed may be selected.

The processing content of the multiplexing apparatus 10 as described above may be applied as firmware mounted on an LSI (Large Scale Integrated circuit), for example.
Further, each function of the multiplexing apparatus 10 may be realized by a program executed by a computer as follows.

FIG. 10 is a diagram illustrating an example of computer hardware used in this embodiment.
The computer 100 is entirely controlled by a CPU (Central Processing Unit) 101. A random access memory (RAM) 102, a ROM 103, and a plurality of peripheral devices are connected to the CPU 101 via a bus 109.

The RAM 102 is used as a main storage device of the computer 100. The RAM 102 temporarily stores at least part of an OS (Operating System) program and application programs to be executed by the CPU 101. The RAM 102 stores various data used for processing by the CPU 101. The ROM 103 stores basic programs and data executed by the CPU 101. Various standard values of the buffer model described above are also stored in the ROM 103, for example.

  Peripheral devices connected to the bus 109 include a hard disk drive (HDD) 104, a graphic processing device 105, an input interface 106, an optical drive device 107, and a communication interface 108.

  The HDD 104 magnetically writes data to and reads data from a built-in disk. The HDD 104 is used as a secondary storage device of the computer 100. The HDD 104 stores an OS program, application programs, and various data. Note that a semiconductor storage device such as a flash memory can also be used as the secondary storage device.

  A monitor 105 a is connected to the graphic processing device 105. The graphic processing device 105 displays an image on the screen of the monitor 105a in accordance with a command from the CPU 101. Examples of the monitor 105a include a display device using a CRT (Cathode Ray Tube) and a liquid crystal display device.

  A keyboard 106 a and a mouse 106 b are connected to the input interface 106. The input interface 106 transmits signals sent from the keyboard 106a and the mouse 106b to the CPU 101. Note that the mouse 106b is an example of a pointing device, and other pointing devices can also be used. Examples of other pointing devices include a touch panel, a tablet, a touch pad, and a trackball.

  The optical drive device 107 reads data recorded on the optical disc 107a using a laser beam or the like. The optical disc 107a is a portable recording medium on which data is recorded so that it can be read by reflection of light. The optical disc 107a includes a DVD (Digital Versatile Disc), a DVD-RAM, a CD-ROM (Compact Disc Read Only Memory), a CD-R (Recordable) / RW (ReWritable), and the like.

  The communication interface 108 is connected to the network 108a. The communication interface 108 transmits and receives data to and from other computers or communication devices via the network 108a.

With the hardware configuration as described above, the processing functions of the present embodiment can be realized.
As described above, the processing functions of the present embodiment can be realized by a computer. In that case, a program describing the processing contents of the functions that the multiplexing apparatus 10 should have is provided. By executing the program on a computer, the above processing functions are realized on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium. Examples of the computer-readable recording medium include a magnetic storage device, an optical disk, a magneto-optical recording medium, and a semiconductor memory. Magnetic storage devices include HDDs, flexible disks (FD), and magnetic tapes. Optical discs include DVD, DVD-RAM, CD-ROM / RW, and the like. Magneto-optical recording media include MO (Magneto-Optical disk).

  When distributing the program, for example, portable recording media such as a DVD and a CD-ROM in which the program is recorded are sold. It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network.

  The computer that executes the program stores, for example, the program recorded on the portable recording medium or the program transferred from the server computer in its own storage device. Then, the computer reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. In addition, each time a program is transferred from a server computer connected via a network, the computer can sequentially execute processing according to the received program.

  In addition, at least a part of the above processing functions can be realized by an electronic circuit such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), or a PLD (Programmable Logic Device).

  As described above, one aspect of the program, the multiplexing apparatus, and the multiplexing method of the present invention has been described based on the embodiments. However, these are merely examples, and the present invention is not limited to the above description.

The following additional notes are further disclosed with respect to the plurality of embodiments described above.
(Supplementary note 1) The priority in multiplexing is calculated for each of a plurality of bit streams to be multiplexed,
Comparing the integrated value of the information amount of the plurality of bitstreams in the multiplexing section with a predetermined upper limit value,
Based on a comparison result between the integrated value and the upper limit value and the priority, a bit stream to be multiplexed is selected from the plurality of bit streams.
A program that causes a computer to execute processing.

  (Supplementary note 2) When the integrated value exceeds the upper limit value, the bit stream having an access unit straddling the end boundary of the multiplexing section is selected according to the priority, and the bit stream of the selected bit stream is selected. The program according to appendix 1, wherein an access unit is excluded from multiplexing targets.

  (Supplementary Note 3) When the access unit straddling the end boundary is multiplexed in the next multiplexing section, if the integrated value in the next multiplexing section exceeds the upper limit value, The program according to claim 2, wherein another access unit of the bit stream is selected according to the priority, and the other access unit is excluded from the multiplexing target.

  (Additional remark 4) The said integrated value contains the information content of the said access unit excluded from the said multiplexing object in the previous multiplexing area, The program of Additional remark 2 or 3 characterized by the above-mentioned.

  (Supplementary Note 5) The priority of each bitstream is the maximum value of the virtual buffer amount set corresponding to each of the plurality of bitstreams in the multiplexing section and the upper limit value of the virtual buffer amount. Any one of Supplementary notes 1 to 4, wherein the difference is a value obtained by normalizing a smaller value of the minimum value of the virtual buffer amount in the multiplexing section with the upper limit value of the virtual buffer amount. The program according to one item.

  (Supplementary note 6) The upper limit value is a value obtained by subtracting the amount of time information included in the bitstream from a value obtained by multiplying the upper limit of the bit rate of the multiplexing section by the time of the multiplexing section. The program according to any one of appendices 1 to 5, which is characterized.

(Supplementary note 7) The program according to any one of supplementary notes 1 to 6, wherein an information amount of data information to be multiplexed is determined according to a difference between the upper limit value and the integrated value.
(Supplementary note 8) In the bitstreams selected in descending order of priority, multiplexing is performed when the difference between the decoding start time and the current time is within a predetermined value. The program as described in any one.

(Supplementary note 9) The program according to any one of supplementary notes 1 to 8, wherein the plurality of bit streams to be multiplexed are bit streams of a plurality of different media.
(Supplementary note 10) According to any one of Supplementary notes 1 to 9, wherein whether or not to multiplex the time information is selected according to a difference between the current time and the time when the previous time information is inserted. Program.

  (Supplementary Note 11) According to the difference between the average bit rate at which all data information is transmitted at the scheduled transmission completion time and the average bit rate at which the rest of the data information is transmitted at the scheduled transmission completion time at the current time, 11. The program according to any one of appendices 1 to 10, wherein whether to multiplex data information is selected.

  (Supplementary note 12) The program according to any one of Supplementary notes 1 to 11, wherein when the integrated value does not exceed the upper limit value, multiplexing is performed in order from the bit stream with the highest priority.

(Supplementary note 13) A priority calculation unit that calculates a priority in multiplexing for each of a plurality of bit streams to be multiplexed;
The integrated value of the information amount of the plurality of bit streams in the multiplexing section is compared with a predetermined upper limit value, and the plurality of bits are based on the comparison result between the integrated value and the upper limit value and the priority. A selector for selecting a bitstream to be multiplexed from the stream;
A multiplexing apparatus comprising:

(Additional remark 14) The priority in multiplexing is calculated for each of a plurality of bit streams to be multiplexed,
Comparing the integrated value of the information amount of the plurality of bitstreams in the multiplexing section with a predetermined upper limit value,
2. A multiplexing method, comprising: selecting a bit stream to be multiplexed from the plurality of bit streams based on a comparison result between the integrated value and the upper limit value and the priority.

DESCRIPTION OF SYMBOLS 10 Multiplexer 11 Multiplexer 12-1, 12-2, ..., 12-n Priority calculator 13 Selector 14 Storage unit 20-1, 20-2, ..., 20-n Encoder 21-1 , 21-2,..., 21-n memory

Claims (5)

A priority in multiplexing is calculated for each of a plurality of bit streams to be multiplexed,
Comparing the integrated value of the information amount of the plurality of bitstreams in the multiplexing section with the first upper limit value,
When selecting a bitstream to be multiplexed from the plurality of bitstreams based on a comparison result between the integrated value and the first upper limit value and the priority,
When the integrated value exceeds the first upper limit value, the bit stream having a decoding unit at the receiving side decoder across the end boundary of the multiplexing section is selected and selected according to the priority. And excluding the decoding unit of the bitstream from being multiplexed,
When the decoding unit straddling the end boundary is multiplexed in the next multiplexing section, if the integrated value in the next multiplexing section exceeds the first upper limit value, other bits straddling the end boundary Select another decoding unit of the stream according to the priority, exclude the other decoding unit from the multiplexing target,
The integrated value includes the information amount of the decoding unit excluded from the multiplexing target in the previous multiplexing section,
The priority of each bitstream is a maximum value in the multiplexing section and a second upper limit value of the buffer amount of the buffer amount of the decoder set corresponding to each of the plurality of bitstreams. Of the difference and the minimum value of the buffer amount in the multiplexing section, the smaller value is normalized by the second upper limit value of the buffer amount,
The first upper limit value is a value obtained by subtracting the amount of time information included in the bitstream from a value obtained by multiplying the upper limit of the bit rate of the multiplexing section by the time of the multiplexing section.
A program that causes a computer to execute processing. The program according to claim 1, wherein an information amount of data information to be multiplexed is determined according to a difference between the first upper limit value and the integrated value. 3. The program according to claim 1, wherein multiplexing is performed when a difference between a decoding start time and a current time is within a predetermined value in the bitstreams selected in order of priority. 4. . A priority calculation unit for calculating a priority in multiplexing for each of a plurality of bit streams to be multiplexed;
Comparing the integrated value of the information amount of the plurality of bitstreams in the multiplexing section and the first upper limit value, based on the comparison result of the integrated value and the first upper limit value and the priority, A selection unit that selects a bitstream to be multiplexed from the plurality of bitstreams ,
The selection unit includes:
When the integrated value exceeds the first upper limit value, the bit stream having a decoding unit at the receiving side decoder across the end boundary of the multiplexing section is selected and selected according to the priority. And excluding the decoding unit of the bitstream from being multiplexed,
When the decoding unit straddling the end boundary is multiplexed in the next multiplexing section, if the integrated value in the next multiplexing section exceeds the first upper limit value, other bits straddling the end boundary Select another decoding unit of the stream according to the priority, exclude the other decoding unit from the multiplexing target,
The integrated value includes the information amount of the decoding unit excluded from the multiplexing target in the previous multiplexing section,
The priority of each bitstream is a maximum value in the multiplexing section and a second upper limit value of the buffer amount of the buffer amount of the decoder set corresponding to each of the plurality of bitstreams. Of the difference and the minimum value of the buffer amount in the multiplexing section, the smaller value is normalized by the second upper limit value of the buffer amount,
The first upper limit value is a value obtained by subtracting the amount of time information included in the bitstream from a value obtained by multiplying the upper limit of the bit rate of the multiplexing section by the time of the multiplexing section.
A multiplexing device characterized by the above. A priority in multiplexing is calculated for each of a plurality of bit streams to be multiplexed,
Comparing the integrated value of the information amount of the plurality of bitstreams in the multiplexing section with the first upper limit value,
When selecting a bitstream to be multiplexed from the plurality of bitstreams based on a comparison result between the integrated value and the first upper limit value and the priority,
When the integrated value exceeds the first upper limit value, the bit stream having a decoding unit at the receiving side decoder across the end boundary of the multiplexing section is selected and selected according to the priority. And excluding the decoding unit of the bitstream from being multiplexed,
When the decoding unit straddling the end boundary is multiplexed in the next multiplexing section, if the integrated value in the next multiplexing section exceeds the first upper limit value, other bits straddling the end boundary Select another decoding unit of the stream according to the priority, exclude the other decoding unit from the multiplexing target,
The integrated value includes the information amount of the decoding unit excluded from the multiplexing target in the previous multiplexing section,
The priority of each bitstream is a maximum value in the multiplexing section and a second upper limit value of the buffer amount of the buffer amount of the decoder set corresponding to each of the plurality of bitstreams. Of the difference and the minimum value of the buffer amount in the multiplexing section, the smaller value is normalized by the second upper limit value of the buffer amount,
The first upper limit value is a value obtained by subtracting the amount of time information included in the bitstream from a value obtained by multiplying the upper limit of the bit rate of the multiplexing section by the time of the multiplexing section. Feature multiplexing method.

Images