JP4559126B2 - Video transmission method, video transmission apparatus, video transmission program, and computer-readable recording medium recording the program - Google Patents

Description

The present invention relates to a video transmission technique used in a system that encodes a digital signal such as a moving image or audio in accordance with, for example, an MPEG (Moving Picture Experts Group) system and transmits / receives it through a packet communication network, and more particularly, is connected via an IP network. The present invention relates to a video transmission technique suitable for a low-delay video communication system that provides a videophone function between remote locations.

  Real-time transmission / reception of high-quality digital compressed video is becoming a practical application as the speed and bandwidth of digital packet communication networks represented by the Internet increase. As an example, there is a videophone system in which a camera or a microphone is connected to a PC connected to the Internet, and video that is real-time encoded by the MPEG method or the like is bidirectionally transmitted with low delay.

  In general, packet loss occurs in transmission over the Internet. As means for recovering packet loss, techniques such as retransmission of lost packets (ARQ: Automatic Repeat Request) and recovery on the receiving side by adding redundant error correction codes (FEC: Forward Error Correction) are known. However, error correction by FEC is appropriate for bidirectional communication systems that require low-delay transmission.

  Known types of error correction codes include parity (XOR) codes, Hamming codes, BCH codes, Reed-Solomon codes, and the like. In any code, (n−k) redundant symbols are added to k information symbols for error detection and error correction, and a total of n symbols are sent to the transmission line (n, k). ) Expressed as an error correction code.

  As an example, the parity code adds 1 symbol as a result of performing an XOR operation on k information symbols, so that n−k = 1. As a typical example of a Hamming code, a Hamming (7, 4) code or a Hamming ( 15, 11) There is a code. The Reed-Solomon code is n = 255 when one symbol is 8 bits (1 byte), and a (255, k) error correction code can be designed for any k, and a shortened Reed-Solomon code By using, (n, k) error correction codes can be designed freely within the range of n ≦ 255 and n> k.

  FIG. 11 shows an example of the shortened Reed Solomon (128, 124).

  As shown in this figure, the shortened Reed-Solomon (128, 124) is designed with the Reed-Solomon encoder and decoder (255, 251), and 127 symbols (= 127 bytes) of the first half of the information symbol on both the transmitting and receiving sides. The operation is performed assuming that is zero. The part assumed to be zero is not actually transmitted. With such a method, an arbitrary (n, k) error correction code can be designed with n ≦ 255 and n> k using a (255, k) Reed-Solomon encoder and decoder.

  FIG. 12 shows an example of an encoding method when a Reed-Solomon (n, k) error correction code is applied for packet loss countermeasures (for example, see Non-Patent Document 1).

  As shown in this figure, in the Reed-Solomon (n, k) error correction code, the encoded information is divided into packets of a certain length (hereinafter referred to as media packets), and the lines are processed from the top of the buffer. Stored sequentially in the direction. When the input of k packets is completed, error correction codes are generated in the column direction in order from the leftmost of the buffer.

  That is, k symbols are extracted in the vertical direction for each symbol (= 1 byte) from k packets, and an inspection symbol of nk symbols (= nk bytes) is generated. Write to FEC packet buffer. When the calculation is completed, the calculation is performed in the column direction in the same manner by moving one symbol (= 1 byte) to the right.

  In this way, when the calculation for all the columns is completed and the generation of the FEC packet is completed, the media packet and the FEC packet are sent out to the transmission path. At this time, a sequence number is assigned to each packet in order to identify the order of the packets on the receiving side. In FIG. 12, sequence numbers 0 to n-1 are assigned.

  When a general RTP (Real-Time Trasnport Protocol) is used as a real-time media transmission format defined by RFC3550, a 16-bit sequence number is assigned to each packet. Therefore, the remainder obtained by dividing the sequence number by n The order of the packets can be identified by the values of 0 to n−1.

  On the receiving side, for each arrived packet, the packet is stored in the corresponding position in the buffer based on the sequence number. When a part of the media packet is lost during transmission, the error correction arithmetic unit performs error correction arithmetic processing in the column direction in order from the leftmost of the buffer to recover the lost packet. When the calculation for all the columns is completed and the lost packet is recovered, the contents of the media packet are output as encoded information output.

  If the position of the lost symbol can be identified, the Reed-Solomon (n, k) error correction code can recover the loss of the maximum (n−k) symbols. That is, in the above example, under the condition that no code error such as bit inversion occurs, if the media packet and the FEC packet are lost within (n−k) packets, this can be completely recovered.

  Hereinafter, a unit of error correction calculation composed of k media packets and FEC packets (n−k) described above is referred to as an error correction calculation block.

  Now, it is known that the amount of bits generated by encoding one picture is not constant in a high-compression video encoding system represented by the MPEG system. In particular, there are three types of pictures: an I picture in which one picture is independently encoded, a P picture in which the difference from the previous picture is encoded, and a B picture in which the difference from the previous and subsequent pictures is encoded It is known that the amount of generated code per picture differs greatly in the order of I> P> B pictures.

  Conventional video transmission applications have used CBR (Constant Bit Rate) transmission in which a generated code amount that differs depending on a picture is temporarily stored in a buffer and then smoothed to a constant bit rate before transmission. This is due to the limitation that the conventional information transmission path is a typical CBR type and can transmit only a certain amount of bits per second, but a delay occurs as a price for storing in the buffer.

On the other hand, in a best effort type packet communication network represented by an IP network, a change in transmission band is allowed. Therefore, all codes generated in each picture can be transmitted during one picture period regardless of the size. Since the buffer is not stored, the transmission delay is kept small, which is advantageous for a videophone application that needs to keep the round trip delay small.
Otsuka et al., “Development and Evaluation of MPEG2 over IP System Using FEC”, Information Research Report “Distributed System / Internet Operation Technology” No.024, 2002

  However, when an (n, k) error correction code is applied to a low-delay application that completes transmission in units of one picture, the following problems occur.

  FIG. 13 shows an example in which an (n, k) error correction code is applied to a conventional CBR transmission type application.

  In this case, since output is performed at a constant bit amount per second through the smoothing buffer, error correction encoding is performed each time k bits corresponding to the media packet are output, and (n−k) number of bits are output. FEC packets can be generated and output. That is, error correction calculation blocks can be configured and output at regular time intervals.

  On the other hand, when an (n, k) error correction code is applied to a low-delay application that completes transmission in one picture unit, the generated code amount of each picture changes. The error correction calculation block cannot be configured with good delimiters, resulting in excess or shortage of packets.

  For example, in the example of FIG. 14, it is assumed that the code amount generated in the first I picture (1) corresponds to (k + 1) media packets. Then, the last media packet of I picture (1) must be accommodated in the second error correction calculation block <2>. Here, assuming that the number of media packets finally reaches k due to the amount of codes generated in the subsequent B picture (2) and B picture (3), and the second error correction operation block <2> is satisfied. The last media packet of the I picture (1) cannot be subjected to the error correction code generation operation until the code of the B picture (3) is generated, and is awaited for transmission.

  Here, even if only the last media packet of the I picture (1) is transmitted before the error correction code generation operation, if the media packet is lost on the receiving side, In order to recover it, the FEC packet of the error correction operation block <2> is required. Therefore, in order to decode and reproduce the I picture, the arrival of the entire packet of the error correction operation block <2> must be waited. As a result, a delay corresponding to two pictures is unavoidable.

  As described above, when an error correction code is applied to a low-delay type application that completes transmission in units of one picture, there arises a problem that the delay increases.

  Here, in order to prevent an increase in delay, a method of completing packet transmission in units of pictures can be considered.

  For example, in the example of FIG. 15, when the last media packet of the I picture (1) is accommodated in the second error correction operation block <2>, the remaining (k−1) media packets are dummy, Error correction coding is performed assuming that the contents are all zero, and an FEC packet is generated. In this way, the error correction operation blocks <1> and <2> can be transmitted immediately without waiting for subsequent pictures. Similarly, when the number of media packets of B picture (2) is less than k, the remaining media packets are dummy, that is, error correction coding is performed assuming that the contents are all zero, and FEC packets are generated. Thus, it is possible to transmit immediately without waiting for the subsequent picture.

  At this time, if both sides of the transmission and reception side make an agreement in advance, “If there are not enough media packets that make up the error correction calculation block, dummy media packets are used and the contents are all zero”, the dummy media There is no need to actually send the packet.

However, considering the I picture (1) error-corrected and encoded by this method, for the error correction operation block <1>, (n−k) FEC packets are added to k media packets. However, for the error correction calculation block <2>, (n−k) FEC packets are added to only one media packet.

That is, only the media packet at the end of the picture is in a state where the redundancy is very high, which causes a problem that the local change in the redundancy becomes large. Furthermore, there is a problem that the amount of error correction calculation increases due to an increase in the total number of error correction calculation blocks due to insertion of dummy.

  As described above, when the (n, k) error correction coding is applied to a low-delay video transmission application in which transmission is instantly performed in units of pictures, the delay increases or an unintended local change in redundancy occurs. There is a problem that causes an increase in the amount of computation.

The present invention has been made in view of such circumstances, and error correction calculation is performed for each picture without causing an increase in transmission delay due to addition of an error correction code, a local change in redundancy, or an increase in calculation amount. The object is to provide a new video transmission technology that can be implemented independently.

[1] Configuration of Video Transmission Device of the Present Invention [1-1] First Configuration In order to achieve the above object, the video transmission device of the present invention packetizes encoded information of a moving image and performs error correction calculation. In order to add a redundant packet for recovering the packet loss and to send it to the communication network, (a) based on the bit amount generated as a result of encoding one image, A first determining means for calculating the number of packets required to store the bit amount, and determining the same number as the number of information symbols of the error correction code; and (b) determining the first determining means. about the number of information symbols, I picture to be set according to the picture type of the encoding target picture, P-picture, the proportional ratio of B-pictures, respectively R _I, R _P, if the R _B, R _I ≧ R proportional rate to be _P ≧ R _B Therefore, encoding of a second determining means for determining a test symbol number of false Ri correction code in a form which is proportional to the number of information symbols according to a picture type of the one image, (c) Part one image by entering all the information symbols Ru obtained from the information, the error correction operation, and configured with a generating means for generating a number of inspection symbol determined in the second determining means.
In this way, the second determining means protects the image having a large influence on the subsequent image more thickly, and sets the number of error correction code check symbols in order to keep the redundancy in the same type of image within a certain range. It is processed to decide.

In the case that this is configured, generate means, by combining the light sign computationally intensive and high error correction capability code efficiently, in order to reduce the computational load as a whole, determining the first Based on the number of information symbols determined by the means and the number of check symbols determined by the second determining means, one of a plurality of error correction codes may be selected and used.

  The video transmission method of the present invention realized by the operation of each of the processing means described above can also be realized by a computer program. This computer program is provided by being recorded on an appropriate recording medium or via a network. The present invention is realized by being provided and installed on implementing the present invention and operating on a control means such as a CPU.

[1-2] Second Configuration In order to achieve the above object, the video transmission apparatus of the present invention packetizes encoded information of a moving image, and performs redundancy to recover packet loss by error correction calculation. In order to perform processing for adding a packet and transmitting to a communication network, (a) it is necessary to store this bit amount based on the bit amount generated as a result of encoding a single image. The number of packets is calculated, and the number equal to the number of packets is determined as the number of information symbols of the error correction code to identify the number of information symbols used for the error correction calculation. The total number of error correction operation blocks is determined based on the maximum number of information symbols and is as large as possible within the range of the maximum value, and the variation in the number of information symbols is the smallest. As a result, for each of the first determination means for determining the number of information symbols entering each error correction calculation block, and (b) the number of information symbols determined by the first determination means, the picture type of the image to be encoded If the proportional ratios of the I picture, P picture, and B picture set according to are R _I , R _P , and R _B , respectively, according to the proportional ratio that R _I ≧ R _P ≧ R _B ,
Second determining means for determining the number of check symbols of the error correction code in proportion to the number of information symbols in accordance with the picture type of the one image, and (c) 1 for each error correction operation block.
Generating means for inputting the number of information symbols determined by the first determination means obtained from the encoded information of one image and generating the number of check symbols determined by the second determination means by error correction operation; It comprises so that it may be provided.
Here, the first determining means determines the number of information symbols entering each error correction calculation block in order to make the redundancy uniform. Specifically, the number of packets is N, and the number of information symbols. If the maximum value is k _MAX , the total number of error correction operation blocks is q, and the basic value of the number of information symbols is k _BASE ,
q = ceil (N ÷ k _MAX )
However, ceil () is an integer rounded up after the decimal point.
k _BASE = floor (N ÷ q)
However, floor () is calculated by an integer rounded down.
r = N− (q × k _BASE )
, K _q corresponding to the number of information symbols k ₁ , k _{2 ,.}
(I) When r = 0
k ₁ , k ₂ ,..., k _q = k _BASE
(Ii) When r> 0
k ₁ , k ₂ ,..., k _r = k _BASE +1
k _{r + 1} , ..., k _q = k _BASE
In this way, processing is performed so as to determine the number of information symbols that enter each error correction calculation block.
Then, the second determining means determines the number of check symbols of the error correction code so as to protect an image having a large influence on the subsequent image thicker and to keep the redundancy in the same type of image within a certain range. It is processed like this.

When this configuration is adopted, in order to cope with the case where information symbols in the same group are continuously lost, the check symbol generated by the generation means and the information symbol that is the generation source thereof are grouped into one group. , Transmission means may be provided for transmitting symbols of each group in an interleaved manner.
This transmission means does not take such a transmission form, and transmits the check symbol generated by the generation means and the information symbol that is the generation source as one group, and the symbols of each group are collected for each group. It may be processed as follows.

In the case that this is configured, generate means, by combining the light sign computationally intensive and high error correction capability code efficiently, in order to reduce the computational load as a whole, determining the first Based on the number of information symbols determined by the means and the number of check symbols determined by the second determining means, one of a plurality of error correction codes may be selected and used.

  The video transmission method of the present invention realized by the operation of each of the processing means described above can also be realized by a computer program. This computer program is provided by being recorded on an appropriate recording medium or via a network. The present invention is realized by being provided and installed on implementing the present invention and operating on a control means such as a CPU.

[2] receives the packet transmitted from the video transmission apparatus of the present invention constructed as the above-described configuration of Film image receiving apparatus related to the present onset bright, film image receiving apparatus related to the present onset Ming, In order to receive a packet in which encoding information of a moving image is stored, and to add a redundant packet for recovering the packet loss by the error correction operation, and to recover the lost packet by the error correction operation (B) a determination means for determining the same number of information symbols and number of inspection symbols as those used on the video transmission side for a certain image based on information notified from the video transmission side; The number of information symbols and the number of check symbols determined by the determining means are received, and a variable number of information symbols and check symbols obtained from the encoded information of one image are input, and the input And an arithmetic means for performing an error correction calculation.

  When configured in this way, the computing means is based on the number of information symbols and number of check symbols determined by the determining means in accordance with the processing of the video transmitting apparatus of the present invention. One of them may be selected and used.

Film image receiving method related to the present onset bright realized by operating the above-described devices are those that can be realized by a computer program, the computer program may be provided by being recorded on a suitable recording medium, is provided via a network, thereby realizing the image receiving method related to the present onset bright by operating on the control means, such as installed CPU in practicing the video receiving method related to the present onset bright .

[3] Framework of Configuration of the Present Invention In the video transmission device of the present invention, an (n, k) error correction encoder with variable n and k values is used. Independent error correction operation blocks are configured for each picture, and the values of n and k are changed according to the amount of generated codes per picture so that it is not necessary to insert dummy media packets at the time of calculation. Then, error correction calculation is performed, and an FEC packet is generated and transmitted.

In response to this, when the picture receiving apparatus related to the present onset Ming, n, the value of k for a variable (n, k) using the error correction decoder, the same as that selected by the video transmitting apparatus n, Using the value of k, lost packets are recovered in units of pictures.

  As described above, according to the present invention, by changing the values of n and k in accordance with the generated code amount per picture, the local change in redundancy and the amount of calculation due to the insertion of the dummy media packet are performed. The error correction operation block can be made independent for each picture unit, and an increase in delay can be prevented.

As described above, in the present invention, the number of information symbols is variable according to the generated code amount per one picture, and variably set according to the number of check symbols also information symbol number and a picture coding parameter Then, a configuration is adopted in which error correction code calculation and error correction calculation are performed.

  According to this configuration, according to the present invention, the error correction operation can be performed independently for each picture, an increase in transmission delay due to the addition of the error correction code can be prevented, and the local change in redundancy can be prevented. And an increase in the amount of calculation can be avoided.

[1] First Embodiment First, a first embodiment will be described first.

  FIG. 1 illustrates an embodiment of a device configuration of a video transmission device 1 of the present invention according to the first embodiment.

  As shown in this figure, a video transmission apparatus 1 according to the present invention includes a picture encoder / packetizer 11, a media packet buffer 12, an error correction operation parameter selector 13, an error correction code generator 14, an FEC. A packet buffer 15 and a network transmitter 16 are provided. The error correction code generation unit 14 includes an information symbol register 141, an error correction code calculator 142, and a check symbol register 143.

  The picture encoder / packetizer 11 encodes an input video signal in accordance with, for example, the MPEG system, generates a media packet 113 by dividing the code of each generated picture into a predetermined amount of bytes, and generates a media packet 113. Store in the packet buffer 12.

  FIG. 2 illustrates a method for generating the media packet 113 and storing it in the media packet buffer 12 at this time.

  Now, it is assumed that the maximum size of the media packet 113 is 3000 bytes. At this time, in order to accommodate the maximum media packet 113, the horizontal width of the media packet buffer 12 is also set to 3000 bytes.

  The maximum size of the media packet 113 is related to the maximum size of the packet transmitted to the network. As an example, when fragmentation of a packet on the network should not occur, the packet is set to a value obtained by subtracting various header sizes such as an IP header, a UDP header, and an RTP header from an MTU (Maximum Transfer Unit) of the network.

  Here, as shown in FIG. 2, if a code amount generated by encoding a picture is 17000 bytes, 17000 ÷ 3000 = 5 remainder 2000, so that 5 pieces having a maximum length of 3000 bytes A total of six media packets are configured, that is, the media packet and the last media packet having a length of 2000 bytes. These media packets are stored in the media packet buffer 12. The part where the length of the last media packet is not filled is filled with a value of zero in preparation for an error correction code calculation later. The rest of the buffer remains empty.

  Returning to FIG. 1, the picture encoder / packetizer 11 gives the media packet number information 111 indicating the number of packets of the target picture, the type of the picture, and the media packet 113 of the encoded picture. Coding parameter information 112 is output.

  In response to the media packet number information 111 and the encoding parameter information 112, the error correction calculation parameter selector 13 uses the two pieces of information to determine the k value and n− of the (n, k) error correction calculation. The value of k is determined.

  This k, that is, the number of information symbols, is set to be the same as the media packet number information 111. On the other hand, nk, that is, the number of check symbols, is determined by any one of the following four methods.

[Method 1]
n−k = a constant value. That is, the value of n−k is always the same regardless of the media packet number information 111 and the encoding parameter information 112.

[Method 2]
Let n−k = int (k × A) + B. Here, int () returns an integer value with the fractional part truncated.

If A = 0.2 and B = 1,
k = 1 to 4 → n−k = 1
k = 5-9 → n−k = 2
k = 10-14 → n−k = 3
...
Thus, the value of nk increases in proportion to the number of k. That is, in Method 2, the number of check symbols is proportional to the number of information symbols, and the redundancy is kept within a certain range.

[Method 3]
It is assumed that nk = R _I (in the case of I picture), nk = R _P (in the case of P picture), and nk = R _B (in the case of B picture).

In MPEG encoding, the loss of an I picture or P picture has a larger influence than the loss of a B picture because an error propagates to subsequent pictures by temporal prediction. Therefore, in Method 3, by setting R _I, R _{P, and} R _{B such} that R _I ≧ R _P ≧ R _B , the I picture and the P picture are protected thicker.

[Method 4]
Method 2 and method 3 are combined to determine the value of nk.

That is, for each of the I picture, P picture, and B picture, the values of A and B described in the method 2 are defined, and A _I ≧ A _P ≧ A _B or B _I ≧ B _P ≧ B _B are set. In this way, the number of check symbols is made proportional to the number of information symbols, and the I picture and P picture are protected thicker.

  The k value and the nk value determined by the error correction operation parameter selector 13 in this way are notified to the error correction code generation unit 14 as the symbol number selection information 131.

  In response to this notification, the error correction code generation unit 14 starts calculation in order from the leftmost of the media packet buffer 12. As shown in FIG. 2, the media packet buffer 12 stores k media packets. From this, a total of k symbols are extracted one by one in the vertical direction and set in the information symbol register 141. The check symbol register 143 is cleared with zero symbols before operation.

  Then, the error correction code calculator 142 sequentially reads information symbols from the information symbol register 141 and reflects the calculation result in the check symbol register 143. When the calculation is completed, nk check symbols 144 generated on the check symbol register 143 are output and written in the corresponding positions in the FEC packet buffer 15 in the vertical direction.

  Next, the relationship between the above error correction code calculation, the use of a shortened code, and the fact that k and nk are variable will be described with reference to FIG.

  In FIG. 3, an 8-bit symbol Reed-Solomon code is used as the error correction code, and the number of information symbols is 255 symbols (255 bytes). However, by using the shortened code described with reference to FIG. 11, only the latter half k symbols (k bytes) are set as information symbols, and the first half 255-k symbols (255-k bytes) are set as zero symbols.

  The error correction code calculator 142 initializes the check symbol value stored in the check symbol register 143 to zero, and then reads the symbols in order from the left to perform check symbol generation and updates the check symbol value. I will do it. Then, the value of the test symbol obtained in a state where reading up to the rightmost symbol is completed becomes the value of the test symbol to be obtained.

  Note that such a calculation method of the error correction code calculator 142 is a general calculation method for obtaining a cyclic code by dividing by a generator polynomial (for example, “Technical Mathematics for TECHI Redundancy CQ Publisher, p. 105 ”), which is not directly related to the gist of the present invention, and the details are omitted.

  Here, in the state <2> in which the error correction code calculation has been completed up to the first half of the zero symbol portion, all the symbols read so far are zero symbols, and thus the values of the generated check symbols are all zero symbols. Therefore, the calculation may actually be started from the state <2>. That is, for any value of k, after the check symbol is cleared with zero symbols, the calculation may be started from the top of the k information symbols.

Further, in order to set the check symbol length to nk symbols (nk bytes), the order of the generator polynomial is set to nk-1 order. For example, as a generator polynomial G (x),
G (x) = (x-1) (x-α) (x-α ² ) (x-α ^nk-1 )
And a cyclic code generation operation (see, for example, “Technical Mathematics for Redoing TECHC, published by CQ Publishing, p. 105”). However, α is the primitive element of the field, and is the root of “x ⁸ + x ⁴ + x ³ + x ² + 1 = 0” in the Reed-Solomon code of the 8-bit symbol, for example.

  In this way, the error correction code calculator 142 selects the generator polynomial G (x) corresponding to the value of n−k, and sequentially calculates n−k check symbols for the k information symbols. Thus, a check symbol can be generated even if k and nk are variable.

  Returning to FIG. 1, when the above calculation is performed up to the right end of the media packet buffer 12 and nk FEC packets are generated in the FEC packet buffer 15, the network transmitter 16 determines the media packet buffer 12. To k media packets and n−k FEC packets from the FEC packet buffer 15 are read in the horizontal direction, and a network packet header (for example, RTP / UDP / IP header) is added to each and sent to the network. To do.

  At this time, in order to transmit the order of the media packet and the FEC packet, the media packet number information 111 and the encoding parameter information 112 to the receiving device, the network transmitter 16 adds these information to the header of the network packet. For example, the packet order uses the sequence number field of the RTP header, and other information is stored in a unique header. As the information stored in the unique header, the value of k and nk may be stored in pairs, or the value of k and a value indicating the picture type of MPEG coding may be stored.

  FIG. 4 shows an example of a network packet when the above-described method 2 is used as the parameter selection method of the error correction calculation parameter selector 13 and A = 0.2 and B = 1.

  It can be seen that the number of FEC packets becomes variable according to the value of the number of media packets k, and at the same time, the error correction operation block is completed in each picture and can be transmitted. In this example, the RTP header stores a packet sequence number, the unique header stores a picture serial number, and a pair of k and nk values.

  It should be noted that the portion that has been zero-filled when calculating the error correction code is not actually transmitted.

  Next, the operation on the receiving device side will be described.

Figure 5 illustrates an example of the structure of Film image receiving apparatus 2 related to the present onset bright.

As shown in this figure, Film image receiving apparatus 2 related to the present onset Ming, a network receiver 21, a media packet buffer 22, the FEC packet buffer 23, an error correction calculation parameter selector 24, an error correction operation A unit 25 and a picture decoder 26 are provided. The error correction calculation unit 25 includes a symbol register 251 and a symbol correction calculator 252.

  The network receiver 21 receives a packet from the network, and stores the media packet 212 in the media packet buffer 22 and the FEC packet 213 in the FEC packet buffer 23.

  At this time, the order of the media packet and the FEC packet is determined by referring to the sequence number recorded in the header of the network packet (in the above example, the sequence number stored in the RTP header). And stored in the same arrangement as the FEC packet buffer 15. If the packet length is less than the packet buffer width, the empty part is filled with zero symbols. If a packet does not arrive due to packet loss on the network, the buffer position is left blank.

  When all the packets constituting one picture arrive (this can be determined, for example, by the arrival of the last FEC packet of the picture or the arrival of the packet constituting the next picture), the error correction calculation unit 25 Starts the lost packet recovery operation.

  First, the error correction calculation parameter selector 24 calculates k and n−k values based on the network header information 211 sent from the network receiver 21, and uses them as symbol number selection information 241 for symbol correction. This is transmitted to the calculator 252. The calculation formulas for the values of k and nk at this time use the same method as the error correction calculation parameter selector 13 of the video transmission apparatus 1 and obtain the same result as that on the transmission side. Note that, as in the example of FIG. 4 described above, when a set of k and nk values is directly described in the header, this value may be directly output.

  In addition, from the media packet buffer 22 and the FEC packet buffer 23, erasure position information 222 indicating a position where a packet has not arrived and is blank is transmitted to the error correction calculation unit 25.

  In response to this, the error correction calculation unit 25 refers to the number of erasure position information 222, and if this does not exceed the range where error correction calculation is possible, for example, in the case of a Reed-Solomon code, it exceeds nk. If not, the calculation is started in order from the leftmost of the media packet buffer 22. A total of n symbols are extracted from the media packet buffer 22 and the FEC packet buffer 23 one by one in the vertical direction and set in the symbol register 251. At this time, the blank portion where the packet has not arrived is replaced with a zero symbol.

  The symbol correction calculator 252 calculates a syndrome value from each symbol stored in the symbol register 251 based on the k and nk values provided as the symbol number selection information 241 and the erasure position information 222. The symbol is corrected by specifying an error position and calculating a correct symbol value by a Berlekamp-Massey algorithm or the like. Since this method is not directly related to the gist of the present invention, the details are omitted (see, for example, “Industrial Mathematics for Redoing TECHC, p. 106” published by CQ publisher).

  When this calculation is completed, the symbols in the symbol register 251 that have been corrected are written back to the original positions in the media packet buffer 22 and the FEC packet buffer 23, and the symbol correction calculation for the right column is continued.

  When the operations for all the columns are completed, the media packets stored in the media packet buffer 22 are read in the row direction in order from the top and supplied to the picture decoder 26.

As described above, for a variable number of media packets per picture, the number of error correction code information symbols is the same as the number of media packets, and a variable number of FEC packets are generated and transmitted / received. Operations can be performed independently for each picture to prevent an increase in transmission delay, suppress local changes in redundancy, and reduce the total number of error correction operation blocks by inserting dummy in error correction operations . The increase and the amount of error correction calculation can be prevented from increasing.

In general, there is an upper limit on the number k of information symbols that can be handled by the error correction code calculator 142 and the symbol correction calculator 252 due to limitations on the circuit scale and the amount of installed memory. Let this upper limit be k _MAX .

If the generated code amount per picture is very large and the value of the number of media packets becomes very large, it is assumed that the number of media packets exceeds k _MAX . At this time, it is conceivable to perform an error correction operation on k _MAX media packets to form an FEC packet, and the remaining media packets are transmitted and received without protection by the FEC packet.

[2] Second Embodiment Next, a second embodiment will be described.

  In the second embodiment, the configurations of the video transmission device 1 and the video reception device 2 are the same as those in the first embodiment. However, the selection methods of the values of n and nk in the error correction operation parameter selector 13 on the transmission side and the error correction operation parameter selector 24 on the reception side are different, and in accordance with this, the error correction code generation unit on the transmission side 14 and the error correction operation unit 25 on the receiving side are slightly different. The difference will be described below.

In the first embodiment, the number of media packets constituting one picture is set as it is as the value of k, and one error correction calculation block is constituted by one picture.

Therefore, in order to cope with the fluctuation of the number of media packets, it is necessary to deal with a wide range of k, and if the number of media packets becomes very large, FEC will be used for media packets exceeding the upper limit number k _MAX that can be calculated. There was no choice but to send and receive without packet protection.

On the other hand, in the second embodiment, one picture can be composed of a plurality of error correction operation blocks.

Specifically, first, the error correction calculation parameter selector 13 determines the total number q of error correction calculation blocks by the following method. Now, assuming that the maximum value of information symbols that can be calculated by the error correction code generation unit 14 and the error correction calculation unit 25 is k _MAX ,
Number of error correction calculation blocks q = ceil (number of media packets ÷ k _MAX )
However, ceil () determines the total number q of error correction operation blocks according to an integer rounded up to the nearest decimal point.

Next, the basic value k _BASE of the number of information symbols k is
k _BASE = floor (number of media packets ÷ q)
However, floor () is determined according to an integer obtained by rounding down the decimals.

In response to this determination, a value r corresponding to the remainder of the above division is now
r = number of media _packets− (q × k _BASE )
, The number of information symbols k ₁ , k ₂ ... K _q corresponding to each of the error correction operation blocks ₁ , ₂ ,.
If r = 0
k ₁ , k ₂ ... k _q = k _BASE
If r> 0
k ₁ ,... k _r = k _BASE +1
k _{r + 1} , ... k _q = k _BASE
It is determined as follows.

The purpose of the above calculation is to select k ₁ , k ₂ ... K _q that are as large as possible without exceeding k _MAX and have the smallest variation in the size of each error correction operation block.

As an example of the above calculation, when the maximum value k _MAX of information symbols that can be calculated is 15, 41 media packets are input to the media packet buffer 12. This state is shown in FIG.

At this time, in order to construct an error correction calculation block as large as possible without exceeding k _MAX = 15, two error correction calculation blocks with k = 14 and one error correction calculation block with k = 13, a total of 3 It will be readily understood that the blocks need only be constructed.

When the above calculation is actually performed,
Number of error correction calculation blocks q = ceil (number of media packets ÷ k _MAX )
= Ceil (41 ÷ 15)
= 3
k _BASE = floor (number of media packets ÷ q)
= Floor (41 ÷ 3)
= 13
r = number of media _packets− (q × k _BASE )
= 41− (3 × 13)
= 2
It becomes.

Therefore, k ₁ , k ₂ = k _BASE + 1 = 13 + 1 = 14
k ₃ = k _BASE = 13
Thus, it can be seen that a desired value is obtained.

Subsequently, the error correction calculation parameter selector 13 uses the method described in the first embodiment based on the values of k ₁ , k ₂ ... K _q for each of the q error correction calculation blocks. Based on any one of methods 1 to 4, the value of (n−k) _1, (n−k) ₂ ,... (N−k) _q is determined. A set of these k and (n−k) values, a total of q sets, is provided to the error correction code generation unit 14 as the symbol number selection information 131.

  In the example of FIG. 6, the three error calculation blocks are indicated as “group 1” to “group 3”, respectively. In addition, as the value of n−k, 3, 3 and 2 are selected for each group.

  In response to this, the error correction code generation unit 14 generates an error correction code as soon as a media packet constituting each group arrives at the media packet buffer 12, and places the error correction code at the position of the corresponding group in the FEC packet buffer 15. Write. When the calculation is completed for all the groups, the media packet and the FEC packet are output to the network transmitter 16.

  Also in the video receiver 2, the error correction calculation parameter selector 24 reproduces the number q of error calculation blocks and the values of k and nk of each group by the same method as described above, and sends the error correction calculation unit 25 to the error correction calculation unit 25. introduce.

  Therefore, as shown in FIG. 7, the same group as that on the video transmission apparatus 1 side is configured in the media packet buffer 22 and the FEC packet buffer 23. The error correction calculation unit 25 performs error correction calculation for each group, and outputs the media packet to the picture decoder 26 when the calculation is completed for all the groups.

  FIG. 8 illustrates a packet order when transmitting media packets and FEC packets constituting one picture on the network.

  FIG. 8A shows that all FEC packets are transmitted after all media packets are transmitted, as in the first embodiment.

  On the other hand, as shown in FIG. 8B, media packets and FEC packets can also be transmitted in pairs for each group.

  In this case, since the video transmission device 1 transmits the FEC packet from the group in which generation is completed, and the video reception device 2 can start the error correction calculation in order from the group that has arrived, the error correction code for one picture can be started. The generation operation and the error correction operation can be distributed a plurality of times, which has an effect of smoothing the operation load. Further, the packets stored in the buffer are limited to only one group, and after the operation of a certain group is completed and the output is completed, the operation is scheduled so as to input the packet of the subsequent group. Since the buffer can be shared, there is an effect of reducing the amount of use of the buffer.

  Furthermore, as shown in FIG. 8C, the packets of each group can be nested and transmitted one by one.

  It is known that burst loss often occurs in which a plurality of packets are continuously lost on the network. By nesting in this way, for example, even when three packets are continuously lost, each group Only one packet has been lost. Therefore, when nk = 2 is selected in the Reed-Solomon code, three consecutive packet losses cannot be recovered by the transmission method shown in FIGS. 8A and 8B, but in FIG. If it is a method, it can recover.

As described above, according to the second embodiment, even when the error correction code calculator 142 and the symbol correction calculator 252 having a relatively small upper limit number k _{MAX of} information symbols are used, a variable number of media can be used. An error correction code can be adaptively added to the packet. Further, depending on the transmission order of packets, it is possible to smooth the load of error correction operation and to increase the resistance to burst loss.

[3] Third Embodiment Next, a third embodiment will be described.

9, an example embodiment of a device configuration of the video transmission apparatus 1 of the present invention shown in accordance with the third embodiment example, in FIG. 10, film images received related to the present onset bright according to the third embodiment An example of a device configuration of the device 2 is illustrated.

  The difference from the first and second embodiments is only that a plurality of error correction code calculators 142 and symbol correction calculators 252 are arranged in the third embodiment.

  The Reed-Solomon code that has been mainly described so far has a high error correction capability but is complicated in computation, and particularly when implemented by software, the computation load becomes a problem.

  On the other hand, the parity code (XOR) has an advantage that the calculation is very light although the value of n−k is limited to 1. When n−k = 1, the Reed-Solomon code and the parity code have the same resistance to packet loss (both can recover only one packet loss).

  Therefore, in FIG. 9, when both the Reed-Solomon code calculator and the parity code calculator are implemented as the error correction code calculator 142, the error correction calculation parameter selector 13 determines the value of n−k as 1. For the other nk, a Reed-Solomon code can be selected. The selection result is transmitted as error correction code selection information 132, and an FEC packet is generated using the selected error correction code calculator 142.

  Also on the receiving side, as shown in FIG. 10, both the Reed-Solomon code and the parity code are implemented as the symbol correction computing unit 252, and the error correction computing parameter selector 24 uses the same error correcting code selection information 242 as the transmitting side. By outputting, it is possible to select the symbol correction calculator 252 that matches the code used on the transmission side and perform error correction calculation.

  As described above, in the third embodiment, by changing the coding method to be used according to the values of k and nk, the code having a high error correction capability and the code having a light computation load are efficiently combined, The overall calculation load can be reduced. Examples of error correction codes to be combined include Hamming (7, 4) codes in addition to the above Reed-Solomon codes and parity (XOR) codes.

It is a figure which shows the structure of the video transmission apparatus in the 1st Example of this invention. It is the figure which showed the production | generation method of the media packet, and the storage system to a media packet buffer. It is the figure shown about the error correction code production | generation system at the time of using a shortened code. It is the figure shown about the error correction code production | generation system at the time of using a shortened code. It is a figure which shows the structure of the video receiver relevant to this invention in the 1st Example of this invention . It is a figure which shows grouping of the error correction arithmetic block by the transmission side in the 2nd Example of this invention. It is a figure which shows grouping of the error correction calculation block in the receiving side in the 2nd Example of this invention. It is a figure which shows the transmission order of the packet on the network in the 2nd Example of this invention. It is a figure which shows the structure of the video transmission apparatus in the 3rd Example of this invention. It is a figure which shows the structure of the video receiver relevant to this invention in the 3rd Example of this invention . It is a figure explaining the shortened Reed-Solomon code. It is a figure which shows the case where an error correction code is applied for packet loss countermeasures. It is a figure which shows the case where an error correction code is applied with respect to the conventional CBR transmission type application. It is a figure which shows the 1st case where it was going to apply an error correction code about the low delay type application which completes transmission per 1 picture. It is a figure which shows the 2nd case where it was going to apply an error correction code | cord | chord about the low-delay type application which completes transmission per 1 picture.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Video transmitter 2 Video receiver 11 Picture encoder / packetizer 12 Media packet buffer 13 Error correction operation parameter selector 14 Error correction code generator 15 FEC packet buffer 16 Network transmitter 21 Network receiver 22 Media packet buffer 23 FEC packet buffer 24 Error correction calculation parameter selector 25 Error correction calculation unit 26 Picture decoder 141 Information symbol register 142 Error correction code calculator 143 Check symbol register 251 Symbol register 252 Symbol correction calculator

Claims (10)

In a video transmission method in which encoded information of a moving image is packetized, a redundant packet for recovering packet loss by error correction operation is added, and transmitted to a communication network.
Based on the amount of bits generated as a result of encoding a single image, the number of packets necessary to store this amount of bits is calculated, and the same number as the number of packets is calculated as an information symbol of the error correction code. The number of information symbols used for error correction calculation is determined by determining as a number, and the total number of error correction calculation blocks is determined based on the specified number of information symbols and a preset maximum number of information symbols, A process of determining the number of information symbols that enter each error correction operation block so that the variation in the number of information symbols is minimized as much as possible within the range of the maximum value, and
If the proportions of I picture, P picture, and B picture set according to the picture type of the encoding target picture are R _I , R _P , and R _B respectively for the determined number of information symbols, R according proportional ratio which _I ≧ R _P ≧ R _B, and determining the number of check symbols of an error correction code in a manner that proportional to the number of information symbols according to a picture type of the one image,
For each error correction operation block, a process of inputting the determined number of information symbols obtained from the encoded information of the one image and generating the determined number of check symbols by error correction operation. Prepared,
In the process of determining the number of information symbols, if the number of packets is N, the maximum value is k _MAX , the total number is q, and the basic value of the number of information symbols is k _BASE ,
q = ceil (N ÷ k _MAX )
However, ceil () is an integer rounded up after the decimal point.
k _BASE = floor (N ÷ q)
However, floor () is calculated by an integer rounded down.
r = N− (q × k _BASE )
, K _q corresponding to the number of information symbols k ₁ , k _{2 ,.}
(I) When r = 0
k ₁ , k ₂ ,..., k _q = k _BASE
(Ii) When r> 0
k ₁ , k ₂ ,..., k _r = k _BASE +1
k _{r + 1} , ..., k _q = k _BASE
To determine the number of information symbols that enter each error correction operation block,
A featured video transmission method. The video transmission method according to claim 1,
Comprising the step of transmitting the generated check symbol and the information symbol that is the generation source thereof as one group, and interleave the symbols of each group,
A featured video transmission method. The video transmission method according to claim 1,
Including the step of transmitting the generated check symbol and the information symbol that is the generation source as one group, and transmitting the symbols of each group in a group.
A featured video transmission method. The video transmission method according to any one of claims 1 to 3,
In the process of generating the check symbol, based on the determined information symbol number and check symbol number, selecting and using one of a plurality of error correction codes,
A featured video transmission method. In a video transmitting apparatus that packetizes moving image coding information, adds a redundant packet for recovering packet loss by error correction calculation, and transmits the packet to a communication network.
Based on the amount of bits generated as a result of encoding a single image, the number of packets necessary to store this amount of bits is calculated, and the same number as the number of packets is calculated as an information symbol of the error correction code. The number of information symbols used for error correction calculation is determined by determining as a number, and the total number of error correction calculation blocks is determined based on the specified number of information symbols and a preset maximum number of information symbols, Means for determining the number of information symbols entering each error correction operation block so as to be as large as possible within the range of the maximum value and the variation in the number of information symbols is minimized;
If the proportions of I picture, P picture, and B picture set according to the picture type of the encoding target picture are R _I , R _P , and R _B respectively for the determined number of information symbols, R according proportional ratio which _I ≧ R _P ≧ R _B, means for determining a test number of symbols of an error correction code in a manner that proportional to the number of information symbols according to a picture type of the one image,
Means for inputting the determined number of information symbols obtained from the encoded information of the one image for each error correction calculation block and generating the determined number of check symbols by error correction calculation; Prepared,
If the means for determining the number of information symbols is N, the maximum value is k _MAX , the total number is q, and the basic value of the information symbol number is k _BASE ,
q = ceil (N ÷ k _MAX )
However, ceil () is an integer rounded up after the decimal point.
k _BASE = floor (N ÷ q)
However, floor () is calculated by an integer rounded down.
r = N− (q × k _BASE )
, K _q corresponding to the number of information symbols k ₁ , k _{2 ,.}
(I) When r = 0
k ₁ , k ₂ ,..., k _q = k _BASE
(Ii) When r> 0
k ₁ , k ₂ ,..., k _r = k _BASE +1
k _{r + 1} , ..., k _q = k _BASE
To determine the number of information symbols that enter each error correction operation block,
A featured video transmission device.
The video transmission device according to claim 5,
Means for transmitting the generated check symbol and the information symbol that is the generation source thereof as one group, and transmitting the symbols of each group in an interleaved manner;
A featured video transmission device. The video transmission device according to claim 5,
A means for transmitting the generated check symbol and the information symbol that is the generation source as one group, and transmitting the symbols of each group in a grouped manner,
A featured video transmission device. The video transmission device according to any one of claims 5 to 7,
The means for generating the check symbol selects and uses one of a plurality of error correction codes based on the determined information symbol number and check symbol number.
A featured video transmission device.   A video transmission program for causing a computer to execute the video transmission method according to any one of claims 1 to 4. A computer-readable recording medium having recorded thereon a video transmission program for causing a computer to execute the video transmission method according to claim 1.

Images