JP4068540B2 - Encoded data transmission apparatus, encoded data decoding apparatus, encoded data transmission method, and encoded data decoding method - Google Patents

Description

  The present invention relates to a transmission technique and a decoding technique for encoded data, and in particular, an encoded data transmission apparatus, an encoded data decoding apparatus, and an encoded data transmission method capable of priority control transmission in a network having no priority control function. And an encoded data decoding method.

  In a video transmission system using a best effort network in which the transmission rate changes dynamically like the Internet, a reception buffer is prepared in order to absorb the fluctuation of the transmission rate. If this reception buffer size is not sufficient, packet discard due to a decrease in the transmission rate cannot be avoided, and problems such as a disturbed reproduced image occur. However, if a sufficient buffer size is secured, the probability of absorbing the transmission rate increases, but there is a problem that the delay increases. Therefore, the following proposal has been made as a method for suppressing the influence on image quality due to packet discard.

  The first method is a method of estimating a transmission rate fluctuation of a network and performing code amount control according to the estimated transmission rate (for example, see Non-Patent Document 1). FIG. 7 shows the operating principle. In this method, the status of packet transmission is confirmed, and the code amount control is performed based on the result. As a result, the disturbance of the reproduced image due to packet loss is minimized.

  That is, as shown in FIG. 7A, the encoding device 200 estimates the transmission rate fluctuation of the network 202 based on the network congestion information transmitted from the decoding device 201 side, and performs coding according to the estimated transmission rate. The input video signal is encoded while performing quantity control. For example, as shown in FIG. 7B, the transmission rate of the encoded data is estimated in accordance with the actual transmission rate that fluctuates dynamically, and the code amount control is performed in accordance with the estimated transmission rate. The decoding device 201 decodes encoded data transmitted via the network 202 and outputs a video signal.

  The second method employs encoded data divided into partially decodable encoded data or encoded data of a hierarchical encoding method and uses a priority control function of the network (for example, non-patent document). 2).

  FIG. 8 shows the operating principle. In this method, code amount control on the encoding side is not performed, but only priorities are assigned to the divided encoded data. Then, the network 205 side discards the packet according to the priority order according to the congestion situation. That is, the encoding device 203 shown in FIG. 8A hierarchically encodes the input video signal, and transmits the encoded data of each hierarchy with priority. For example, as shown in FIG. 8B, priorities are assigned in the order of basic layer, additional layer 1, additional layer 2, and additional layer 3, and these encoded data are transmitted. When the network 205 is congested, the network 205 side discards packets from information with low priority. Depending on the congestion state, for example, as shown in FIG. 8B, the packets of the additional layer 2 and the additional layer 3 with the lower priority are discarded, and the packets of the basic layer with the higher priority and the packet of the additional layer 1 are transmitted. .

The priority control function of the network 205 has a low probability that a packet with a high priority is discarded, and there is a high possibility that the packet will be transmitted to the decoding device 204 on the receiving side. Therefore, by increasing the priority of the base layer of the layer encoding method, there is a possibility that only the base layer encoded data can be received even when the transmission rate fluctuates. Below, the nonpatent literature 1 and the nonpatent literature 2 regarding a prior art are shown.
Quality Meeting, http://avs.kddlabs.co.jp/qmeet/g/BARC.pdf R.Aravind, MRCIvanlar, and ARReibman, "Packet loss resilience of MPEG-2 scalable video coding algorithm," IEEE Trans.Circuits Syst.Video Technol., Vol.6, pp.426-435, Oct. 1996.

  The actual transmission rate may fluctuate abruptly, and the response speed to the fluctuation is important. In the first conventional method described in Non-Patent Document 1, the timing at which the estimated transmission rate is reflected in the transmission code amount is after the end of encoding, and may not be able to follow rapid fluctuations. In addition, this method cannot be applied to an already encoded bitstream unless re-encoding is performed.

  Further, in the second conventional method described in Non-Patent Document 2, since priority control on the network side is used, it is possible to follow sudden fluctuations. However, in the currently popular network environment, the priority control function is often not implemented or cannot be used. In particular, in the Internet configured by interconnections of a plurality of organizations, it is not possible to use a consistent priority control function with End-End.

  SUMMARY OF THE INVENTION An object of the present invention is to solve the above-described problems of the prior art and provide an encoded data transmission technique and an encoded data decoding technique that enable priority control transmission in a network that does not have a priority control function.

  The present invention has been made to solve the above-described problems, and employs the following method. The present invention is directed to encoded data that can be divided into encoded data that can be partially decoded or encoded data of a hierarchical encoding method that includes multi-level encoded data.

  FIG. 1 shows a configuration diagram of an encoded data transmission apparatus according to the present invention. Reference numeral 1 denotes an encoded data transmission apparatus according to the present invention. In the encoded data transmission apparatus 1, 11 is an encoded data dividing unit that divides encoded data into encoded data that can be partially decoded, 12 is a priority determining unit that determines the priority of the divided encoded data, Reference numeral 13 denotes a transmission module that holds the encoded data in the buffer for each priority and outputs it or discards it according to the situation. Reference numeral 14 denotes an encoded data transmission unit that outputs the encoded data to the network.

  First, the encoded data dividing unit 11 divides the encoded data into encoded data that can be partially decoded. The divided encoded data is determined by the priority determining unit 12 according to its importance, and is sent to the transmission module 13 of each priority. Each transmission module 13 outputs or discards the encoded data stored in the buffer in the module. In addition, each transmission module 13 transmits buffer occupancy information to the transmission module 13 having a lower priority.

  The encoded data transmission unit 14 outputs the encoded data of the transmission module 13 having a high priority based on the current transmission state. The encoded data transmission unit 14 receives the transmission status information from the receiving side, for example, and extracts the encoded data transmitted from the buffer in each transmission module 13 when the reception side confirms the reception.

  The encoded data transmission unit 14 can use network congestion information as transmission status information used for transmission control, instead of reception confirmation information from the reception side. In this case, the amount of code that can be transmitted is calculated based on network congestion information such as packet loss rate, delay time, and transmission bandwidth, and the encoded data is transmitted.

  As the transmission status information, reception confirmation information on the receiving side and network congestion information can also be used together. When these are used together, the encoded data transmission unit 14 weights both and transmits the amount of code that can be transmitted. Or a method of preferentially handling reception confirmation information and using network congestion information only when reception confirmation is impossible.

  Next, the operation of the transmission module 13 will be described. A configuration diagram of the transmission module 13 is shown in FIG. The transmission module 13 includes a buffer 131, a switch control unit 132, a first switch 133, a second switch 134, a timer 135, and an encoded data deletion unit 136. The input signal to the transmission module 13 is encoded data and higher priority. The buffer occupancy information and output signal of the transmission module 13 are encoded data and in-module buffer occupancy information.

  The buffer 131 is for temporarily storing encoded data. The first switch 133 is connected to the second switch 134 and the encoded data deletion unit 136. The timer 135 switches the first switch 133 when a certain time elapses. The switch control unit 132 opens and closes the second switch 134 based on the buffer occupancy information of the higher priority transmission module 13.

  First, the encoded data distributed by the priority order determination unit 12 is accumulated in the buffer 131 in the transmission module 13. At this time, the first switch 133 is connected to the second switch 134. When a certain time has elapsed after the encoded data is input, the first switch 133 is switched to transmit the encoded data to the encoded data deleting unit 136 and delete it from the buffer 131. The timer 135 in the transmission module 13 is for counting elapsed time.

  The switch control unit 132 controls the opening / closing of the second switch 134 from the buffer occupation amount of the transmission module 13 having a higher priority. For example, when the control to close the second switch 134 is performed only when the buffer occupation amount of the higher priority transmission module 13 becomes 0, the buffer of the higher priority transmission module 13 becomes empty, that is, The encoded data is output after the transmission of the encoded data with the higher priority is completed.

  In the transmission module 13 with the highest priority, since there is no higher transmission module 13, the buffer occupancy information and the switch control unit 132 of the transmission module 13 with the higher priority are unnecessary, and the second switch 134 is always used. Closed state. Further, in the transmission module 13 with the lowest priority, there is no lower transmission module, so that it is not necessary to output the occupation amount information of the buffer in the module.

  In this way, according to the present invention, priority control of encoded data transmission can be performed according to the priority order for each divided encoded data. The encoded data transmission apparatus 1 shown in FIG. 1 can also be directly connected as an encoding buffer after the encoding portion of the video encoding system. In this case, since the original encoding buffer can be reduced, the delay time can be shortened.

  The role of the first switch 133 in the encoded data transmission apparatus 1 is to enable unnecessary encoded data to be deleted from the buffer 131 of the transmission module 13. This eliminates the need to delete unnecessary encoded data on the network transmission path or on the decoding side, realizes priority control transmission in a network without a priority control function, and the actual transmission rate fluctuates rapidly. Therefore, it is possible to transmit encoded data that can also be used.

  Next, a method for decoding the encoded data transmitted by the encoded data transmission method of the present invention will be described. In the present invention, since encoded data is transmitted based on priority, the order of encoded frames and the order of received encoded data may not match. For example, in the encoded data of the current frame having a low priority and the encoded data of the next frame having a high priority, the latter is preferentially transmitted. Therefore, it is necessary to divide each encoded data of the decoding target frame.

  FIG. 3 shows a configuration diagram of the encoded data decoding apparatus according to the present invention. Reference numeral 2 denotes an encoded data decoding apparatus according to the present invention. In the encoded data decoding apparatus 2, 21 is a decoded data selecting unit that selects received encoded data for each decoding target frame, 22 is a decoding buffer for storing encoded data of the decoding target frame, and 23 is a decoding target frame. A decoding target frame selection unit 24 that extracts encoded data from the decoding buffer 22 is a decoding unit that decodes the encoded data.

  The encoded data received by the encoded data decoding apparatus 2 is selected for each decoding target frame by the decoded data selection unit 21 and sent to the decoding buffer 22. The decoding target frame selection unit 23 extracts the encoded data of the decoding target frame from the decoding buffer 22 and sends it to the decoding unit 24. The decoding unit 24 decodes the encoded data. In this way, encoded data transmitted under priority control can be decoded.

  In the present invention, priority is determined for the divided partially decoded encoded data, and priority control of encoded data transmission is performed based on the priority. According to the present invention, it is possible to perform priority control transmission in a network having no priority control function. Further, if the encoded bit stream can be divided in advance, re-encoding is not necessary.

  Examples of the present invention are shown below. In the embodiment of the present invention, the encoded data is divided into four, and the transmission status is confirmed by confirming reception of the transmitted encoded data. For transmission modules, the timer setting time is 1 second and is the same for all transmission modules. The condition for closing the second switch is when the buffer occupancy of the higher priority transmission module becomes zero. The encoding frame rate is 2 frames / second.

  FIG. 4 shows a configuration diagram of the encoded data transmission apparatus according to the first embodiment of the present invention. 3 is an encoded data transmission device, 15 is a first transmission module, 16 is a second transmission module, 17 is a third transmission module, and 18 is a fourth transmission module. The transmission modules are numbered 1 to 4 in descending order of priority. The first transmission module 15 has the highest priority, and the fourth transmission module 18 has the lowest priority.

  First, the encoded data dividing unit 11 divides the encoded data of one image frame into four encoded data. The divided encoded data is sent to one of the first to fourth transmission modules 15 to 18 according to the priority determined by the priority determining unit 12. Each of the transmission modules 15 to 18 has the same configuration as that shown in FIG.

  In each transmission module 15 to 18, the encoded data in the buffer 131 shown in FIG. 2 is held for a certain time set in the timer 135. After a predetermined time has elapsed, the encoded data deletion unit 136 discards the data. Also, the encoded data is output when the buffer 131 of the higher priority transmission module becomes empty. Note that the second switch 134 of the transmission module 15 with the highest priority is always closed, and the encoded data of the buffer 131 can be output.

  Outputs of the transmission modules 15 to 18 are connected to the encoded data transmission unit 14 and output encoded data according to the transmission status of the encoded data. The encoded data transmission unit 14 extracts the encoded data transmitted from the buffer 131 in each of the transmission modules 15 to 18 when the reception confirmation on the receiving side is obtained.

  In the present embodiment, the operating condition of the second switch 134 is when the buffer occupation amount of the higher priority transmission module becomes 0. Therefore, there is encoded data in the buffer 131 of the transmission module, and the second switch 134 The number of transmission modules 134 is always closed is 1 or less. For this reason, the inputs of the encoded data transmission unit 14 are connected together as shown in FIG.

  In the above embodiment, the timer setting time set in the timer 135 shown in FIG. 2 is set to 1 second and is the same for all the transmission modules 13. However, the priority is changed by changing the timer setting time according to the priority order. It is also possible to change the timing of discarding the encoded data according to the order. Thereby, the retention period of the encoded data can be lengthened as the priority is higher, and the retention period can be shortened as the priority is lower.

  Next, as a second embodiment, an example will be described in which the operating condition of the second switch 134 is the time when the buffer occupancy of the transmission module with the higher priority becomes equal to or less than a predetermined threshold value TH. The configuration of the encoded data transmission apparatus according to the second embodiment is assumed to be the same as that shown in FIG. The configuration of each transmission module 13 is assumed to be as shown in FIG.

  In the second embodiment, the number of transmission modules in which the second switch 134 is closed is 1 or more. Therefore, the input of the encoded data transmission unit 14 is divided for each transmission module 13, unlike the first embodiment of FIG. 4. By dividing the output of each transmission module 13, it becomes possible to prepare for transmission before the buffer 131 of the higher priority transmission module 13 becomes empty, and it becomes easy to cope with a sudden change in transmission status.

  The operating condition of the second switch 134 is not only when the buffer occupancy of the transmission module 13 of the higher priority becomes equal to or less than a predetermined threshold TH, but also for the encoded data in the buffer 131 in the transmission module 13 of the higher priority. It can also be a time when the occupation ratio becomes equal to or less than a predetermined threshold value THR. In this case, the second switch 134 is closed when the buffer occupancy rate of the transmission module 13 with the higher priority falls below a certain threshold value THR, and the encoded data can be output from the buffer 131.

  Next, as a third embodiment, an example in which the discard of encoded data is linked between the transmission modules will be described. The configuration of the encoded data transmission apparatus according to the third embodiment of the present invention is shown in FIG. In FIG. 5, the encoded data transmission device 4 includes four transmission modules with a higher priority in the order of the first transmission module 31 to the fourth transmission module 34.

  In the previous embodiment, the discard of the encoded data is closed to each transmission module, but wasteful encoded data can be efficiently discarded by cooperation between the transmission modules. For example, when the encoded data is discarded in a certain transmission module, the encoded data of the same encoded frame is discarded regardless of the timer set time in the transmission module of other lower priority. This is because encoded data with higher priority is discarded, and encoded data with lower priority is not necessary. Thereby, unnecessary encoded data can be deleted.

  FIG. 6 shows the configuration of the transmission module used in the third embodiment. Here, the third transmission module 33 will be described as an example. In FIG. 6, 301 is a buffer, 302 is a switch control unit, 303 is a first switch, 304 is a second switch, 305 is a timer, and 306 is an encoded data deletion unit.

  In the third embodiment, the encoded data deletion information is added to the input signal in order to acquire the encoded data deletion information from the higher priority transmission module. Also, an output terminal for transmitting the encoded data deletion to the lower transmission module is added.

  The operation of the third transmission module 33 will be described. The encoded data deletion information input to the third transmission module 33 takes a logical sum with the output of the timer 305 and controls the opening / closing of the first switch 303. As a result, the encoded data in the buffer 301 can be deleted when the encoded data is discarded in the higher priority transmission module regardless of the output of the timer 305. At the same time, the encoded data deletion information is transmitted from the encoded data deletion unit 306 to the lower fourth transmission module 34. Thereby, the encoded data deletion between the transmission modules can be linked.

  The first transmission module 31 with the highest priority is not input with the encoded data deletion information and the buffer occupancy information from the transmission module with the higher priority. The fourth transmission module 34 having the lowest priority does not output buffer occupancy information and encoded data deletion information.

  In each of the embodiments shown here, when the set time of the timer is longer than the interval of the encoded frames, the order of the encoded frames may be different from the order of the encoded data to be transmitted. For example, consider a case where the encoded data of the current encoded frame cannot be transmitted before the encoded data of the next encoded frame arrives.

  The encoded data of the next frame includes data having a higher priority than the encoded data in the buffer. For this reason, the buffer occupancy of the higher priority transmission module becomes non-zero, and the encoded data of the low priority current frame is stored in the buffer, and the high priority encoded data of the next frame first. Is transmitted. By such a method, a priority is set for each divided encoded data, and the encoded data can be transmitted according to the priority.

  Next, decoding of encoded data transmitted using the encoded data transmission apparatus according to the embodiment of the present invention will be described in accordance with the configuration example of the encoded data decoding apparatus 2 shown in FIG. In this embodiment, the set time of the timer is 1 second and the encoding frame rate is 2 frames / second, so the number of necessary decoding buffers is 2.

  The encoded data received by the encoded data decoding apparatus 2 is selected for each decoding target frame by the decoded data selection unit 21 and sent to the decoding buffer 22. The decoding target frame selection unit 23 switches the decoding buffer 22 according to the encoding frame rate on the transmission side, and sends the encoded data to the decoding unit 24. The decoding buffer 22 of the sent decoding target frame is reused as the decoding buffer 22 of a new decoding target frame. In this way, encoded data transmitted under priority control according to the present embodiment can be decoded.

It is a block diagram of an encoded data transmission apparatus. It is a block diagram of a transmission module. It is a block diagram of an encoded data decoding apparatus. It is a block diagram of an encoded data transmission apparatus. It is a block diagram of an encoded data transmission apparatus. It is a block diagram of a transmission module. It is a figure which shows the operation | movement principle of a 1st conventional method. It is a figure which shows the operation principle of the 2nd conventional method.

Explanation of symbols

1, 3, 4 Encoded data transmission device 2 Encoded data decoding device 11 Encoded data division unit 12 Priority determining unit 13 Transmission module 14 Encoded data transmission unit 15, 31 First transmission module 16, 32 Second transmission module 17, 33 Third transmission module 18, 34 Fourth transmission module 21 Decoded data selection unit 22 Decoding buffer 23 Decoding target frame selection unit 24 Decoding unit 131, 301 Buffer 132, 302 Switch control unit 133, 303 First switch 134, 304 Second switch 135, 305 Timer 136, 306 Encoded data deletion unit 200, 203 Encoding device 201, 204 Decoding device 202, 205 Network

Claims (8)

An encoded data transmission device that transmits encoded data divided into encoded data that can be partially decoded, with a transmission amount changed according to a transmission situation,
Means for determining a priority of transmission of the divided encoded data;
Encoded data storage means for storing encoded data according to the determined priority and selecting encoded data to be transmitted according to the priority;
Means for confirming the transmission status of the encoded data, and transmitting the selected encoded data according to the transmission status;
Means for discarding encoded data remaining in the encoded data storage means without being transmitted for a predetermined time or more ;
The encoded data storage means includes
A buffer for storing encoded data for each priority;
Means for checking buffer occupancy or occupancy of higher priority when transmitting encoded data, and controlling output of encoded data from the buffer based on the confirmed result;
Means for transmitting the current buffer occupancy or occupancy to the means for controlling the output from the lower priority buffer;
An encoded data transmission apparatus that performs priority control of encoded data transmission based on the buffer occupancy or occupancy rate of each priority . In the encoded data transmission apparatus according to claim 1,
Means for setting an encoded data retention period for each priority order;
An encoded data transmission device, wherein the timing of discarding encoded data is changed according to each priority. In the encoded data transmission apparatus according to claim 1,
The means for discarding the encoded data is:
Means for detecting the priority of the encoded data to be discarded;
An encoded data transmission apparatus comprising: means for extracting and discarding encoded data of the same encoded frame that is lower than the detected priority order. An encoded data decoding apparatus that receives and decodes encoded data transmitted by the encoded data transmission apparatus according to any one of claims 1 to 3 ,
Means for storing encoded data for each decoding target frame;
Means for selecting encoded data of a decoding target frame after elapse of a predetermined time;
An encoded data decoding apparatus comprising: decoding means for reproducing a decoded image from the encoded data of the selected decoding target frame. An encoded data transmission method for transmitting encoded data divided into encoded data that can be partially decoded by changing a transmission amount according to a transmission situation,
Determining a priority of transmission of the divided encoded data;
Storing encoded data to be transmitted according to the determined priority, and selecting encoded data to be transmitted according to the priority;
Confirming the transmission status of the encoded data, and transmitting the accumulated encoded data according to the transmission status;
Have a discarding step the encoded data remaining to be transmitted a predetermined time or more,
In the step of accumulating and selecting the encoded data, the encoded data is stored in a buffer prepared for each priority,
When transmitting encoded data, the buffer occupancy or occupancy rate of higher priority is confirmed, and the output of encoded data from the buffer is controlled based on the confirmed result,
An encoded data transmission method, wherein priority control of encoded data transmission is performed based on the buffer occupancy or occupancy rate of each priority . In the encoded data transmission method according to claim 5 ,
An encoded data transmission method, wherein an encoded data retention time limit is set for each priority, and the timing of discarding the encoded data is changed according to each priority. In the encoded data transmission method according to claim 5 ,
When discarding encoded data, the priority of the target encoded data is detected, and the encoded data of the same encoded frame that is equal to or lower than the detected priority is extracted and discarded Encoded data transmission method. An encoded data decoding method for receiving and decoding encoded data transmitted using the encoded data transmission method according to any one of claims 5 to 7 , comprising:
Storing encoded data for each decoding target frame;
Selecting encoded data of a decoding target frame after elapse of a predetermined time;
A method of decoding the encoded data of the selected decoding target frame and reproducing the decoded image.

Images