JP2004193850A - Encoding control method and encoding control program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent deterioration of image quality more than in the conventional method. <P>SOLUTION: Only when receiving error notice data D4 from an image decoder, an intra-refresh is executed and an encoding frame rate is decreased at the same time. Thus, the deterioration in the image quality of a frame itself can be prevented by increasing a code amount assignable to the frame at a fixed bit rate by a share of the encoded frame rate set smaller and deterioration in the image quality in an entire moving picture such as uneven motion or loss of smoothness can also be prevented by suppressing a variation rate in the frame rate by a share of the frame rate made constant when no decoding error actually takes place on the decoder side. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an encoding control method and an encoding control program, and is suitably applied to, for example, a case where so-called streaming is performed in which compressed and encoded moving image data is reproduced while being received via a network.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, when an image encoding device performs compression encoding for each of a plurality of frames forming a moving image, for example, encoding using the entire encoding target frame (hereinafter, this is referred to as intra-frame encoding), Alternatively, coding using only a prediction error between a frame to be coded and a reference image corresponding to a previous frame that is one frame before the frame to be coded (hereinafter referred to as inter-frame coding) ).
[0003]
In this case, as illustrated in FIG. 9, the image encoding device performs intra-frame encoding on the first frame Fr1 of the plurality of frames to generate encoded data S1, and generates the encoded data S1 on the second and subsequent frames. For Fr2, Fr3,..., The encoded data S2, S3,. By doing so, the code amount is significantly reduced as compared with the moving image data forming the moving image.
[0004]
On the other hand, when decoding the encoded data S1, S2, S3,... Received from the image encoding device, the image decoding device performs intra-frame decoding on the encoded data S1 to execute the first frame. .. Is restored, and the second frame Fr2 is restored by performing inter-frame decoding using the frame Fr1 and the encoded data S2. Similarly, the third and subsequent frames Fr2, Fr3,. Restore sequentially.
[0005]
Therefore, in the image decoding apparatus, for example, if the encoded data S2 cannot be received due to loss during transmission or the like, not only a restoration error occurs in the second frame Fr2, but also the third and subsequent frames Fr3, Fr4, .., A restoration error is sequentially propagated.
[0006]
Therefore, the image coding apparatus performs intra-frame coding (hereinafter, referred to as intra refresh) for the second and subsequent frames Fr3, Fr4,... At predetermined intervals in advance to avoid propagation of a restoration error. By doing so, it is possible to prevent the propagation of the restoration error for all frames after the restoration error has occurred in the image decoding apparatus.
[0007]
In this case, in the image encoding device, since the amount of generated code is increased by the amount of performing the intra refresh (intra-frame encoding) at predetermined intervals, the encoded data is encoded via the network having the band limitation. , The amount of code to be generated in several frames after the frame on which the intra refresh is executed must be suppressed, and as a result, there is a problem in that the image quality of the suppressed frame itself deteriorates.
[0008]
As one conventional method for solving such a problem, it is set in advance so that intra refresh is executed at regular intervals, and the time until the next intra refresh is executed based on the amount of codes generated at the time of execution. By reducing the frame rate between frames to a value smaller than the frame rate at the time of input by thinning out a predetermined number of frames, the code amount that can be allocated to the frames during the fixed bit rate is increased by the amount set to a small frame rate. There has been proposed an image coding device designed to improve the image quality of a frame itself (for example, Patent Document 1).
[0009]
[Patent Document 1]
JP-A-10-229561
[0010]
[Problems to be solved by the invention]
However, in an image encoding device to which such a method is applied, the frame rate is variable based on the amount of generated codes when intra refresh is executed at fixed intervals regardless of whether or not an actual restoration error occurs in the image decoding device. By doing so, the number of frames to be thinned out varies according to the generated code amount at the time of execution.
[0011]
Therefore, for example, every time an intra-refresh is executed, the image coding apparatus performs a fixed interval for executing the intra-refresh when the complexity of a picture in the frame at the time of the execution is different and a generated code amount is significantly different. As the fluctuation width of the number of frames to be thinned out increases every time, the image quality of the entire moving image such as uneven motion and smoothness deteriorates, and it is still insufficient to prevent the image quality from deteriorating.
[0012]
The present invention has been made in consideration of the above points, and has as its object to propose an encoding control method and an encoding control program capable of preventing image quality deterioration as compared with the related art.
[0013]
[Means for Solving the Problems]
In order to solve such a problem, according to the present invention, among a plurality of frames forming a moving image, intra-frame encoding is performed on a first frame, and inter-frame coding is performed on each frame after the first frame. Encoding is sequentially executed, and when error notification data indicating the occurrence of a decoding error is received from a decoding side that decodes a frame on which intra-frame encoding or inter-frame encoding has been performed, encoding is performed next to the time when the error notification data is received. After the intra-frame encoding is performed on the target next target frame, the inter-frame encoding is sequentially re-executed for each frame after the next target frame, and the encoded frames per unit time are encoded. An encoding frame rate representing a number is reduced to a predetermined state.
[0014]
Upon receiving the error notification data from the decoding side, the intra-frame coding is performed on the next target frame, and the coding frame rate is reduced to a predetermined state, so that the code amount that can be allocated to the frame is reduced. In addition, it is possible to prevent the image quality of the frame itself from deteriorating by increasing the frame rate by a small amount.
[0015]
In addition to this, of the frames after the first frame, by executing intra-frame encoding and reducing the encoding frame rate only when error notification data is received from the decoding side, When no decoding error actually occurs on the decoding side compared to the conventional method in which the frame rate was changed when intra-frame encoding was performed at regular intervals regardless of whether or not an actual decoding error occurred. The deterioration of the image quality of the entire moving image, such as uneven motion or smoothness, can be prevented by making the frame rate constant.
[0016]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
[0017]
(1) Configuration of moving image transmission system
In FIG. 1, reference numeral 1 denotes a moving image transmission system as a whole, in which an encoding side camera apparatus 2, an image encoding apparatus 3, and an encoding side transmission / reception apparatus 4, which are transmission sources, and a decoding side, which is a transmission destination, are decoded. The transmission / reception device 5 includes an encoding / transmission / reception device 5, an image decoding device 6, and a display device 7, and is connected between the encoding / transmission / reception device 4 and the decoding / transmission / reception device 5 via a network 8 such as the Internet. Various types of information are transmitted and received.
[0018]
The camera device 2 generates moving image data D <b> 1 by capturing an image of a subject, and sends the generated moving image data D <b> 1 to the image encoding device 3.
[0019]
The image encoding device 3 compresses and encodes the moving image data D1 on a frame-by-frame basis. For the frame data corresponding to the first frame (hereinafter, referred to as the first frame), the entire frame is encoded. By executing the used encoding (hereinafter, referred to as intra-frame encoding), encoded data (hereinafter, referred to as intra-frame encoded data) D2 is generated.
[0020]
In addition, the image encoding device 3 determines that each frame data corresponding to the frames after the first frame is a frame to be currently encoded (hereinafter, referred to as a current frame), and is encoded one frame before the current frame. By performing an encoding process (hereinafter, referred to as inter-frame encoding) using only a prediction error with respect to a reference image corresponding to a frame to be encoded (hereinafter, referred to as a previous frame), Coded data (hereinafter, referred to as inter-frame coded data) D3.
[0021]
The image encoding device 3 sequentially transmits the intra-frame encoded data D2 and the plurality of inter-frame encoded data D3 at predetermined timings via the encoding-side transmitting / receiving device 4, the network 8, and the decoding-side transmitting / receiving device 5. The data is transmitted to the decoding device 6.
[0022]
The image decoding device 6 sequentially accumulates the intra-frame encoded data D2 and the plurality of inter-frame encoded data D3 sequentially given at a predetermined timing from the decoding transmitting / receiving device 5 in the internal buffer 6A for a predetermined time.
[0023]
Then, each time the intra-frame encoded data D2 and the plurality of inter-frame encoded data D3 are read out in the decoding order from the internal buffer 6A, the image decoding device 6 performs the same compression encoding (intra-frame encoding) as the image encoding device 3. The decoding processing is performed in accordance with the decoding or the inter-frame coding, thereby sequentially restoring the moving image data D1 on a frame-by-frame basis and sending the restored moving picture data D1 to the display device 7 in the order of the restored frames.
[0024]
In this manner, the moving image transmission system 1 executes streaming for reproducing the moving image data D1 obtained from the camera device 2 while downloading the moving image data D1 via the network 8, and sequentially performs the video of the subject based on the moving image data D1. It is provided through a display device 7.
[0025]
Incidentally, the error detection unit 6B of the image decoding device 6 uses the error correction algorithm based on the intra-frame coded data D2 and the plurality of inter-frame coded data D3 accumulated in the internal buffer 6A to perform correction by an error correction algorithm or a network protocol. Thus, irrecoverable coded data, such as when discarded and not retransmitted, is detected as a transmission error.
[0026]
Specifically, for example, upon detecting a transmission error in an arbitrary inter-frame encoded data D3 among the plurality of inter-frame encoded data D3, the error detection unit 6B notifies the image encoding device 3 of the transmission error. The notification data D4 is generated and transmitted to the image encoding device 3 via the decoding transmitting / receiving device 5, the network 8, and the encoding transmitting / receiving device 4 in order.
[0027]
Upon receiving the error notification data D4, the image coding apparatus 3 refreshes the frame to be coded next to the current frame CF at the time of receiving the error notification data D4 (hereinafter, refreshed as shown in FIG. 2). Only the RF (referred to as a target frame) is subjected to intra-frame encoding (hereinafter referred to as intra-refresh) for avoiding propagation of a restoration error as described above with reference to FIG. This is called refresh coded data) D5, and this is transmitted to the image decoding device 6.
[0028]
Incidentally, the image encoding device 3 executes inter-frame encoding for frames after the refresh target frame RF (hereinafter, referred to as refreshed frames) Fr10, Fr11,....
[0029]
The image decoding device 6 executes a decoding process using the refresh coded data D5 received from the image coding device 3 via the network 8, thereby detecting any transmission error detected by the transmission error detection unit 6B. It is possible to prevent a restoration error from being propagated for all frames after the frame corresponding to the inter-frame coded data D3.
[0030]
The image decoding device 6 detects a fatal transmission error that may cause a restoration error in advance, and transmits the error notification data D4 to the image encoding device 3, thereby generating an actual restoration error. Is detected, the refresh coded data D5 can be obtained much earlier than in the case of detecting the recovery error to prevent the propagation of a restoration error.
[0031]
As described above, in the moving image transmission system 1, the inter-frame coding is performed for each frame after the first frame, and the intra refresh (intra-frame coding) is performed only when the image decoding device 6 detects a transmission error. By doing so, it is possible to sequentially transmit and receive the intra-frame encoded data D2 and the plurality of inter-frame encoded data D3 whose code amount is significantly reduced as compared with the moving image data D1.
[0032]
(2) Configuration of image encoding device
As shown in FIG. 3, the image encoding device 3 includes an encoding processing unit 10 and an encoding control processing unit 11.
[0033]
The encoding processing unit 10 temporarily accumulates the moving image data D1 of 30 frames per second (30 [fps (frame per sec)]) provided from the camera device 2 (FIG. 1) in the image reordering buffer 21 and temporarily stores the moving image data D1. The moving image data D1 is rearranged in a compression coding order in frame units, and the rearranged frames are sequentially transmitted to the skip switch 22 as frame data D10.
[0034]
The skip switch 22 opens and closes the switch terminal 22a in accordance with opening and closing control data D24 for opening and closing the switch terminal 22a. At the time of opening, the skip switch 22 discards the frame data D10 given from the image rearranging buffer 21. On the other hand, when the frame is closed, the frame data D10 is sent to the encoding changeover switch 23, so that the frame rate (30 [fps]) of the moving image data D1 is reduced by the number of frames to be encoded per unit time. To the frame rate to be represented (hereinafter, referred to as an encoding frame rate).
[0035]
Specifically, the skip switch 22 thins out a plurality of frames forming a moving image every other frame as shown in FIG. , The frame rate of 30 frames per second is changed to an encoding frame rate of 15 frames per second.
[0036]
The encoding changeover switch 23 (FIG. 3) connects the connection piece 23a to the second terminal 23c, and sequentially transmits each frame data D10 sequentially supplied from the skip switch 22 to the inter-frame encoding unit 25.
[0037]
Further, when the switching control data D23 for switching the connection piece 23a from the second terminal 23c to the first terminal 23b for one frame is provided, the encoding switch 23 switches the connection piece 23a connected to the second terminal 23c. To the first terminal 23b, and sends the frame data D10 given from the skip switch 22 to the intra-frame encoding unit 24 at the time of the switching.
[0038]
Then, the encoding changeover switch 23 sends the frame data D10 given from the switch 22 at the time of the switching to the intra-frame encoding unit 24, and then reconnects the connection piece 23a to the second terminal 23c again, and sequentially supplies the connection piece 23a from the skip switch 22. The frame data D10 is sequentially transmitted to the inter-frame encoding unit 25.
[0039]
The intra-frame coding unit 24 performs intra-frame coding on the frame data D10 provided from the first terminal 23b based on the target code amount data D22 provided from the target code amount calculation unit 31, thereby performing the intra-frame coding. The coded data D2 is generated and temporarily stored in the buffer memory 26.
[0040]
The inter-frame coding unit 25 performs inter-frame coding on the frame data D10 supplied from the second terminal 23c based on the target code amount data D22 provided from the target code amount calculation unit 31, thereby performing the inter-frame coding. The coded data D3 is generated and temporarily stored in the buffer memory 26.
[0041]
In this way, the encoding processing unit 10 sends the intra-frame encoded data D2 and the plurality of inter-frame encoded data D3 stored in the buffer memory 26 to the encoding-side transmitting / receiving apparatus 4 (FIG. 1) at a predetermined timing. It has been made to be.
[0042]
Incidentally, in the encoding processing unit 10, when the frame data D10 corresponding to the current frame to be encoded is input to the inter-frame encoding unit 25 (the intra-frame encoding unit 24), the frame data D10 corresponding to the previous frame is input. With respect to D10, after completion of the inter-frame encoding, it is accumulated in the buffer memory 26 as inter-frame encoded data D3 (intra-frame encoded data D2), and also corresponds to a frame to be encoded next to the current frame. The frame data D10 is input to the encoding switch 23.
[0043]
On the other hand, the VBV (Video Buffering Verify) occupation amount calculation unit 30 of the encoding control processing unit 11 is a virtual buffer (hereinafter referred to as VBV) corresponding to the actual internal buffer 6A (FIG. 1) in the image decoding device 6. Buffer), and calculates the occupancy in the VBV buffer.
[0044]
Specifically, the VBV occupancy calculating unit 30 reads the inter-frame coded data D2 (intra-frame coded data D3) corresponding to the previous frame from the buffer memory 26, and generates the read inter-frame coded data D2. Recognize the code amount.
[0045]
Then, the VBV occupancy calculating unit 30 converts the frame data D10 corresponding to the current frame by the inter-frame coding unit 25 (the intra-frame coding unit 24) based on the recognized generated code amount and a predetermined fixed bit rate. The occupation amount in the VBV buffer immediately before encoding is calculated, and the calculation result is sent to the target code amount calculation unit 31 and the frame rate determination unit 32 as occupation code amount data D21.
[0046]
The target code amount calculation unit 31 calculates a target code amount based on the occupied code amount data D21 given from the VBV occupation amount calculation unit 30 according to a predetermined target code amount calculation algorithm, and sets a target code amount representing the calculated target code amount. After generating the code amount data D22, it is sent to the intra-frame encoding unit 24 and the inter-frame encoding unit 25.
[0047]
For example, as shown in FIG. 5, the frame rate control unit 32 includes a coding frame rate table TB that indicates the reduction rate when the coding frame rate indicating the number of frames to be coded per unit time is reduced in a stepwise manner. It is stored in an internal memory (not shown) and is based on the frame rate (30 [fps]) and the frame size of the moving image data D1 stored in the image rearranging buffer 21 in the encoding frame rate table TB. The initial value of the encoding frame rate is set to, for example, 15 [fps].
[0048]
Then, based on the initially set encoding frame rate (15 [fps]) and the frame rate of moving image data D1 (30 [fps]), the frame rate control unit 32 performs frame skipping as shown in FIG. The number and the skip interval are determined, and the switching control data D24 is generated according to the determined frame skip number and the skip interval, and is transmitted to the skip switch 22.
[0049]
The encoding switching control unit 33 generates the switching control data D23 when the frame data D10 corresponding to the first frame is input to the encoding switching switch 23, and sends out the generated switching control data D23 to the encoding switching switch 23.
[0050]
As described above, until receiving the error notification data D4 (FIG. 1), the encoding control processing unit 11 instructs the encoding processing unit 10 on the basis of the initially set encoding frame rate (15 [fps]). Interframe coding is performed on each frame data D10 corresponding to a frame other than the first frame.
[0051]
On the other hand, upon receiving the error notification data D4 (FIG. 1), the encoding control processing unit 11 generates the switching control data D23 by the encoding switching control unit 33 and sends it to the encoding switch 23.
[0052]
In this case, the encoding processing unit 10 sends only the frame data D10 corresponding to the frame (the refresh target frame RF (FIG. 2)) input to the encoding changeover switch 23 to the intra-frame encoding unit 24, and Intra refresh is executed via the inner encoding unit 24 to generate refresh encoded data D5.
[0053]
When receiving the error notification data D4, the frame rate control unit 32 of the coding control processing unit 11 sets the initially set coding frame rate (15 [fps]) in the coding frame rate table TB (FIG. 5). An encoding frame rate of 10 frames per second, which is smaller by one stage, is selected, and determined according to the selected encoding frame rate (10 [fps]) and the frame rate of the moving image data D1 (30 [fps]). The switch change data D24 is generated according to the frame skip number and the skip interval, and is transmitted to the skip switch 22.
[0054]
In this case, as shown in FIG. 6, the encoding processing unit 10 outputs frames every three frames via the skip switch 22 for each of the refreshed frames Fr10, Fr11... As a result, the coding frame rate is reduced from 15 frames per second to 10 frames per second.
[0055]
As described above, when receiving the error notification data D4 (FIG. 1), the encoding control processing unit 11 causes the encoding processing unit 10 to execute the intra refresh and at the same time lower the encoding frame rate by one step. On the basis of the reduced encoding frame rate (10 [fps]), the inter-frame encoding is executed for each frame data D10 corresponding to the frames after the refresh target frame RF.
[0056]
Here, when the intra refresh is performed, the code amount generated by the inter-frame coding that should have been performed originally (hereinafter, referred to as the inter-frame code amount) is larger than the code amount. As long as the generated code amount (hereinafter, referred to as a difference code amount) is not offset from the inter-frame code amount to be generated in several frames after the intra refresh is executed, as shown by a dashed line in FIG. Then, the occupancy in the VBV buffer eventually falls below the minimum value Vmin, causing a breakdown (underflow).
[0057]
Further, when the difference code amount is offset from the inter-frame code amount to be generated in several frames after the intra refresh is executed, the image quality of the several frames themselves deteriorates.
[0058]
However, since the encoding control processing unit 11 reduces the encoding frame rate to the encoding frame rate simultaneously with the execution of the intra refresh, the encoding control processing unit 11 compares the encoding time BT of one frame before the intra refresh with the encoding time BT of the one frame after the intra refresh. The underflow is prevented without reducing the amount of generated code bits allocated per frame by an amount corresponding to the long encoding time AT, and as a result, intra refresh can be performed without deteriorating the image quality of the frame itself. It has been made executable.
[0059]
In addition to this, the frame rate control unit 32 determines the occupation amounts P1, P2, P3,... Of the VBV buffers represented by the occupied code amount data D21 sequentially given from the VBV occupation amount calculation unit 30, and a predetermined threshold value Vthr, and when the threshold Vthr is exceeded, the coding frame rate (10 [fps]) reduced by one stage is returned to the state of the coding frame rate (15 [fps]) at the time of the initial setting. It has been made.
[0060]
Here, the frame rate control unit 32 selects 2/3 of the maximum value Vmax of the VBV buffer as the predetermined threshold Vthr.
[0061]
This allows the frame rate control unit 32 to return to the encoding frame rate at the initial setting, for example, when the picture in each frame after returning to the encoding frame rate at the initial setting is complicated. Even if the generated code amount for the frame is increased, underflow can be almost certainly prevented.
[0062]
(3) Encoding control processing procedure
An encoding control processing procedure in the encoding control processing unit 11 will be described with reference to a flowchart shown in FIG.
[0063]
That is, when a predetermined input operation for executing, for example, streaming is performed through, for example, an input unit (not shown), the encoding control processing unit 11 proceeds from the start step of the routine RT to the next step SP1.
[0064]
In step SP1, the encoding control processing unit 11 sets an initial value of the encoding frame rate from the encoding frame rate table TB (FIG. 5) based on the moving image data D1 stored in the image reordering buffer 21, After transmitting the switching control data D24 generated according to the set initial value to the encoding processing unit 10, the process proceeds to the next step SP2.
[0065]
In step SP2, the encoding control processing unit 11 causes the encoding processing unit 10 to execute the intra-frame encoding for the first frame, and proceeds to the next step SP3.
[0066]
In step SP3, the encoding control processing unit 11 causes the encoding processing unit 10 to execute inter-frame encoding for the second frame, and proceeds to the next step SP4.
[0067]
In step SP4, the encoding control processing unit 11 determines whether or not the error notification data D4 has been received. Here, if a negative result is obtained, this means that the intra-frame coded data D2 or the inter-frame coded D3 obtained by causing the coding processing unit 10 to perform coding in step SP2 or SP3 is transmitted. Indicates that no transmission error has occurred. At this time, the encoding control processing unit 11 returns to step SP3 and continues to execute inter-frame encoding for the third frame.
[0068]
As described above, the encoding control processing unit 11 repeats the processing of step SP3 every time a negative result is obtained in step SP4, thereby performing the processing on the frames after the first frame based on the encoding frame rate initially set in step SP1. The interframe coding is sequentially performed.
[0069]
On the other hand, if a positive result is obtained in step SP4, this means that the intra-frame coded data D2 or inter-frame coding obtained by causing the coding processing unit 10 to perform coding in step SP2 or SP3. This indicates that a transmission error has occurred when D3 was transmitted, and at this time, the encoding control processing unit 11 proceeds to the next step SP5.
[0070]
In step SP5, the encoding control processing unit 11 selects an encoding frame rate reduced by one stage from the encoding frame rate initially set in step SP1, based on the encoding frame rate table TB (FIG. 5). After transmitting the switching control data D24 generated according to the selected encoding frame rate to the encoding processing unit 10, the process proceeds to the next step SP6.
[0071]
In step SP6, the encoding control processing unit 11 instructs the encoding processing unit 10 to execute the intra-frame encoding for the refresh target frame RF (FIG. 2) to be encoded next to the time when the positive result is obtained in step SP4. Then, the process proceeds to the next step SP7.
[0072]
In step SP7, the coding control processing unit 11 performs the inter-frame coding on the first refreshed frame Fr10 (FIG. 2) among the refreshed frames after the refresh target frame RF with respect to the coding processing unit 10. Then, the process proceeds to the next step SP8.
[0073]
In step SP8, the encoding control processing unit 11 performs the second refresh based on the interframe encoded data D3 obtained by executing the interframe encoding on the first refreshed frame Fr10 in step SP7. The occupation amount P1 in the VBV buffer immediately before the frame data D10 corresponding to the subsequent frame Fr11 (FIG. 2) is inter-coded is calculated, and the process proceeds to the next step SP9.
[0074]
In step SP9, the encoding control processing unit 11 determines whether or not the occupation amount P1 calculated in step SP8 has exceeded 最大 of the maximum value Vmax.
[0075]
If a negative result is obtained here, this means that if the generated code amount for each frame after returning to the coding frame rate at the initial setting increases, an underflow may occur. At this time, the encoding control processing unit 11 returns to step SP7 and continues to execute inter-frame encoding.
[0076]
As described above, the encoding control processing unit 11 repeats the processing of steps SP7 and SP8 every time a negative result is obtained in step SP9, thereby obtaining the remaining refreshed frames based on the encoding frame rate reduced in step SP5. (FIG. 2) corresponding to Fr11, Fr12... (FIG. 2) are sequentially executed, and the occupation amounts P2, P3... In the VBV buffer immediately before the execution are calculated.
[0077]
On the other hand, if an affirmative result is obtained in step SP9, this means that the underflow is almost certainly caused even if the generated code amount for each frame after returning to the encoding frame rate at the initial setting increases. In this case, the encoding control processing unit 11 proceeds to step SP10.
[0078]
In step SP10, the coding control processing unit 11 returns the coding frame rate reduced in step SP5 to the coding frame rate initialized in step SP1, returns to step SP3, and returns to the initially set coding frame rate. , The inter-frame coding is continuously executed.
[0079]
As described above, the encoding control processing unit 11 transmits the code of the initial setting to the encoding processing unit 10 according to the loop of steps SP2 and SP3 (hereinafter referred to as a normal processing loop) until receiving the error notification data D4. When the error notification data D4 is received, the intra-frame refresh is performed at the same time as the intra-refresh is performed at the same time when the error notification data D4 is received.
[0080]
Thereafter, the encoding control processing unit 11 performs inter-frame encoding on a frame-by-frame basis based on the encoding frame rate reduced by one step according to a loop of steps SP7 and SP8 (hereinafter, referred to as a reduction processing loop). .., Or Pn in the VBV buffer (FIG. 7) immediately before the execution is monitored in frame units. When the occupancy exceeds the threshold value Vthr, which is 2/3 of the maximum value Vmax, the normal operation is resumed. A processing loop is executed.
[0081]
(4) Operation and effect
In the above configuration, the encoding control processing unit 11 as the encoding control device instructs the encoding processing unit 10 to perform inter-frame encoding on each frame data D10 corresponding to frames after the first frame in the moving image. By doing so, the code amount is significantly reduced as compared with the moving image data D1.
[0082]
In this state, upon receiving the error notification data D4 from the image decoding device 6, the encoding control processing unit 11 instructs the encoding processing unit 10 to execute the refresh target frame to be encoded next to the time when the error notification data D4 is received. Intra-frame encoding is performed on frame data D10 corresponding to RF.
[0083]
After that, the encoding control processing unit 11 causes the encoding processing unit 10 to sequentially perform inter-frame encoding on each frame data D10 corresponding to the refreshed frames Fr10, Fr11,... After the refresh target frame RF. At the same time, the coding frame rate after the refresh target frame RF is reduced by one step from the initially set coding frame rate according to the coding frame rate table TB (FIG. 5).
[0084]
Therefore, the encoding control processing unit 11 performs the intra refresh by an amount corresponding to securing a longer encoding time AT for one frame after the intra refresh than the encoding time BT for one frame before the intra refresh (FIG. 7). , The underflow can be prevented without suppressing the code amount to be generated in several frames after the execution, and as a result, the intra refresh can be executed without deteriorating the image quality of the frame itself. .
[0085]
In addition, in the related art, the frame rate is changed based on the generated code amount or the like when the intra refresh is executed at regular intervals regardless of whether or not an actual restoration error is generated or not in the image decoding device 6. In the case where the complexity of the picture in each frame in D1 was different, the fluctuation of the frame rate became larger at regular intervals, whereas the encoding control processing unit 11 Since the intra-refresh is executed only when D4 is received and the encoding frame rate is reduced, the fluctuation of the frame rate is equivalent to keeping the encoding frame rate constant when no restoration error actually occurs. Rate can be significantly reduced, and therefore the image quality of the entire moving image, such as uneven motion or smoothness, deteriorates. It can also be prevented.
[0086]
Further, the encoding control processing unit 11 determines the code amount of the VBV buffer immediately before encoding the frame data D10 for each frame data D10 corresponding to the frames Fr10, Fr11... (FIG. 2) after the refresh target frame RF. .. Are calculated, and when the calculated result exceeds a predetermined threshold value Vthr in the VBV buffer, the coding frame rate is returned to the state of the encoding frame rate at the initial setting.
[0087]
In this case, the encoding control processing unit 11 sequentially calculates the code amounts P1, P2, P3,... Of the VBV buffer immediately before encoding for each frame, so that the image quality of the frame itself deteriorates (underflow). The timing at which inter-frame coding can be executed can be detected quickly based on the coding frame rate at the time of initialization, and the coding frame rate at the time of initialization can be quickly returned.
[0088]
As a result, the encoding control processing unit 11 can shorten the encoding time AT which is longer than the encoding time BT for one frame before the intra refresh (FIG. 7) as much as possible, and thus reduce the moving image data D1. It can be transmitted as fast as possible.
[0089]
According to the above configuration, the intra-refresh is executed only when the error notification data D4 is received, and at the same time, the encoding frame rate is reduced, so that the intra-refresh is performed without deteriorating the image quality of the frame itself. Can be executed, and the fluctuation rate of the frame rate when a restoration error does not actually occur is significantly suppressed as compared with the conventional method, so that the deterioration of the image quality of the entire moving image such as uneven motion and smoothness can be reduced. Can be prevented.
[0090]
(5) Other embodiments
In the above embodiment, when the error notification data D4 is received, an encoding frame rate smaller by one stage than the initially set encoding frame rate is selected based on the encoding frame rate table TB (FIG. 5). However, the present invention is not limited to this, and the occupation amount PX of the VBV buffer when the refresh target frame RF (FIG. 2) upon receiving the error notification data D4 is intra-refreshed (FIG. 7) , The reduction rate of the encoding frame rate may be selected.
[0091]
More specifically, the encoding control processing unit 11 causes the encoding processing unit 10 to execute the intra-frame encoding for the refresh target frame RF (FIG. 2) in step SP6 (FIG. 8), and then sets the VBV occupation amount. The calculating unit 30 calculates the occupation amount PX in the VBV buffer based on the refresh coded data D5 obtained by the execution, and when the calculated occupation amount PX is greatly apart from the minimum value Vmin of the VBV buffer, After the frame rate control unit 32 selects a coding frame rate that is smaller than the initially set coding frame rate by, for example, one step, the process proceeds to next step SP7 and performs inter-frame coding of the refreshed frame Fr10 (FIG. 2). Is executed.
[0092]
On the other hand, when the calculated occupation amount PX is close to the minimum value Vmin of the VBV buffer, the encoding control processing unit 11 causes the frame rate control unit 32 to perform two steps, three steps,. After selecting an encoding frame rate that is smaller by a plurality of steps as in..., The process proceeds to the next step SP7 to execute inter-frame encoding for the refreshed frame Fr10 (FIG. 2).
[0093]
As described above, the reduction rate of the coding frame rate is determined according to the occupancy PX (FIG. 7) of the VBV buffer at the time when the refresh target frame RF (FIG. 2) upon receiving the error notification data D4 is intra-refreshed. If this is selected, the difference code amount (the difference between the code amount at the time of executing the intra refresh and the code amount generated by the inter-frame coding which should have been originally executed at the time of the execution) is cancelled. The encoding time AT (FIG. 7) can be secured accurately without waste.
[0094]
Further, in the above-described embodiment, the case where the initially set encoding frame rate is reduced by one step when the error notification data D4 is received has been described. However, the present invention is not limited to this. When the error notification data D4 is received during the execution of the inter-frame encoding based on the encoding frame rate, the encoding frame rate may be further reduced.
[0095]
Specifically, for example, when the encoding control processing unit 11 receives the error notification data D4 again during execution of a loop of steps SP7 and SP8 (hereinafter, referred to as a withdrawal processing loop), The coding frame rate reduced by one step in step SP5 is further reduced by one step, so that inter-frame coding is performed on a frame basis based on the coding frame rate reduced by two steps from the coding frame rate at the time of initial setting. , Or Pn of the VBV buffer (FIG. 7) immediately before the execution is monitored in frame units. When the occupancy exceeds the threshold value Vthr which is 2/3 of the maximum value Vmax, the normal operation is resumed. Execute a processing loop. In this way, even if the error notification data D4 is continuously received, it is possible to prevent the image quality from deteriorating.
[0096]
Further, in the above-described embodiment, the case where the encoding frame rate is set according to the encoding frame rate table TB (FIG. 5) stored in the internal memory in advance has been described. The encoding frame rate may be set according to a predetermined operation using the amount of generated code of the encoded data D5.
[0097]
Furthermore, in the above-described embodiment, a case has been described where the error detection unit 6B (FIG. 1) detects a transmission error of the intra-frame encoded data D2 and the plurality of inter-frame encoded data D3 accumulated in the internal buffer 6A. However, in addition to this, the present invention may be configured to detect a restoration error that actually occurs during the decoding process of the image decoding device 6. What is necessary is just to detect the restored error.
[0098]
Further, in the above-described embodiment, a case has been described in which the encoding control processing unit 11 having the hardware configuration described above with reference to FIG. 3 executes the encoding control processing in accordance with the encoding control processing procedure described above with reference to FIG. However, the present invention is not limited to this, and instead of such a hardware configuration, an encoding control process is executed according to an encoding control program for executing an encoding control process along the encoding control process procedure. May be.
[0099]
In this case, the encoding control processing unit 11 stores the encoding control program in an internal ROM (not shown) or the like, and expands the encoding control program in an internal RAM (not shown) or the like to thereby perform the encoding control. The processing procedure may be executed, or the encoding control processing procedure may be executed by installing a program storage medium storing the encoding control program.
[0100]
As a program storage medium for installing the encoding control program for executing the above-described series of encoding control processes in the encoding control processing unit 11 and making it executable, for example, a floppy disk, CD -Not only a package medium such as a ROM (Compact Disc-Read Only Memory) and a DVD (Digital Versatile Disc), but also a semiconductor memory or a magnetic disk in which an encoding control program is temporarily or permanently stored may be realized. . As a means for storing the encoding control program in these program storage media, a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting may be used, or via various communication interfaces such as a router or a modem. It may be stored.
[0101]
【The invention's effect】
As described above, according to the present invention, among a plurality of frames forming a moving image, an intra-frame encoding is performed on a first frame, and an inter-frame encoding is performed on each frame after the first frame. And the error notification data indicating the occurrence of a decoding error is received from the decoding side that decodes the frame on which the intra-frame coding or the inter-frame coding has been performed. After performing the intra-frame coding for the next target frame, the inter-frame coding is sequentially performed again for each frame after the next target frame, and the number of coded frames per unit time. Is reduced to a predetermined state.
[0102]
In this case, the intra-frame coding is performed and the coding frame rate is reduced only when error notification data is received from the decoding side. Not only is the encoding frame rate increased by a small amount to prevent the deterioration of the image quality of the frame itself, but also the frame rate in the case where no decoding error actually occurs on the decoding side is fixed. By suppressing the rate of change of the frame rate, it is possible to prevent the deterioration of the image quality of the entire moving image such as uneven motion or smoothness, and thus it is possible to prevent the deterioration of the image quality as compared with the related art.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of a moving image transmission system.
FIG. 2 is a schematic diagram for explaining a temporal relationship between error notification data and intra refresh.
FIG. 3 is a block diagram illustrating a configuration of an image encoding device.
FIG. 4 is a schematic diagram illustrating a frame skip example (1).
FIG. 5 is a schematic diagram illustrating an example of an encoding frame rate table.
FIG. 6 is a schematic diagram illustrating a frame skip example (2).
FIG. 7 is a schematic diagram illustrating a transition example of a code amount in a VBV buffer.
FIG. 8 is a flowchart showing an encoding control processing procedure.
FIG. 9 is a schematic diagram used for describing an encoding example.
[Explanation of symbols]
1 ... moving image transmission system, 3 ... image encoding device, 6 ... image decoding device,
8: Network, 10: Encoding processing unit, 11: Encoding control processing unit, 30: VBV occupancy calculation unit, 32: Frame rate control unit, 33: Encoding switching control unit

Claims (5)

Among a plurality of frames forming a moving image, a first frame in which intra-frame encoding is performed on the first frame and an inter-frame encoding is sequentially performed on each of the frames subsequent to the first frame. Steps and
When receiving the error notification data indicating the occurrence of a decoding error from the decoding side that decodes the frame that has been subjected to the intra-frame encoding or the inter-frame encoding, the frame to be encoded next to the received time is A second step of performing the above-described intra-frame encoding on a certain next target frame;
A third method of sequentially re-executing the inter-frame coding for each of the frames following the next target frame and reducing the coding frame rate representing the number of frames to be coded per unit time at a predetermined reduction rate; Encoding control method, comprising the steps of: In the third step,
An occupancy calculation step of calculating the occupancy of the virtual buffer immediately before the encoding, each time the frames after the next target frame are sequentially inter-frame encoded,
When the occupancy calculated in the occupancy calculation step exceeds a predetermined threshold, the coding frame rate reduced at the predetermined reduction rate is returned to the original state. The encoding control method according to claim 1, wherein In the third step,
The encoding control method according to claim 1, wherein the reduction rate of the encoding frame rate is determined according to an occupation amount of a virtual buffer when the next target frame is encoded in the intra-frame. . In the third step,
4. The method according to claim 1, wherein said next target frame is selected from a predetermined table indicating a step-down rate of said encoding frame rate in accordance with an occupation amount of a virtual buffer at a time when said intra-frame encoding is performed. 3. The encoding control method according to 3. Among a plurality of frames forming a moving image, a first frame in which intra-frame encoding is performed on the first frame and an inter-frame encoding is sequentially performed on each of the frames subsequent to the first frame. Steps and
When receiving the error notification data indicating the occurrence of a decoding error from the decoding side that decodes the frame that has been subjected to the intra-frame encoding or the inter-frame encoding, the frame to be encoded next to the received time is A second step of performing the above-described intra-frame encoding on a certain next target frame;
A third method of sequentially re-executing the inter-frame coding for each of the frames following the next target frame and reducing the coding frame rate representing the number of frames to be coded per unit time at a predetermined reduction rate; An encoding control program, comprising the steps of:

Images