JP2004007619A - Image coding apparatus and method therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image coding apparatus capable of preventing the impairment of sharpness caused by the sharp decrease of the number of bits which can be assigned and the sudden change of a frame rate, in a case that scenes difficult to be coded are continued. <P>SOLUTION: A deriving part 31 for deriving the frame constellation target number of bits which can be assigned reads out from a memory 36 the total number of bits of frame constellation units to remaining frames, subtracts the generating number of bits used by coding the previous frames and updates, and transmits it to a deriving part 32 for deriving the following frame target number of bits. The deriving part 32 for deriving the following frame target number of bits computes the target number of bits assigned to the following frames based on the remaining frame number in consideration of the received total number of bits and frame rate values Rf. A deriving part 33 for deriving the mean or average frame number of bits computes the mean rate of the number of bits assigned to the former frames. An operation part 34 multiplies the mean rate of the computed number of bits by considered factors, compares this with a target value of the number of bits, and selects a greater one of values and transmits it to a deriving part 35 for a quantization step. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image encoding technique and an image decoding technique, and more particularly, to an image encoding technique using a variable bit rate method or a variable frame rate method.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, a variable bit rate (Variable Bit Rate) method is one of methods for controlling a bit rate in an image coding technique such as the MPEG (Moving Picture Expert Group) standard. This is mainly because of the restriction of the capacity of the buffer memory for storing the encoded data and the restriction of the video recording time when recording on the recording medium, so that local bits associated with the complexity of the image to be encoded are limited. This is a control method in which the number of generated bits in a predetermined time length (for example, one second) is suppressed to a certain value while allowing a change in the rate. That is, in a scene where encoding is difficult, more bits for encoding are generated (bit rate is increased), and in a scene where encoding is easy, the number of generated bits is reduced (bit rate is decreased). Controls the number of bits to be allocated according to the scene. That is, encoding is performed so that a high-quality image can be reproduced while keeping the number of generated bits within a fixed time constant (that is, within the constraints of the buffer memory capacity and the recording time). (For example, see Patent Document 1).
[0003]
An image encoding device using the above-mentioned conventional variable bit rate method will be described with reference to an image encoding device 100 in FIG. The image encoding apparatus 100 includes an orthogonal transform unit 105, a quantization unit 106, a variable length encoding unit 107, an inverse quantization unit 108, an inverse orthogonal transform unit 109, a frame memory 102, a motion detection unit 103, and a motion compensation unit 104. And a bit rate control unit 110.
[0004]
The orthogonal transform unit 105 performs a discrete cosine transform (DCT) on the received encoded frame signal 101 (image signal data) in macroblock units, generates DCT coefficients, and sends the DCT coefficients to the quantization unit 106. Output. Here, for a frame of an I (Intra coded) picture, a DCT operation is performed in the intra-picture coding mode. For a frame of a P (Predictive coded) picture, a DCT operation is performed in a forward prediction coding mode based on an I picture or a P picture located in the past in time. For a B (Bidirectionally) picture frame, a DCT operation is performed in a bidirectional predictive coding mode based on an I picture or a P picture located temporally before and after.
[0005]
The quantization unit 106 quantizes the DCT coefficient input from the orthogonal transformation unit 105 at a quantization step (or a quantization parameter) received from the bit rate control unit 110 for each macroblock, and performs variable length. It outputs to encoding section 107 and inverse quantization section 108. The variable-length coding unit 107 performs variable-length coding and multiplexing on the quantized DCT coefficients and the like input from the quantization unit 106, and outputs the result to an output buffer (not shown).
[0006]
The inverse quantization unit 108 performs an inverse quantization operation on the quantized DCT coefficient received from the quantization unit 106, and outputs the result to the inverse orthogonal transform unit 109. The inverse orthogonal transform unit 109 performs an inverse orthogonal transform operation based on the inversely quantized DCT coefficient input from the inverse quantization unit 108 to reproduce image signal data, and outputs the image signal data to the frame memory 102.
[0007]
The frame memory 102 adds and stores the decoded image signal data of the I picture or the P picture and the motion compensation data generated by the motion compensation unit 104. The motion detection unit 103 detects a motion vector from the reference image stored in the frame memory 102 and outputs data representing the motion vector to the motion compensation unit 104.
[0008]
The motion compensation unit 104 performs motion compensation data (reference image data) based on a reference image stored in the frame memory 102 for encoding a P picture or a B picture and data representing a motion vector input from the motion estimation unit 103. ). The bit rate control unit 110 receives the number of generated bits from the variable length coding unit 107, determines a quantization step based on the number of generated bits, and transmits the quantization step to the quantization unit 106.
[0009]
The overall control unit 140 is, for example, a microcomputer including a ROM, a RAM, and the like, and is a unit that controls the entire image encoding device 100. The overall control unit 140 controls each processing timing based on a control signal and the like.
[0010]
FIG. 33 is a block diagram illustrating a functional configuration of the bit rate control unit 110 in the conventional image encoding device 100. As shown in FIG. 33, the bit rate control unit 110 includes a frame group target bit number deriving unit 111, a next frame target bit number deriving unit 112, and a quantization step deriving unit 113.
[0011]
The frame group target bit number deriving unit 111 receives the generated bit number Nn131 from the variable length coding unit 107 and stores it in an internal memory (not shown). At this time, the frame group target bit number deriving unit 111 counts the number of times the number of generated bits Nn131 has been received (that is, the number of encoded frames). Further, the frame group target bit number deriving unit 111 calculates the number of bits that can be allocated to a frame that has not yet been coded for each frame group, transmits the calculated bit number to the next frame target bit number deriving unit 112, The number of bits that can be allocated is sequentially updated using the number of generated bits Nn131 generated as a result. Here, the “frame group” refers to a group of frames that can be encoded with a predetermined time length.
[0012]
The next frame target bit number deriving unit 112 derives a target value of the number of bits to be allocated to the next frame based on the number of bits that can be allocated for each frame group received from the frame group target bit number deriving unit 111, and performs quantization. This is transmitted to the step derivation unit 113. The target value is calculated by dividing the number of bits that can be allocated for each frame group at that time by the number of remaining frames.
[0013]
The quantization step deriving unit 113 calculates a quantization step 141 (or a quantization parameter) based on the target value of the number of bits to be allocated to the next frame received from the next frame target bit number deriving unit 112 and performs quantization. Output to the conversion unit 106.
[0014]
FIG. 34 is a flowchart showing the flow of processing in the overall control unit 140 and the bit rate control unit 110 of the conventional image encoding device 100.
[0015]
First, when the bit rate control unit 110 receives the number of generated bits Nn131 (S1401), the overall control unit 140 determines whether the next frame to be encoded is the head of the frame group (S1402). In this case, if the next frame is the first frame of the frame group (S1402: Yes), the bit rate control unit 110 initializes the number Na of assignable bits, the assignment target period Ta, and the number Nt of assignment target frames. (S1405 to 1407). Here, “NA” is an initial value of the number of bits to be allocated for each frame group. “TA” is the period of the entire frame group. “Rf” is the frame rate of the encoding of the image encoding device 100.
[0016]
On the other hand, when the next frame is not the first frame of the frame group (S1402: No), the bit rate control unit 110 updates the number Na of assignable bits and the assignment target period Ta (S1403 to 1404).
[0017]
Next, the bit rate control unit 110 calculates the number Nb of bits to be allocated based on the number Na of bits that can be allocated and the number Nt of frames to be allocated (S1408), and updates (decrements) the number Nt of frames to be allocated (S1409). .
[0018]
Thereafter, the bit rate control unit 110 determines whether or not the frame to be coded needs to be decimated. If the decimation is necessary, the bit rate control unit 110 notifies the general control unit 140 of that (S1410: Yes). On the other hand, when it is not necessary to perform the thinning (S1410: No), a quantization step is calculated and output to the quantization unit 106 (S1411).
[0019]
The overall control unit 140 and the bit rate control unit 110 repeat the above processing until the encoding processing ends (S1401 to 1412).
[0020]
FIG. 35 is a diagram showing a specific example of a method of calculating the number of allocated bits Nb in the conventional bit rate control section 110. In this case, for convenience, it is assumed that the number of generated bits Nn131 and the number of allocated bits Nb match.
[0021]
In the example shown in FIG. 35, image signal data (encoded frame signal 101) input at 30 [fps] is encoded at a frame rate of 15 [fps], and is composed of 15 frames in principle. One frame group is generated every second. In this example, the input encoded frame signal 101 is represented by “input image frame number 1701”, and the encoded frame signal 121 is represented by “frame group frame number 1702”. In FIG. 35, the columns of the frame group frame number 1702 and the number of generated bits Nn131 are indicated by “x” because the corresponding frame of the input image frame number 1701 is not coded and is “decimated”. Represents
[0022]
Further, in FIG. 35, for example, the number of generated bits Nn with respect to the input image frame number “9” (that is, the frame group frame number “5”) is 1180 bits in total of the number of bits already allocated to four frames, and Is (11), so ((3600-1180) / 11 = 220 bits).
[0023]
On the other hand, the number of generated bits for the input image frame number “13” (that is, the frame group frame number “6”) is the sum of the already allocated bit numbers because the input image frame number “11” is “decimated”. Is 1400 bits, and the number of remaining frames is “9”, so the fractional part is rounded down to ((3600-1400) / 9 = 244 bits).
[0024]
Note that “(14)” described in the column of the frame group frame number 1702 corresponding to the input image frame number “29” in FIG. 35 is the last frame when the input image frame number 1701 is “13”. This is the expected value of the group frame number 1702.
[0025]
As described above, in the conventional image encoding device 100, bits are assigned to the next encoding target frame.
[0026]
[Patent Document 1]
JP 2001-25015 A
[0027]
[Problems to be solved by the invention]
However, in the image encoding apparatus 100 using the variable bit rate in the above-described conventional technology, when frames that are difficult to encode (complex) continue, even if the quantization step is changed to a large value, the encoding is performed. The number of generated bits increases (in this case, the value of the bit rate increases). Then, in order to keep the number of generated bits in a predetermined time length (one frame group) constant, the number of bits to be assigned to a frame after a series of difficult-to-encode frames decreases. For this reason, there is a first problem that the quantization step must be increased for the subsequent frames, and the image quality of those frames is extremely deteriorated. The rate will be significantly smaller).
[0028]
FIG. 36 is a diagram for explaining the first problem of the above-described conventional technology. In FIG. 36, 15 frames from a frame 1601 to a frame 1602 are defined as a frame group, and 3600 [bits] are assigned to each frame group as a target value (as shown in FIG. 35). In this case, if scenes that are difficult to encode continue from time t1 and a large number of bits are assigned to the encoding of those frames, the number of bits that can be assigned in the latter half of the frame group will decrease, so the quantization step Needs to be increased (the bit rate decreases accordingly), but when the quantization step is set to a large value, the image quality is extremely deteriorated. In FIG. 36, the quantization step is particularly large between “t3 and t4”, and it can be assumed that the image quality is degraded accordingly.
[0029]
Furthermore, when the number of bits that can be finally allocated becomes smaller and the frame rate must be changed in the middle of a frame group (that is, the frames must be thinned out), the number of bits can be increased even though the frame rate has changed. Since the number of bits allocated to one frame is determined based on the initial frame rate, it is not possible to allocate more bits in spite of lowering the frame rate, and further reducing the frame rate or performing a quantization step There is a second problem that it is necessary to make the roughness coarse, which further deteriorates the image quality.
[0030]
FIG. 37 is a diagram for explaining the second problem of the above-described conventional technology. In the example of FIG. 37, scenes that are difficult to encode continue, and many bits are allocated to the encoding of those frames. As a result, the frame rate gradually decreases, resulting in a large frame interval. This shows an image in which the movement is awkward, and the quality of the image is degraded.
[0031]
Therefore, the present invention has been made in view of the above-described problems, and even when a series of difficult-to-encode scenes continues, image quality degradation due to an extremely shortage of bits that can be allocated to the remaining frames. It is an object of the present invention to provide an image encoding apparatus and method capable of performing encoding for a high-quality image while also considering the change in the rate of a frame to be encoded.
[0032]
[Means for Solving the Problems]
In order to achieve the above object, an image encoding apparatus according to the present invention is an image encoding apparatus that encodes image signals sequentially input in frame units, wherein a frame rate indicating a cycle in which the encoding is performed Frame rate receiving means for receiving, a total bit number specifying means for specifying the total number of bits that can be allocated as a whole to a frame group composed of a plurality of frames, and Frame number specifying means for specifying the number of unencoded frames in the frame group based on the number of frames, based on the specified total number of bits and the specified number of frames, Target bit number calculating means for calculating a target value of the number of bits to be allocated to the frame to be encoded, and Using a target value of the number of bits, a quantization step deriving unit that derives a quantization step for a frame to be encoded next, performs quantization based on the derived quantization step, and performs the quantization. Encoding means for performing encoding based on the information.
[0033]
With this, the number of bits to be allocated to the next frame to be encoded is calculated based on the total number of bits allocated to the entire frame group and the number of bits already allocated so that the frame group has a constant bit rate. Therefore, it is possible to avoid the case where the number of allocated bits is locally biased beforehand, and to perform averaged bit allocation, and ultimately to prevent extreme deterioration in image quality.
[0034]
In order to achieve the above object, the image encoding apparatus according to the present invention further includes an average value calculation unit that calculates an average value of the number of bits whose allocation is determined along with encoding of the frames in the frame group. Computing means for performing a predetermined operation on the calculated target value of the number of bits and the average value of the number of bits to specify a new target value of the number of bits, and deriving the quantization step. The means derives the quantization step using the target value of the number of bits specified by the calculating means.
[0035]
Accordingly, the number of bits to be allocated to the next frame to be encoded is determined based on the number of uncoded frames calculated from the received frame rate and the number of bits that can be allocated in the frame group. Coding according to the change in rate becomes possible.
[0036]
Further, in order to achieve the above object, the image encoding apparatus according to the present invention further comprises a storage unit for storing already coded but untransmitted data, and a data amount stored in the storage unit. A frame rate calculation unit for calculating a frame rate for a frame to be encoded next, based on the frame rate, and wherein the frame rate receiving unit receives the calculated frame rate.
[0037]
With this, while maintaining a constant bit rate when observed for a predetermined time length, if there is sufficient space in the buffer, encoding is performed with a low quantization step, so that encoding can be performed with higher image quality than before. It is possible to suppress a sharp decrease in the frame rate (that is, skip many frames without being encoded) when performing a transition to a complicated scene, and to perform encoding with smoother motion than before. Become.
[0038]
In order to achieve the above object, the present invention can be realized as an image coding method having the characteristic means of the image coding apparatus as steps, or as a program including these steps. it can. The program can be distributed not only in a ROM or the like included in the image encoding device but also in a recording medium such as a CD-ROM or a transmission medium such as a communication network.
[0039]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the following description, the same reference numerals are given to the same components as those of the related art, and the description thereof will be omitted.
[0040]
(Embodiment 1)
FIG. 1 is a block diagram illustrating a functional configuration of an image encoding device 10 according to Embodiment 1. The image coding apparatus 10 suppresses local bit rate fluctuations while maintaining a constant bit rate when observation is performed for a predetermined time length (for example, one second), so that a higher quality image can be obtained. Enables encoding for reproduction.
[0041]
The image encoding device 10 includes a frame memory 102, a motion detection unit 103, a motion compensation unit 104, an orthogonal transformation unit 105, a quantization unit 106, a variable length encoding unit 107, an inverse quantization unit 108, an inverse orthogonal transformation unit 109, It comprises a frame thinning unit 20, a rate control unit 30, and an overall control unit 40.
[0042]
The frame thinning unit 20 determines encoding / non-encoding of the frame for the encoded frame signal 101 input at a constant cycle, and sends the frame rate value Rf21 at that time to the rate control unit 30. Notice.
[0043]
Based on the number of generated bits received from the variable length coding unit 107 and the frame rate value Rf21 notified from the frame thinning unit 20, the rate control unit 30 performs a quantization step (or The quantization parameter is determined and transmitted to the quantization unit 106.
[0044]
The overall control unit 40 is, for example, a microcomputer including a ROM, a RAM, and the like, and controls the entire image encoding device 10. More specifically, the overall control unit 40 controls the processing timing of each unit based on a control signal (the broken line in the overall control unit 40 in FIG. 1 represents a control signal line). Further, the overall control unit 40 identifies a picture type (identification of an I frame, a B frame, and a P frame) of the encoded frame signal 101 input at a constant cycle, and sends the identification result to the rate control unit 30. Notice.
[0045]
FIG. 2 is a block diagram showing a detailed functional configuration of the rate control unit 30 in FIG. The rate control unit 30 includes a frame group target bit number derivation unit 31, a next frame target bit number derivation unit 32, an average frame bit number derivation unit 33, a calculation unit 34, a quantization step derivation unit 35, and a storage unit 36. .
[0046]
The frame group target bit number deriving unit 31 sequentially calculates and updates the target value of the total number of bits that can be allocated to all the remaining frames to be encoded in frame group units. More specifically, the frame group target bit number deriving unit 31 receives the generated bit number Nn131 from the variable length coding unit 107 and stores it in the storage unit 36. Further, the frame group target bit number deriving unit 31 reads out the total number of bits that can be allocated from the next frame to be encoded to the last frame of the frame group from the storage unit 36, and calculates the number of generated bits from the total bit number. Nn 131 is subtracted, and its value is transmitted to the next frame target bit number deriving unit 32 and stored in the storage unit 36.
[0047]
The next frame target bit number deriving unit 32 calculates the remaining frames to be encoded based on the frame rate value Rf21 received from the frame thinning unit 20 and the allocatable total bit number received from the frame group target bit number deriving unit 31. , The target value of the number of bits to be allocated to the next frame is calculated.
[0048]
The average frame bit number deriving unit 33 calculates the average of the number of allocated bits based on the frame rate value Rf21 received from the frame thinning unit 20 and the number of bits allocated to past frames stored in the storage unit 36. Calculate the value.
[0049]
The calculation unit 34 calculates the number of bits obtained by multiplying the number of bits calculated by the average frame bit number derivation unit 33 by a predetermined coefficient (for example, 0.8). The calculated number of bits is compared with a target value, and the larger number of bits is selected. Further, the arithmetic unit 34 compares the selected number of bits with a preset lower limit, and selects a larger number of bits (that is, selects a number of bits not less than the lower limit). This is transmitted to the quantization step deriving unit 35. The above-mentioned predetermined coefficient is not limited to “0.8”, but may be another value larger than 0 and equal to or smaller than 1.
[0050]
The quantization step deriving unit 35 derives a quantization step value such that the number of bits received from the arithmetic unit 34 and the number of generated bits are equal.
[0051]
The storage unit 36 stores the number of generated bits Nn131 as a result of encoding each frame received from the variable length encoding unit 107. Further, the storage unit 36 updates the frame group target bit number under the control of the frame group target bit number deriving unit 31.
[0052]
FIG. 3 is a diagram showing general characteristics of the number of bits generated for each picture type in image coding. As shown in FIG. 3, usually, the frame (P or B picture) of the inter-picture coding (P picture or B picture) is obtained from the number of generated bits Nn in the frame of the intra-picture coding (I picture) (I frames 210 to 250). The number of generated bits Nn in the (B frame) is smaller. Therefore, when a frame group is defined based on another rule by defining a frame group from an I frame to the next I frame as one frame group (for example, a frame group is formed every 15 frames regardless of a picture type). Case), it is possible to reduce the deviation of the number of generated bits in each frame group. In the example of FIG. 3, for the sake of convenience, the time required for encoding each frame group is set to TA and the same period.
[0053]
It should be noted that the effect of reducing the deviation of the number of bits in each frame group can be obtained only by the first frame of the frame group being the above-described I frame or the last frame of the frame group being the I frame. .
[0054]
FIG. 4 is a diagram showing a specific example of the frame group defined in FIG. As shown in FIG. 4, the frame group 310 is composed of fifteen frames from the intra-coded (I) frame 210 to the frame immediately before the intra-coded (I) frame 220. Similarly, the frame group 220 and the frame group 230 are each composed of 15 frames. In this case, an initial value of the total number of bits that can be allocated to one frame group is determined in advance (for example, 3600 “bits” etc.) Note that the method of defining the frame group is not limited to FIG. .
[0055]
FIG. 5 is a modified example of the frame group defined in FIG. As shown in FIG. 5, a group of frames composed of a first frame set and a second frame set corresponding to the two frame groups 310 and 320 in FIG. 4 may be defined as one frame group 410. In this case, the initial value of the total number of bits that can be allocated to the frame group 410 is 7200 [bits] (3600 [bits] for the first frame set and the second frame set, respectively).
[0056]
FIG. 6 is a diagram for explaining an effect obtained by forming a frame group by two frame sets as shown in FIG. FIG. 6A shows the number of bits to be allocated to the next frame to be encoded, including the number of bits that can be allocated in the next second frame set even if there is a scene that is difficult to encode in one frame group. Can be determined, so that a change in bit rate can be further reduced. For example, it is assumed that 1000 bits are allocated to the entire frame group 410 as initial values, 500 bits are temporarily allocated to the first frame set, and 500 bits are temporarily allocated to the second frame set. Thereafter, even if 700 bits are used in the first frame set in actual encoding, the next frame group 420 has an initial value of 800 [bits] as a whole (the remaining 300 [bits] of the previous frame group 410). +500 [bit] for the new second frame set).
[0057]
FIG. 6B shows a case where the bit rate suddenly becomes small in the same manner as in FIG. 6A even when the frame rate has to be changed due to the difficulty in encoding. That is, it is possible to prevent the quantization step from rapidly increasing).
[0058]
FIG. 7 is a flowchart illustrating a flow of processing in the overall control unit 40 and the rate control unit 30 of the image encoding device 10.
[0059]
First, the rate control unit 30 receives the number of generated bits Nn131 from the variable length coding unit 107 (S701), and receives the frame rate value Rf21 from the frame thinning unit 20 (S702).
[0060]
Next, the overall control unit 40 identifies whether or not the next frame is an I frame (S703). In this case, if the next frame is an I frame (S704: Yes), the rate control unit 30 initializes the number Na of assignable bits, the assignment target period Ta, and the number Nt of assignment target frames (S708 to S710). .
[0061]
On the other hand, when the next frame is not an I frame (S704: No), the rate control unit 30 updates the number Na of allocable bits and the number Nt of frames to be allocated (S705, S706). Furthermore, the rate control unit 30 rounds down the decimal portion of the number Nt of allocation target frames and converts this Nt into an integer (S707).
[0062]
Thereafter, the rate control unit 30 calculates the number Nb of bits to be allocated based on the number Na of bits that can be allocated and the number Nt of frames to be allocated (S711), and updates the allocation target period Ta (S712). In this case, the allocation target period Ta is updated in consideration of the case where the value of the frame rate value Rf21 changes in the middle of the same frame group.
[0063]
Thereby, the rate control unit 30 derives a quantization step based on the number of allocated bits Nb, and transmits it to the variable length coding unit 107 (S713).
[0064]
The overall control unit 40 and the rate control unit 30 repeat the above processing until the encoding processing ends (S701 to S714).
[0065]
FIG. 8 is an example illustrating a change in bit rate when encoding is performed using the image encoding device 10. Looking at the change in the bit rate shown in FIG. 8, the number of bits that can be allocated is suppressed to be smaller than in the case of the related art (FIG. 36) due to the coefficient (for example, 0.8) in the arithmetic unit 34. Therefore, the change in the bit rate tends to slow down. Further, in FIG. 8, the arithmetic unit 34 selects the number of bits that does not fall below the lower limit, and calculates the quantization step so that the number of bits after encoding becomes the number of bits. Further, since encoding is performed in accordance with the calculated quantization step and the number of bits to be generated is determined, the bit rate is shown to have a value equal to or greater than the lower limit.
[0066]
FIG. 9 is an example showing how the bit rate changes when one frame group is composed of two frame sets as shown in FIG. As shown in FIG. 9, in the related art (FIG. 36 described above), the bit rate value is small during “t3-t4” when the bit rate value is significantly small (that is, the quantization step is extremely large). Although it was possible to suppress the image quality from deteriorating, the effect of using a large number of bits in the first frame set appeared in the first part of the second frame set (between "t4" and "t6"). (That is, the quantization step cannot be reduced and the value of the bit rate remains small).
[0067]
FIG. 10 is an example showing how the bit rate changes when one frame group is configured by two frame sets and the encoding frame rate changes in the middle of the frame group. In FIG. 10, the frame rate changes from 15 [fps] to 7.5 [fps], but the manner of changing the bit rate is the same as the bit rate of FIG. 8 (that is, in the case of FIG. , 1/2 of the quantization step, the bit rate is assumed to be the same because the number of bits is doubled. In this case as well, the time between "t3-t4" Deterioration of image quality is suppressed).
[0068]
FIG. 11 is a diagram showing a specific example of a method of calculating the number of bits in the arithmetic unit 34 of FIG. In the example of FIG. 11, encoding is performed on the encoded frame signal 101 input at 30 [fps] at a frame rate of 15 [fps]. In this example, similarly to FIG. 37, the input encoded frame signal 101 is represented by “input image frame number 1701”, and the encoded frame signal 121 is represented by “frame group frame number 1102”. In FIG. 11, the columns of the frame group frame number 1102 and the number of generated bits Nn131 are indicated by "x" because the frame of the corresponding input image frame number 1701 is not coded and is "decimated". Represents
[0069]
Further, in FIG. 11, for example, the target value of the number of generated bits for the input image frame number “9” (that is, the frame group frame number “5”) is that the total number of bits already allocated is 1180 [bits] and the remaining Is (11), and thus ((3600-1180) / 11 = 220 [bits]). The average frame bit number deriving unit 33 calculates the average value of the number of generated bits in the past frame (1180/4 = 295 [bits]), and the arithmetic unit 34 adds a coefficient (for example, 0.8), and compares the result with the target value of the number of bits to select the larger value. As a result, the number of generated bits for the frame group frame number “5” is 236 [bits].
[0070]
On the other hand, the number of bits generated for the input image frame number “13” (ie, the frame group frame number “6”) is “decimated” for the frame of the input image frame number “11” (therefore, the frame rate value Rf21 is changed to 7.5 [fps]) and the number of remaining frames is calculated based on “5” (since the frame rate value Rf21 is calculated at 7.5 [fps]). Furthermore, since the calculated value is rounded down to the decimal point, (3600-1416) / 5 = 436 [bits]. The average frame bit number deriving unit 33 calculates the average value of the number of generated bits in the past frame (1416/5 = 383 [bits]), and the arithmetic unit 34 adds a coefficient (for example, 0.8), and compares this with the target value of the number of bits to select the larger value. As a result, the number of generated bits for the input image frame number “13” is 436 [bits].
[0071]
Comparing the number of generated bits Nn of the frame group frame numbers “5” and “6” in FIG. 11 and FIG. 35 of the prior art described above, all of the generated bits in the image encoding apparatus 10 (ie, FIG. Since the number of bits is a larger value and a larger number of bits can be assigned, it is clear that encoding for a higher quality image than before can be performed.
[0072]
Although two numbers are described in the frame group frame number column corresponding to the input image frame number “29” in FIG. 11, “(15)” indicates the time when the input image frame number is “9”. Are the expected values of the frame group frame number 1102 when the frame rate is 15 [fps], and “(10)” indicates that the frame rate at the time when the input image frame number is “13” is 7.5 [fps]. ], The predicted value of the frame group frame number 1102.
[0073]
In the first embodiment, the frame thinning section 20 is provided, and the bit rate is controlled based on the frame rate value Rf21 transmitted from the frame thinning section 20. The value may be obtained from a unit other than the frame thinning unit.
[0074]
FIG. 12 is a block diagram showing a part of the functional configuration of an image encoding device 50 in which the frame thinning unit 20 in FIG. 1 is not provided and the value of the frame rate value Rf21 is obtained from another unit. It is. The image encoding device 50 is different from the image encoding device 10 of FIG. 1 only in that the frame thinning unit 20 is not provided and that an overall control unit 60 is provided instead of the overall control unit 40.
[0075]
Further, since the quantization step in the above embodiment uniquely corresponds to the quantization parameter, the quantization step may be replaced with the quantization parameter.
[0076]
As described above, according to the image coding apparatus according to the present embodiment, the number of bits to be allocated is determined while corresponding to a change in the frame rate of coding. In addition, it is possible to secure a good image quality on average, and to avoid extremely awkward movements and poor image quality.
[0077]
Further, the bit stream encoded by the image encoding device described in the first embodiment can be decoded by a general image decoding method. Also, in the stream that has been encoded according to the first embodiment, the quantization step of each frame varies with the complexity of the image, but the variation is not extreme. It is possible to make the number of generated bits almost constant in a frame group composed of frames immediately before the encoded frame.
[0078]
In the first embodiment, the cycle required for encoding each frame group is assumed to be the same as “TA”. However, the present invention is not limited to the case where the cycle is the same, and the time required for encoding each frame group differs. May be.
[0079]
Further, in the above embodiment, only the average value of the number of bits calculated by the average frame bit number deriving unit is multiplied by a predetermined coefficient (for example, 0.8), and this is multiplied by the next frame target bit number deriving unit. Although the configuration is such that the larger number of bits is selected by comparing with the target value of the set number of bits, it may be configured that both of them are multiplied by a predetermined coefficient and compared.
[0080]
(Embodiment 2)
FIG. 13 is a block diagram illustrating a functional configuration of an image encoding device 2100 according to Embodiment 2. The image coding apparatus 2100 controls the quantization step and the frame thinning while considering a group of frames composed of twice as many frames as the conventional one as shown in FIG. 5 of the first embodiment. By doing so, it is possible to perform high-quality encoding while suppressing sudden fluctuations in the bit rate and the frame rate.
[0081]
The image encoding device 2100 includes a frame thinning unit 2101, a frame memory 102, a motion detecting unit 103, a motion compensating unit 104, an orthogonal transform unit 105, a quantizing unit 106, a variable length encoding unit 107, a transmission buffer 2108, an inverse quantum It comprises a transform unit 2109, an inverse orthogonal transform unit 2110, a differentiator 2111, an adder 2112, and a frame rate control unit 2113.
[0082]
The frame thinning unit 2101 performs a thinning process on each frame of an image signal input at a reference frame rate (for example, 30 Hz) according to the thinning information input from the frame rate control unit 2113, and thins out. The image signal is output to the motion detector 103 and the differentiator 2111. For example, when the value of the thinning-out information input from the frame rate control unit 2113 to the frame thinning-out unit 2101 is “1”, it indicates an instruction to perform encoding without thinning-out, and when the value is “0”, Here, an instruction to perform encoding by performing one-frame thinning is shown. That is, when the thinning information is “1”, the frame thinning unit 2101 outputs the frame input as the image signal to the motion detecting unit 103 and the differentiator 2111 as a frame to be encoded as it is. Conversely, when the thinning information is “0”, a process of thinning out (eg, discarding) the input frame is performed.
[0083]
The frame memory 102 is a storage device that is realized by a RAM, a hard disk, or the like, and that holds image data in frame units. Further, the frame memory 102 stores reference frame data serving as a reference image of the next inter-picture prediction coding frame. When the input encoded frame is a frame on which inter-frame prediction encoding is to be performed, the motion detecting unit 103 detects the motion of the encoded frame using the reference frame data stored in the frame memory 102. The motion compensator 104 generates motion compensation data corresponding to the motion detected by the motion detector 103. The motion compensation data is data indicating a motion vector indicating a correspondence between blocks of a reference frame and a frame to be coded, and a difference of an image signal between blocks indicated by the motion vector.
[0084]
The difference unit 2111 obtains a difference between a motion vector of the encoded frame and a motion vector based on the motion compensation data generated by the motion compensation unit 104 when the encoded frame is a frame on which inter-picture prediction encoding is performed. . On the other hand, if the frame to be encoded is a frame for which intra prediction coding is to be performed, the differentiator 2111 performs prediction within the screen without performing processing for obtaining a difference between the input image signal and the reference frame. After that, an image signal is output to the orthogonal transformation unit 105.
[0085]
The orthogonal transform unit 105 orthogonally transforms the motion compensation data generated by the motion compensating unit 104 into data representing frequency components by DCT (Discrete Cosine Transform) transform or the like, and outputs the transform result to the quantization unit 106. The quantization unit 106 quantizes the orthogonally transformed data using the quantization step input from the frame rate control unit 2113, and sends the quantized result to the variable length coding unit 107 and the inverse quantization unit 2109. Output. The variable-length coding unit 107 performs variable-length coding on the data quantized by the quantization unit 106 using a Huffman code or the like. The data that has been variable-length coded by the variable-length coding unit 107 is stored in the transmission buffer 2108 and output as a coded bit stream.
[0086]
The transmission buffer 2108 is a FIFO (First-InFirst-Out) memory realized by a RAM or the like. This transmission buffer 2108 stores, for example, as shown in FIG. 5 of the first embodiment, the total number of bits allocated to a frame group including a plurality of frames, and also stores the remaining data amount or free space inside the frame group. The capacity is monitored, and when the variable-length coded data is transmitted to the transmission buffer 2108, that is, when the encoding of the image signal for one frame is completed, the remaining amount of data in the transmission buffer 2108 is changed to a frame. The data is transmitted to the rate control unit 2113.
[0087]
The inverse quantization unit 2109 inversely quantizes the data quantized by the quantization unit 106 to generate a predicted image, and outputs the data to the inverse orthogonal transform unit 2110. The inverse orthogonal transform unit 2110 performs an inverse orthogonal transform on the data inversely quantized by the inverse quantization unit 2109. The adder 2112 adds the data dequantized by the inverse orthogonal transform unit 2110 and the motion compensation data generated by the motion compensation unit 104, and outputs the result to the frame memory 102. The frame rate control unit 2113 receives the remaining amount of data in the transmission buffer 2108 from the transmission buffer 2108. The frame rate control unit 2113 calculates a quantization step according to the remaining amount of data in the transmission buffer 2108 and outputs it to the quantization unit 106, and calculates a frame rate according to the calculated quantization step. The thinning-out information indicating whether to thin out the frame is determined and output to the frame thinning unit 2101. Here, the “quantization step” is a value (width) that is a reference when quantizing the frequency component of each frame, and is, for example, a natural number from “1” to “31” (this is a “quantization parameter”). "). The quantization parameters correspond to the quantization steps on a one-to-one basis, and are defined such that the larger the numerical value, the larger the quantization width. Hereinafter, the quantization parameter will be described as a “quantization step”.
[0088]
FIG. 14 is a block diagram showing a detailed functional configuration of the frame rate control unit 2113 in FIG. The frame rate control unit 2113 includes a quantization step prediction unit 2201, a frame rate calculation unit 2202, a quantization step determination unit 2203, a frame number calculation unit 2204, and a comparator 2205. The quantization step prediction unit 2201 performs a predetermined operation on the current quantization step according to the remaining amount of data input from the transmission buffer 2108, calculates a quantization step prediction value, and calculates the frame rate calculation unit 2202 and Output to the quantization step determination unit 2203. The quantization step determination unit 203 selects one natural number from “1” to “31” that is the closest to the quantization step prediction value input from the quantization step prediction unit 201, and converts the selected natural number to the next quantization. Decide on a step. Specifically, when the input quantization step prediction value includes a value below the decimal point, rounding processing such as rounding, rounding down, or rounding up is performed. In this case, if the value is less than "1", "1" is selected, and if it exceeds "31", "31" is selected.
[0089]
The frame rate calculation unit 2202 performs the next encoding based on the current frame rate based on the remaining amount of data input from the transmission buffer 2108 and the quantization step prediction value input from the quantization step prediction unit 201. Calculate the optimal frame rate at. The frame number calculation unit 204 calculates the current frame rate based on the thinning information output from the comparator 205. The comparator 205 compares the optimum frame rate output from the frame rate calculation unit 202 with the current frame rate output from the number-of-frames calculation unit 204. If the optimum frame rate is higher than the current frame rate, Then, "1" indicating an instruction not to thin out is output as thinning information. On the other hand, if the optimum frame rate is smaller than the current frame rate, “0” indicating an instruction to perform thinning is output as thinning information.
[0090]
FIG. 15 is a block diagram for explaining the function of the quantization step prediction unit 201 in FIG. FIG. 15A is a diagram schematically illustrating a quantization step prediction method in the quantization step prediction unit 201. FIG. 15B is a diagram illustrating the prediction coefficient table 303 stored in the quantization step prediction unit 201. The quantization step prediction unit 201 holds a current value memory 2302 shown in FIG. 15A and a prediction coefficient table 2303 shown in FIG. The current value memory 302 holds the initial value of the quantization step at the start of encoding, and holds the quantization step predicted value calculated by the quantization step prediction unit 201 after the start of encoding. The prediction coefficient table 303 stores the data remaining amount of the transmission buffer 2108 and the prediction coefficient in association with each other, and the quantization step prediction unit 201 calculates the prediction coefficient corresponding to the data remaining amount input from the transmission buffer 2108. Is multiplied by the current quantization step prediction value held in the current value memory 302 to calculate the quantization step prediction value of the next frame.
[0091]
As shown in FIGS. 15A and 15B, the prediction coefficient is “1.2” when the remaining amount of data in the transmission buffer 2108 is 50% or more and less than 80%, and is 80% or more and less than 90%. As the free space in the transmission buffer 2108 decreases, the value increases and the bit amount of the data to be encoded decreases. It is set as follows. Further, when the remaining data amount of the transmission buffer 2108 is 40% or more and less than 50%, the prediction coefficient is set to “1” so as to maintain the quantization step as it is. Conversely, the prediction coefficient is “0.8” when the remaining data amount of the transmission buffer 2108 is 30% or more and less than 40%, and “0.5” when the remaining data amount is 20% or more and less than 30%. In the case of 0% or more and less than 20%, it is set as "0.4". In other words, the prediction coefficient is set such that the larger the free space in the transmission buffer 2108, the larger the bit amount of the data to be coded (that is, the quantization step is made smaller and the quantization is finely performed, and the quality of the image quality is improved). To increase).
[0092]
For example, assuming that the predicted value of the quantization step in the current value memory 302 is “7.5” and the remaining amount of data in the transmission buffer 2108 is 75%, the quantization in the current value memory 302 is performed in the next frame. The step prediction value “7.5” is multiplied by the prediction coefficient “1.2”, and the quantization step prediction value for the frame is “9”. The calculated quantization step prediction value “9” is overwritten in the current value memory 2302 by the quantization step prediction unit 2201. Since the quantization step prediction value “9” is a natural number from “1” to “31”, the quantization step determination unit 2203 determines the quantization step as the quantization step for the frame. After the frame is quantized in the quantization step “9”, if the remaining amount of data in the transmission buffer 2108 is 70% even in the next frame, the quantization step prediction value in the current value memory 2302 is “ 9 is multiplied by the prediction coefficient “1.2”, and the next quantization step prediction value is “10.8”. If the quantization step determination unit 203 determines the quantization step by rounding off decimals or less, the next frame is quantized in the quantization step “11”. By setting the prediction coefficient in this manner, when the remaining data amount in the transmission buffer 2108 becomes 50% or more, the quantization step increases, so that the image quality of the image to be encoded gradually becomes coarse and is encoded. The bit amount of data can be gradually reduced. On the other hand, when the remaining amount of data in the transmission buffer 2108 is less than 40%, the quantization step is reduced, so that the data bit amount gradually increases, but the image quality of the image to be encoded can be gradually improved. .
[0093]
FIG. 16 is a block diagram for explaining the function of the frame rate calculation unit 2202 in FIG. The frame rate calculation unit 202 is a unit that calculates an optimum frame rate based on the remaining amount of data in the transmission buffer 2108 and the predicted quantization step value, and includes a comparator 2401, a comparator 2402, a frame rate table 2403, and a threshold memory. 2404, a threshold memory 2405 and a free space calculation unit 2406. The threshold memory 2404 and the threshold memory 2405 are realized by a latch circuit, a RAM, or the like. The comparator 2401 receives the quantization step prediction value output from the quantization step prediction unit 2201 and the threshold value B stored in the threshold value memory 2404. The comparator 2401 compares these two inputs. For example, if the quantization step prediction value exceeds the threshold B, the comparator 2401 outputs “1” as the output D1. If the quantization step prediction value is equal to or smaller than the threshold B, the comparator 2401 outputs “0”. Is output. The free capacity of the transmission buffer 2108 calculated by the free capacity calculation unit 2406 and the threshold C stored in the threshold memory 2405 are input to the comparator 2402. The comparator 2402 compares these two inputs, and outputs “1” as the output D2 when the free space of the transmission buffer 2108 exceeds the threshold C, and outputs “0” as the output D2 when the free space is equal to or less than the threshold C. I do. The frame rate table 2403 is a table indicating an optimal frame rate corresponding to a combination of the output D1 of the comparator 2401 and the output D2 of the comparator 2402. The threshold value memory 2404 holds a threshold value B of a quantization step prediction value set in advance. The threshold memory 2405 holds a preset threshold C of the transmission buffer 2108. The free space calculation unit 406 calculates the current free space of the transmission buffer 2108 based on the total data capacity of the transmission buffer 2108 and the remaining amount of data input from the transmission buffer 2108.
[0094]
Specifically, the frame rate calculation unit 2202 calculates the optimum frame rate for the encoding target frame according to the frame rate table 2403. For example, when the free space of the transmission buffer 2108 exceeds the threshold value C and the predicted quantization step value is equal to or less than the threshold value B, that is, when there is room in the transmission buffer 2108 despite the small quantization step value. , The optimal frame rate is determined (for example, to 30 Hz) so that the frame rate is maximized. Conversely, when the free space of the transmission buffer 2108 is equal to or less than the threshold C and the predicted quantization step value exceeds the threshold B, that is, although the quantization step is large, the transmission buffer 2108 has a margin. If not, the optimal frame rate is determined to be (eg, 10 Hz) so that the frame rate is minimized. Further, when both the free space of the transmission buffer 2108 and the quantization step prediction value are larger than the threshold value or both are smaller than the threshold value, the optimum frame rate is determined so that the frame rate becomes an intermediate value.
[0095]
FIG. 17 is a diagram showing an example of each input signal and output signal of the frame rate calculation unit 2202 in FIG. FIG. 17A is a diagram illustrating an optimum frame rate which is an output signal of the frame rate calculation unit 2202. FIG. 17B is a diagram illustrating a quantization step prediction value that is one input signal of the frame rate calculation unit 202. FIG. 17C is a diagram illustrating the free space of the transmission buffer 2108 which is the other input signal of the frame rate calculation unit 202. This free space is accurately calculated by the free space calculation unit 2406 from the input data remaining amount. A curve L1 in FIG. 17B shows how the predicted value of the quantization step changes over time, and a curve L2 in FIG. 17C shows a time change in the free space of the transmission buffer 2108. I have. As shown in FIG. 17B, the comparator 2401 outputs D1 = 0 while the quantization step prediction value is equal to or smaller than the threshold B, and when the quantization step prediction value exceeds the threshold B, the comparator 2401 outputs D1. = 1 is output. When the predicted value of the quantization step becomes the threshold B or less again, the comparator 401 outputs D1 = 0. On the other hand, as shown in FIG. 17C, while the free space of the transmission buffer 2108 is equal to or smaller than the threshold C, the comparator 2402 outputs D2 = 0, and when the free space exceeds the threshold C, the comparator 2402 outputs D2 = Outputs 1. When the free space falls below the threshold value C again, D2 = 0 is output.
[0096]
The frame rate table 2403 held by the frame rate calculation unit 2202 stores four values of the value “0” or “1” of the output D1 of the comparator 2401 and the value “0” or “1” of the output D2 of the comparator 2402. Four types of frame rates are described corresponding to the combinations. While D1 = 0 (the quantization step predicted value is small) and D2 = 0 (the free space is small), the frame rate calculation unit 2202 outputs 20 Hz as the optimal frame rate as shown in FIG. . Also, as shown in FIGS. 17B and 17C, when only the quantization step prediction value indicated by the curve L1 exceeds the threshold value B, that is, D1 = 1 (quantization step prediction While the value is large and D2 = 0 (the free space is small), the frame rate calculation unit 2202 outputs 15 Hz as the optimum frame rate. Subsequently, when both the quantization step prediction value and the free space exceed the threshold, that is, while D1 = 1 (the quantization step prediction value is large) and D2 = 1 (the free space is large), the frame rate calculation is performed. The unit 2202 outputs 20 Hz as the optimum frame rate in order to control the quantization step to be small. Further, when only the quantization step prediction value is equal to or smaller than the threshold B, that is, while D1 = 0 (the quantization step prediction value is small) and D2 = 1 (the free space is large), the frame rate calculation unit 202 outputs 30 Hz as the optimum frame rate. As described above, the frame rate calculation unit 202 gradually increases or decreases the frame rate according to the quantization step prediction value and the free space of the transmission buffer 2108 (although there are four levels here).
[0097]
FIG. 18 is a diagram for explaining the operation of the frame number calculation unit 204 shown in FIG. FIG. 18A is a diagram illustrating an example of the thinning information output from the comparator 2205. FIG. 18B is a block diagram illustrating a detailed configuration of the frame number calculation unit 2204. As shown in FIG. 18A, the thinning information output from the comparator 2205 is input to the frame number calculation unit 2204 at 30 Hz. The thinning information is a binary signal of “0” or “1”. The thinning information “0” indicates that the frame has been thinned, and the thinning information “1” indicates that the frame has been coded. 18B is a processing unit that calculates the current frame rate based on the thinning information output from the comparator 2205 and outputs the calculation result at a predetermined cycle (for example, 5 Hz). Or a circuit. The frame number calculation unit 2204 includes a thinning information memory 2601, a counter 2602, and a calculator 2603. The thinning information memory 2601 is a FIFO (first_in first_out) memory device that holds first-in first-out, and holds 30 bits of 1-bit thinning information input at 30 Hz, retroactively from the current frame. The counter 2602 counts, at a predetermined cycle (for example, 5 Hz), the number m of frames in which the latest thinning information in the thinning information memory 2601 is “1” and the number n of frames in which the thinning information is “0”. The values of m and n are reset at a period (for example, after 1/5 second). Arithmetic unit 2603 calculates “m / (m + n)” at the above-described predetermined cycle, and outputs the result as the current frame rate. Thus, when the frame number calculation unit 2204 calculates and outputs the current frame rate, for example, five times per second, it is possible to calculate a more appropriate quantization step every 0.2 seconds.
[0098]
The comparator 2205 of the frame rate control unit 2113 compares the current frame rate calculated by the frame number calculation unit 2204 with the optimum frame rate calculated by the frame rate calculation unit 2202, and determines that the optimum frame rate is the current frame. If the rate is equal to or more than the rate, "1" indicating an instruction for encoding is output as thinning information. On the other hand, if the optimum frame rate is less than the current frame rate, “0” indicating a thinning instruction is output as thinning information. As a result, the frame rate of the frame to be encoded becomes equal to the optimum frame rate calculated by the frame rate calculation unit 2202. That is, since the present image encoding device 2100 controls the quantization step according to the free space of the transmission buffer 2108, when the quantization step is controlled by alleviating the time change of the free space of the transmission buffer 2108, That is, when encoding is performed by gradually increasing or decreasing the frame rate in accordance with the predicted quantization step value and the available capacity of the transmission buffer 2108, it is possible to avoid rapid image degradation.
[0099]
FIG. 19 is a diagram for describing an outline of an input image signal and a frame to be encoded in the image encoding device 2100. FIG. 19A is a diagram illustrating each frame of an image signal input to the image encoding device 2100. As shown in FIG. 19A, the image coding apparatus 2100 configures one frame group 2604 with two frame sets as in the case of FIG. One unit is used for determining the quantization step and the frame rate. FIG. 19B is a diagram illustrating each encoded frame encoded by the image encoding device 2100.
[0100]
As shown in FIG. 19A, the image signal input to the image encoding device 2100 includes a scene that is easy to encode and a scene that is difficult to encode. The amount of generated bits of the encoded data is reduced. On the other hand, in a scene where encoding is difficult, the amount of generated bits of encoded data increases. For example, if a scene that is difficult to encode is input to the image encoding device 2100 at the time a, the amount of bits generated by encoding the first frame of the scene that is difficult to encode is large, so that the free space of the transmission buffer 2108 is large. Is reduced, and the quantization step prediction unit 2201 increases the prediction value of the quantization step according to the free space. For example, the quantization step prediction value predicted by the quantization step prediction unit 2201 is increased to 1.5 times the quantization step prediction value corresponding to the immediately preceding frame. Thereby, the amount of bits generated by encoding the next frame is suppressed. As a result, the amount of bits generated by encoding does not become as large as when the quantization is performed using the initial quantization step, and the free space of the transmission buffer 2108 does not suddenly decrease. Therefore, in the related art, even when it is necessary to continuously greatly reduce frames, the image encoding device 2100 can more gradually reduce the frame rate.
[0101]
Nevertheless, if the free space of the transmission buffer 2108 decreases below the threshold value C, the frame rate calculation unit 202 changes the optimal frame rate to a frame rate one step lower. For example, if encoding was performed at a frame rate of 30 Hz until time a, the frame rate calculation unit 2202 changes the optimal frame rate from the next frame to 20 Hz. As described above, even if the quantization step is increased and the frame rate is reduced, the capacity of the transmission buffer 2108 has no margin, and the quantization step prediction value is further increased to exceed the threshold B. The unit 2202 changes the next optimum frame rate to a frame rate one step lower. For example, if encoding was performed at a frame rate of 20 Hz until time b, the frame rate calculation unit 2202 changes the next optimal frame rate to 15 Hz.
[0102]
As described above, even when a scene that is difficult to encode is input, the image encoding device 100 calculates the quantization step prediction value according to the amount of bits generated by encoding, and calculates the quantization step prediction value. Since the frame rate can be reflected on the optimum frame rate, the frame rate can be reduced more gradually and gradually. Conversely, in the input image signal, for example, at time c, even when the scene is switched from a scene that is difficult to encode to a scene that is easy to encode, first, the quantization step is reduced to obtain the highest possible image quality. If the encoding is performed and the transmission buffer still has room (the scene is easy to encode), the frame rate is changed to a larger value by one step, so the frame rate is increased more gently and gradually. can do. For example, when a scene that is easy to encode is input at time c and the transmission buffer 2108 still has a margin even if the quantization step is reduced, the frame rate is changed to one step higher at 20 Hz at time d. If the quantization step has a sufficiently small value at time e, the frame rate is changed to another higher level of 30 Hz, so that the frame rate can be increased more gently. As a result, when an image signal encoded by the image encoding device 100 is decoded by the decoding device, even in a scene that is difficult to encode, the image does not suddenly become rough or suddenly awkward. A moving image can be reproduced. Conversely, even when the scene is switched to an easy-to-encode scene, the image does not suddenly have a smoother shading, or the motion does not suddenly smoother, and the human eyes do not feel unnatural. High quality images can be reproduced.
[0103]
As described above, according to the image encoding apparatus 2100 of the present embodiment, if the buffer has enough free space while keeping the bit rate when observed for a predetermined time length (for example, one second) constant. By encoding with a low quantization step, encoding can be performed with higher image quality than before, and a sharp decrease in the frame rate when a transition to a complex scene occurs (many frames are skipped without being encoded). ) Can be suppressed, and coding with smoother motion than before can be performed.
[0104]
In the above embodiment, the frame rate calculation unit 2202 calculates the optimum frame rate using the remaining amount of data in the transmission buffer 2108 and the quantization step prediction value of the quantization step prediction unit 2201. However, the present invention is not limited to this, and the calculation may be performed using either one. Also, the frame rate calculation unit 2202 calculates the four-stage optimal frame rates of 30 Hz, 20 Hz, 15 Hz, and 10 Hz using the frame rate table 2403 shown in FIG. It is not limited to a value, and there is no need to have four levels. Further, there is no need to calculate the optimum frame rate using such a frame rate table. For example, if the available capacity of the transmission buffer is equal to or larger than the threshold C, the optimum frame rate is set to twice the current frame rate, and the threshold C If it is less than the current frame rate, it may be calculated to be １ ／ times the current frame rate. Further, another threshold value D (C <D) is provided. If the available capacity of the transmission buffer is equal to or larger than the threshold value C, the optimum frame rate is set to twice the current frame rate. It may be calculated to be 1/2 times. By doing so, it is possible to prevent the optimum frame rate from being changed each time a slight change in the remaining amount of data in the transmission buffer near each of the threshold C and the threshold D. Further, the optimum frame rate may be obtained by performing the same calculation for a change in the quantization step prediction value. Further, the optimal frame rate is not limited to a value such as 1/2 times or 2 times the current frame rate. The same applies to the following embodiments.
[0105]
Further, in the above embodiment, the case where the quantization step prediction unit 2201 calculates the quantization step prediction value using the prediction coefficient table 2303 shown in FIG. 15 has been described. The method is not limited to this, and may be calculated using a predetermined function. Further, the prediction coefficients of the quantization steps in the prediction coefficient table 2303 are also set to values of “2”, “1.5”, “1.2”, “1”, “0.8”, “0.5”, and “0.4”. Not limited. The same applies to the following embodiments.
[0106]
Although the frame number calculation unit 2204 according to the second embodiment includes the thinning information memory 2601 for storing the thinning information for the past one second (that is, for 30 frames), the present invention is not limited to this. Not done. Also, the calculation method of the current frame rate is not limited to the above calculation method, and another method may be used. The same applies to the following embodiments.
[0107]
Also, an average bit rate may be used instead of the remaining data amount in the transmission buffer. The same applies to the following embodiments.
[0108]
Hereinafter, frame rate control units 2800, 3000, and 3400, which are modifications of the above frame rate control unit 2113, will be described. In the following modified examples, the description of the same configuration as the frame rate control unit 2113 will be omitted, and different points will be mainly described.
[0109]
FIG. 20 is a block diagram illustrating a functional configuration of a frame rate control unit 2800 according to a modification. The frame rate control unit 2800 includes a quantization step prediction unit 2201, a quantization step determination unit 2203, a frame rate calculation unit 2202, a frame number calculation unit 2204, and a comparator 2205. The difference between the frame rate control unit 2800 and the frame rate control unit 2113 shown in FIG. 14 is that instead of the frame rate calculation unit 2202 receiving the quantization step prediction value of the quantization step prediction unit 2201, The point is that the quantization step determined by the determination unit 2203 is input.
[0110]
FIG. 21 is a diagram showing an example of each input signal and output signal of the frame rate calculation unit 2202 in FIG. FIG. 21A is a diagram illustrating an optimum frame rate which is an output signal of the frame rate calculation unit 2202. FIG. 21B is a diagram illustrating a quantization step which is one input signal of the frame rate calculation unit 2202. FIG. 21C is a diagram illustrating the free space of the transmission buffer 2108 which is the other input signal of the frame rate calculation unit 2202. The quantization step determination unit 2203 determines an integer value from “1” to “31” based on the quantization step prediction value obtained by the quantization step prediction unit 2201, and uses the determined value as a quantization step. Output. Therefore, as shown in FIG. 21B, the quantization step input to the frame rate control unit 2800 is "31" at the maximum, unlike the quantization step prediction value.
[0111]
As described above, the frame rate calculation unit 2202 calculates the optimum frame rate based on the quantization step output from the quantization step determination unit 2203 instead of the quantization step prediction value, thereby realizing the actual coding. It is possible to calculate the optimum frame rate without greatly deviating from the amount of generated bits due to.
[0112]
FIG. 22 is a block diagram illustrating a functional configuration of the frame rate control unit 3000 according to the modification. The frame rate control unit 3000 obtains an average quantization step of a previously coded frame from the quantization step obtained by the quantization step determination unit 2203, and determines the optimal quantization step of the frame rate calculation unit 2202 from the obtained average quantization step. It calculates a frame rate, and includes a quantization step prediction unit 2201, a quantization step determination unit 2203, an average quantization step calculation unit 3001, a frame rate calculation unit 3002, a frame number calculation unit 2204, and a comparator 2205. The average quantization step calculator 3001 calculates an average value of quantization steps used for encoding for the past one second. The frame rate calculation unit 3002 calculates an optimum frame rate from the average value of the quantization steps calculated by the average quantization step calculation unit 3001 and the free space of the transmission buffer 2108.
[0113]
FIG. 23 is a block diagram for explaining the function of the average quantization step calculator 3001 in FIG. The average quantization step calculation unit 3001 calculates an average value of quantization steps for frames encoded in the past, and includes a quantization step memory 3101, a thinning information memory 3102, an adder 3103, and a divider 3104. The quantization step memory 3101 is a FIFO that holds the value of the quantization step output from the quantization step determination unit 2203 for 30 frames from the current frame for 1 frame on a first-in first-out basis. The thinning information memory 3102 stores the thinning information (ie, for example, if there are 30 frames per second, a total of 30 bits) output from the comparator 2205 in a first-in first-out manner for one second from the current frame. This is the FIFO to be held. The adder 3103 adds the result obtained by multiplying the corresponding thinning information for each frame by the quantization step (that is, sums the quantization steps of all the frames encoded in the past one second) and divides the result. Output to the unit 3104. The divider 3104 divides the sum of the quantization steps input from the adder 3103 by the number of frames encoded in the past one second to calculate an average value of the quantization steps.
[0114]
FIG. 24 is a block diagram for explaining the function of frame rate calculation section 3002 in FIG. The frame rate calculation unit 3002 holds the frame rate table 2403 shown in FIG. 16 and further includes a hysteresis comparator 3201, a threshold memory 3202, a hysteresis comparator 3203, a threshold memory 3204, and a free space calculation unit 2406. When the thresholds α1 and β1 (β1 <α1) are input to the one input terminal from the threshold memory 3202 and the average quantization step input to the other input terminal exceeds the threshold α1, the hysteresis comparator 3201 determines that the average quantization The comparator outputs D1 = 1 until the quantization step becomes equal to or smaller than the threshold value β1, and outputs D1 = 0 until the average quantization step exceeds the threshold value α1 even if it exceeds the threshold value β1. The threshold memory 3202 is a latch circuit or a memory that holds the thresholds α1 and β1. When the thresholds α2 and β2 (β2 <α2) are input to one input terminal from the threshold memory 3204 and the free space of the transmission buffer 2108 input to the other input terminal exceeds α2, the hysteresis comparator 3203 sets D2 = 1. Then, D2 = 1 is output until the free space of the transmission buffer 2108 becomes equal to or less than β2. The threshold memory 3204 is a latch circuit or a memory that holds the thresholds α2 and β2.
[0115]
FIG. 25 is a diagram showing an example of each input signal and output signal of the frame rate calculation unit 3002 in FIG. FIG. 25A is a diagram illustrating an optimal frame rate which is an output signal of the frame rate calculation unit 3002. FIG. 25B is a diagram illustrating an average quantization step that is one input signal of the frame rate calculation unit 3002. FIG. 25C is a diagram illustrating the free space of the transmission buffer 2108 which is the other input signal of the frame rate calculation unit 3002. As shown in FIG. 25B, the average quantization step takes a value from “1” to “31”, but is not necessarily a natural number as in the quantization step of each frame, and the degree of change is gradual. It is getting smaller. The method by which the frame rate calculation unit 3002 calculates the optimum frame rate based on the values of the output D1 of the hysteresis comparator 3201 and the output D2 of the hysteresis comparator 3203 input as described above is as described above.
[0116]
As described above, the frame rate control unit 3000 controls the frame rate based on the average quantization step output from the average quantization step calculation unit 3001 instead of the quantization step prediction value, thereby enabling the quantization step Can be prevented from being changed for each frame when the value rises or falls near the threshold.
[0117]
Note that, in the above-described embodiment, an example has been described in which the average quantization step calculation unit 3001 calculates the average value of the quantization steps in frames encoded in the past second, but the present invention is not limited to this. Instead, the average value of the quantization step may be calculated for the past several seconds or for the past several frames to several tens of frames. In addition, the average value of past quantization steps may be calculated uniformly, including frames that have not been encoded.
[0118]
FIG. 26 is a block diagram illustrating a functional configuration of a frame rate control unit 3400 according to a modification. The frame rate calculation section 3401 of the frame rate control section 3400 includes a hysteresis comparator 3201 and a hysteresis comparator 3203 in place of the comparator 2401 and the comparator 2402, and, like the frame rate calculation section 202 in the first embodiment, the optimum frame. The rate is calculated. Further, the average value of the optimum frame rates calculated in the past one second is obtained, and the obtained average frame rate is set as the optimum frame rate.
The frame rate control section 3400 includes a quantization step prediction section 2201, a quantization step determination section 2203, a frame rate calculation section 3401, a frame rate memory 3402, a frame number calculation section 2204, and a comparator 2205. The frame rate calculation unit 3401 receives the output D1 of the hysteresis comparator 3201 and the output D2 of the hysteresis comparator 3203, and calculates an optimal frame rate according to the rules (truth table) of the frame rate table 2403. Further, the frame rate calculation unit 3401 calculates an average value of the plurality of past optimum frame rate calculation results.
[0119]
The frame rate memory 3402 is a FIFO that holds an optimum frame rate, which is a calculation result of the frame rate calculation unit 3401, on a first-in, first-out basis for 30 frames that are one second before the current frame.
[0120]
FIG. 27 is a block diagram showing a detailed configuration of the frame rate calculation unit 3401 and the frame rate memory 3402 in FIG. The frame rate calculator 3401 holds a frame rate table 2403, and further includes a hysteresis comparator 3201, a threshold memory 3202, a hysteresis comparator 3203, a threshold memory 3204, a free space calculator 2406, an adder 3501 and a divider 3502. . The frame rate calculation unit 3401 receives, for each frame of the input image signal, the output D1 of the hysteresis comparator 3201 to which the quantization step prediction value is input from the quantization step prediction unit 2201 and the remaining data amount of the transmission buffer 2108. The optimum frame rate is calculated from the frame rate table 2403 based on the output D2 of the hysteresis comparator 3203 and the calculated optimum frame rate is output to the frame rate memory 3402 at 30 Hz. The adder 3501 adds each optimal frame rate in the frame rate memory 3402. The divider 3502 divides the value output from the adder 3501 by “30” and refers to the frame rate table 2403 to determine the frame rate closest to the division result from the four optimal frame rates to the average optimal frame rate. Determined as rate and output. As a result, even when the optimum frame rate changes, for example, from 30 Hz to 10 Hz, the average optimum frame rate always changes via an intermediate frame rate, such as 30 Hz, 20 Hz, 15 Hz, and 10 Hz. Therefore, it is possible to more smoothly control the frame rate in encoding.
[0121]
FIG. 28 is a diagram illustrating an example of each input signal and output signal of the frame rate calculation unit 3401 in FIG. FIG. 28A is a diagram illustrating an average optimum frame rate which is an output signal of the frame rate calculation unit 3401. FIG. 28B is a diagram illustrating a quantization step prediction value that is one input signal of the frame rate calculation unit 3401. FIG. 28C is a diagram illustrating the free space of the transmission buffer 2108 which is the other input signal of the frame rate calculation unit 3401. As shown in FIG. 28 (a), the average optimal frame rate is calculated by averaging the optimal frame rates for 30 frames going back one second from the current frame and selecting the optimal frame rate closest to that value. A large change in the frame rate is suppressed, and as a result, an encoding result with smoother motion can be obtained.
[0122]
It has been described that the present frame rate calculation unit 3401 calculates the average value of the optimum frame rates for 30 frames in the past 1 second, but the present invention is not limited to this, and the range of the optimum frame rate for calculating the average value is as follows. Any number of seconds or frames may be used in the past.
[0123]
(Embodiment 3)
FIG. 29 is a block diagram illustrating a functional configuration of an image encoding device 4100 according to Embodiment 3. The image encoding device 4100 performs more appropriate encoding by using the variable bit rate method in the image encoding device 10 of the first embodiment and the variable frame rate method in the image encoding device 2100 of the second embodiment. To achieve.
[0124]
As illustrated in FIG. 29, the image coding device 4100 has the same functional configuration as the image coding device 10 according to Embodiment 1 except for a transmission buffer 2108, a frame rate control unit 4113, and a frame thinning unit 4116. . In the following, the same components as those in the first embodiment or the second embodiment are denoted by the same reference numerals, and description thereof is omitted.
[0125]
The frame rate control unit 4113 has only the function of generating thinning information among the functions of the frame rate control unit 2113 of the image encoding device 2100 according to the second embodiment.
[0126]
The frame thinning unit 4116 receives the thinning information from the frame rate control unit 4113, determines the frame rate, and notifies the bit rate control unit 30 of the determined frame rate value at that time.
[0127]
FIG. 30 is a block diagram showing a detailed functional configuration of the frame rate control unit 4113 in FIG. 29 described above. As shown in FIG. 30, frame rate control section 4113 has a configuration in which quantization step prediction section 2201 and quantization step determination section 2203 are removed from frame rate control section 2113 (see FIG. 14 above) in the second embodiment. Has become. Therefore, the frame rate control unit 4113 generates only thinning information based on the remaining amount of data obtained from the transmission buffer 2108, and transmits this thinning information to the frame thinning unit 4116.
[0128]
As described above, the image coding apparatus 4100 according to the present embodiment controls the frame rate according to the remaining amount of data in the transmission buffer and, based on this frame rate and the total number of bits allocated to the frame group, More suitable bit allocation can be realized.
[0129]
(Embodiment 4)
In the following embodiment, the configuration of the image encoding device according to the first to third embodiments is realized as a step of an image encoding program, and functions equivalent to those of the image encoding device are implemented on a general computer system. The following describes how to do this.
[0130]
FIG. 31 is an explanatory diagram of a case where functions equivalent to those of the image encoding apparatuses according to the first to third embodiments are realized on a general computer system by using a flexible disk storing an image encoding program.
[0131]
FIG. 31B shows the appearance, cross-sectional structure and flexible disk viewed from the front of the flexible disk, and FIG. 31A shows an example of the physical format of the flexible disk which is the recording medium body.
[0132]
The flexible disk 1301 is built in a case 1302, and a plurality of tracks are formed concentrically on the surface of the disk from the outer circumference to the inner circumference, and each track is divided into 16 sectors in the angular direction. Therefore, in the flexible disk storing the image encoding program, data as the image encoding program is recorded in an area allocated on the flexible disk 1301.
[0133]
FIG. 31C shows a configuration of an apparatus for recording and reproducing the image encoding program on the flexible disk 1301. When recording the image encoding program on the flexible disk 1301, data of the image encoding program is written from the computer system 1304 via the flexible disk drive 1303. If the encoding or decoding device is to be constructed in a computer system using a program in the flexible disk 1301, the program is read from the flexible disk drive 1303 and transferred to the computer system 1304.
[0134]
In the above description, a flexible disk is used as a data recording medium. However, the above-described image encoding apparatus can be realized by using a medium capable of recording a program, such as an optical disk, an IC card, and a ROM cassette. be able to.
[0135]
【The invention's effect】
As described above, according to the image encoding device of the present invention, when encoding is performed at a constant bit rate in a predetermined time length, the deviation of the number of generated bits in a frame group that is a group of frames is reduced. Each time a coding process is performed on an individual frame in the frame group, the number of bits to be allocated to the next frame is calculated. Further, by referring to the number of bits generated in the past frame, it is possible to derive the optimum number of allocated bits that can be encoded with high image quality for the next frame. That is, even when scenes that are difficult to encode continue, it is possible to prevent the number of bits allocated to subsequent scenes from becoming extremely small, and to improve the image quality as a whole.
[0136]
Further, according to the image encoding apparatus of the present invention, even when the rate of a frame to be encoded fluctuates (variable frame rate), each time encoding is performed, an optimal frame rate using the current frame rate is used. Since it is possible to derive the number of bits, it is possible to perform encoding that can avoid extreme degradation of image quality while keeping the average bit rate at a predetermined time length constant.
[0137]
The bit stream encoded by the image encoding device according to the present invention can be decoded by a general decoding device, and high-quality decoding can be performed without performing any special processing.
[0138]
Furthermore, according to the image encoding apparatus of the present invention, as a result of encoding the current frame, first, the width of quantization is adjusted to adjust the signal resolution of each pixel according to the generated code amount. The generated frame rate of the next frame is controlled, and the coding frame rate can be adjusted according to the control of the generated bit rate by adjusting the quantization width. As a result, for example, by increasing the quantization width, it is sufficient to prevent the occurrence of a state in which frame thinning is generated despite the reduced generated code amount and the image quality is deteriorated more than necessary. Thus, there is an effect that the amount of code generated by encoding can be efficiently controlled.
[0139]
Further, the image coding apparatus according to the present invention calculates the quantization width of the next coded frame from the residual code amount of the transmission buffer, for example, when the transmission buffer has room (that is, the coding is easy). Scene), the quantization width is set to a small value. Conversely, if the transmission buffer does not have enough space, the quantization width is set to a large value, so that the bit rate over a sufficiently long time is kept constant, and a code with the highest possible image quality is obtained. There is an effect that conversion can be performed.
[0140]
Furthermore, the image coding apparatus according to the present invention sets the coding frame rate when the calculated predicted value of the quantization width becomes a large value equal to or larger than the threshold value (that is, in the case of a complicated scene in which coding is difficult). Change to a smaller value to increase the frame thinning rate. However, since the amount of generated bits is suppressed by controlling the quantization width, there is an effect that it is possible to perform high-quality encoding without causing a rapid change in the frame rate.
[0141]
According to the image encoding apparatus of the present invention, the encoding frame rate is controlled by using the average value of the quantization width which is controlled and fluctuated by the residual code amount of the transmission buffer, thereby encoding the current frame. When the code amount is extremely generated (the residual code amount of the transmission buffer is increased) and the quantization step of the next frame is controlled to be large, it is suppressed that the frame rate is immediately changed, and the immediately following image is changed. It is possible to prevent the movement from becoming awkward. At the same time, there is an effect that it is possible to suppress a sudden change in the frame rate and perform smooth motion encoding.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a functional configuration of an image encoding device according to Embodiment 1.
FIG. 2 is a block diagram illustrating a detailed functional configuration of a bit rate control unit in FIG. 1;
FIG. 3 is a diagram illustrating characteristics of the number of generated bits for each picture type in general image coding.
FIG. 4 is a specific example of a frame group defined in FIG. 3;
FIG. 5 is a modified example of the frame group defined in FIG. 3;
FIG. 6A is a diagram for explaining an effect obtained by forming a frame group by two frame sets as shown in FIG. 5; FIG. 6B is a diagram for explaining an effect when a frame group is configured by two frame sets as shown in FIG. 5 and the encoding frame rate changes.
FIG. 7 is a flowchart showing a flow of processing in an overall control unit and a bit rate control unit of the image encoding device according to Embodiment 1.
FIG. 8A is a diagram illustrating a configuration example of a frame group when encoding is performed using the image encoding device according to the first embodiment; (B) is an example illustrating a change in a bit rate when encoding is performed using the image encoding device according to Embodiment 1.
FIG. 9A is an example showing a state of each frame when one frame group is configured by two frame sets. (B) is an example showing how the bit rate changes when one frame group is configured by two frame sets.
FIG. 10A is an example showing a state of each frame when one frame group is formed by two frame sets and the encoding frame rate changes. (B) is an example showing how a frame group is composed of two frame sets, and how the bit rate changes when the encoding frame rate changes.
FIG. 11 is a diagram illustrating a specific example of a method of calculating the number of bits in an arithmetic unit.
FIG. 12 is a block diagram showing a part of the functional configuration of an image coding apparatus in which a frame thinning unit is not provided and a frame rate value Rf is obtained from another unit in FIG.
FIG. 13 is a block diagram illustrating a functional configuration of an image encoding device according to Embodiment 2.
FIG. 14 is a block diagram illustrating a detailed functional configuration of a rate control unit in FIG. 13;
FIG. 15 is a block diagram for explaining a function of a quantization step prediction unit in FIG. 14;
16 is a block diagram for explaining a function of a frame rate calculation unit in FIG.
17 is a diagram illustrating input signals and output signals of a frame rate calculation unit in FIG. 16;
18 is a diagram for explaining the operation of the frame number calculation unit in FIG.
FIG. 19 is a diagram illustrating an input image signal and a frame to be encoded of the image encoding device.
FIG. 20 is a block diagram illustrating a functional configuration of a frame rate control unit according to a modification.
21 is a diagram illustrating each input signal and output signal of the frame rate calculation unit in FIG.
FIG. 22 is a block diagram illustrating a functional configuration of a frame rate control unit according to a modification.
FIG. 23 is a block diagram illustrating a configuration of an average quantization step calculator in FIG. 22;
24 is a block diagram for explaining a function of a frame rate calculation unit in FIG.
FIG. 25 is a diagram showing each input signal and output signal of the frame rate calculation unit in FIG. 22.
FIG. 26 is a block diagram illustrating a functional configuration of a frame rate control unit according to a modification.
FIG. 27 is a block diagram for explaining functions of a frame rate calculation unit and a frame rate memory in FIG. 26;
28 is a diagram showing each input signal and output signal of the frame rate calculation unit in FIG. 26.
FIG. 29 is a block diagram illustrating a functional configuration of an image encoding device according to Embodiment 3.
30 is a block diagram illustrating a functional configuration of a frame rate control unit in the image encoding device of FIG. 29.
FIG. 31 is a diagram for describing a storage medium that can store a program for realizing the image encoding device according to the first to third embodiments using a general computer system.
FIG. 32 is a block diagram illustrating a functional configuration of a conventional image encoding device using a variable bit rate method.
FIG. 33 is a block diagram illustrating a functional configuration of a bit rate control unit in a conventional image encoding device.
FIG. 34 is a flowchart showing a flow of processing in a general control unit and a bit rate control unit of a conventional image encoding device.
FIG. 35 is a diagram illustrating a specific example of a method of calculating the number of allocated bits in a conventional bit rate control unit.
FIG. 36 (a) is a configuration example of a frame group for describing a first problem of the related art. (B) is an example of a bit rate for explaining a first problem of the related art.
FIG. 37 (a) is a configuration example of a frame group for describing a second problem of the related art. (B) is an example of a bit rate for explaining a second problem of the related art.
[Explanation of symbols]
10. Image coding device
20 Frame thinning section
30 Rate control unit
31 Frame Group Target Bit Derivation Unit
32nd order frame target bit number deriving unit
33 Average Frame Bit Number Derivation Unit
34 Arithmetic unit
35 Quantization step derivation unit
36 storage unit
40 Overall control unit
50 Image encoding device
60 Overall control unit
100 image encoding device
101 Encoded frame signal
102 frame memory
103 detector
104 Compensation unit
105 orthogonal transform unit
106 Quantization unit
107 Variable length coding unit
108 Inverse quantization unit
109 Inverse orthogonal transform unit
110 Bit rate control unit
111 frame group target bit number deriving unit
112nd frame target bit number derivation unit
113 Quantization step derivation unit
121 coded frame signal
140 Overall control unit
141 quantization step
201 Quantization step prediction unit
202 Frame rate calculator
203 Quantization step determination unit
204 Frame number calculation unit
205 comparator
302 Current value memory
303 prediction coefficient table
401 comparator
406 Capacity calculation unit
1301 Flexible disk
1302 Case
1303 Flexible disk drive
1304 Computer system
2100 Image encoding device
2101 Frame thinning unit
2108 Transmission buffer
2109 Inverse quantization unit
2110 Inverse orthogonal transform unit
2111 Differentiator
2112 Adder
2113 Frame rate control unit
2201 quantization step predictor
2202 Frame rate calculator
2203 quantization step determination unit
2204 Frame number calculator
2205 Comparator
2302 Current value memory
2303 prediction coefficient table
2401 comparator
2402 comparator
2403 Frame rate table
2404 Threshold memory
2405 Threshold memory
2406 Capacity calculation unit
2601 Information memory
2602 counter
2603 arithmetic unit
2604 Frame group
2800 Frame rate control unit
3000 Frame rate control unit
3001 Average quantization step calculator
3002 Frame rate calculator
3101 Quantization step memory
3102 Information memory
3103 Adder
3104 Divider
3201 Hysteresis comparator
3202 threshold memory
3203 Hysteresis comparator
3204 Threshold memory
3400 Frame rate control unit
3401 Frame rate calculator
3402 Frame rate memory
3501 Adder
3502 Divider
4100 Image encoding device
4113 Frame rate control unit
4116 Frame thinning unit

Claims (41)

An image encoding device that performs encoding on an image signal sequentially input in frame units,
Frame rate receiving means for receiving a frame rate indicating a cycle of performing the encoding,
Total bit number specifying means for specifying the total number of bits that can be allocated to a frame group composed of a plurality of frames as a whole,
Based on the accepted frame rate and the number of already encoded frames, a frame number identification unit that identifies the number of unencoded frames in the frame group,
Based on the specified total number of bits and the specified number of frames, a target bit number calculation unit that calculates a target value of the number of bits to be allocated to the next frame to be encoded,
Using the target value of the calculated number of bits, a quantization step deriving unit that derives a quantization step for a frame to be encoded next,
An image coding apparatus comprising: coding means for performing quantization based on the derived quantization step and performing coding based on the quantization. The image encoding device further includes:
Average value calculation means for calculating the average value of the number of bits determined allocation with the encoding of the frame in the frame group,
Calculating means for performing a predetermined operation on the calculated target value of the number of bits and the average value of the number of bits to specify a new target value of the number of bits,
2. The image encoding apparatus according to claim 1, wherein the quantization step deriving unit derives the quantization step using a target value of the number of bits specified by the arithmetic unit. The total bit number specifying means,
If the first frame of the frame group is a predetermined frame, a total bit number initialization unit that initializes the number of bits that can be allocated as a whole,
3. The image encoding apparatus according to claim 2, further comprising: a total bit number updating unit that updates the total number of bits by subtracting the number of bits assigned by the immediately preceding encoding from the total number of allocatable bits. The frame number specifying means includes:
A target period identification unit that identifies a period required for encoding a frame that has not yet been encoded in the frame group,
Each time one frame in the frame group is encoded, a period update unit that updates the period required for the encoding by reducing the period based on the received second frame rate,
The image encoding apparatus according to claim 3, further comprising: a frame number calculating unit that calculates the number of frames based on the updated period required for encoding and the accepted frame rate. The target bit number calculation means,
The image encoding apparatus according to claim 4, wherein the target value of the number of bits is calculated by dividing the total number of bits updated by the total number of bits updating unit by the specified number of frames. The target bit number calculation means further includes:
6. The image encoding apparatus according to claim 5, wherein the target value of the calculated number of bits is multiplied by a predetermined coefficient to obtain a new target value of the number of bits. The calculating means includes:
6. The image encoding apparatus according to claim 5, wherein the calculated target value of the number of bits is compared with an average value of the number of bits, and a larger one is specified as the target value of the new number of bits. apparatus. The calculating means further comprises:
The image coding apparatus according to claim 7, wherein a lower limit set in advance is compared with the target value of the specified number of bits, and a larger one is specified as the target value of the new number of bits. . The average value calculating means further includes:
The image encoding apparatus according to claim 7, wherein the calculated average value of the number of bits is multiplied by a predetermined coefficient to obtain a new average value of the number of bits. The frame group is composed of a plurality of frame sets, and when encoding is completed for one frame set, a new frame set not yet encoded is sequentially added to form a new frame group,
The frames constituting the frame set are any of I pictures, P pictures and B pictures in the MPEG standard, and the arrangement of the pictures constituting the frame set is the same,
The total bit number specifying means further includes:
Each time the assignment of the number of bits is determined along with the encoding for the one frame set in the frame group, the number of bits previously distributed to another frame set and the number of bits determined in the one frame set are determined. In consideration of the difference, the total number of bits that can be allocated as a whole to the new frame group and the number of bits that can be allocated in each frame set are specified,
The frame number specifying means further includes:
Based on the accepted frame rate and the number of already encoded frames, determine the number of unencoded frames in the other frame set,
The target bit number calculation means further includes:
2. The image according to claim 1, wherein a target value of the number of bits to be allocated to the next frame to be encoded is calculated based on the number of bits that can be allocated to the other frame set and the specified number of frames. Encoding device. 2. The image encoding apparatus according to claim 1, wherein the first frame of the group of frames is a frame to be intra-coded. The image encoding apparatus according to claim 1, wherein the last frame of the group of frames is a frame immediately before a frame to be intra-coded. 2. The image encoding apparatus according to claim 1, wherein a first frame of the frame group is a frame to be intra-coded, and a last frame is a frame immediately before a frame to be intra-coded. The image encoding device further includes:
Storage means for storing already encoded but untransmitted data;
Based on the amount of data stored in the storage means, comprises a frame rate calculation means for calculating a frame rate for the next frame to be encoded,
2. The image encoding apparatus according to claim 1, wherein the frame rate receiving unit receives the calculated frame rate. An image encoding method for encoding an image signal sequentially input in frame units,
A frame rate receiving step of receiving a frame rate indicating a cycle of performing the encoding,
A total bit number specifying step of specifying a total bit number that can be allocated as a whole to a frame group composed of a plurality of frames,
Based on the accepted frame rate and the number of already encoded frames, a frame number identifying step of identifying the number of unencoded frames in the frame group,
Based on the specified total number of bits and the specified number of frames, a target number of bits calculation step of calculating a target value of the number of bits to be allocated to the next frame to be encoded,
Using the target value of the calculated number of bits, a quantization step deriving step of deriving a quantization step for a frame to be encoded next,
An encoding step of performing quantization based on the derived quantization step, and performing encoding based on the quantization. The image encoding method further includes:
An average value calculation step of calculating an average value of the number of bits whose allocation has been determined along with encoding of a frame in the frame group,
The calculated, the target value of the number of bits and the average value of the number of bits, performing a predetermined operation to specify as a new target value of the number of bits,
16. The image encoding method according to claim 15, wherein the quantization step calculating step derives a quantization step using a target value of the number of bits calculated in the calculation step. A program for an image encoding device that performs encoding on an image signal input in frame units,
A frame rate receiving step of receiving a frame rate indicating a cycle of performing the encoding,
A total bit number specifying step of specifying a total bit number that can be allocated as a whole to a frame group composed of a plurality of frames,
Based on the accepted frame rate and the number of already encoded frames, a frame number identifying step of identifying the number of unencoded frames in the frame group,
Based on the specified total number of bits and the specified number of frames, a target bit number calculation step of calculating a target value of the number of bits to be allocated to the next frame to be encoded,
Using the target value of the calculated number of bits, a quantization step deriving step of deriving a quantization step for the next frame to be encoded,
A coding step of performing quantization based on the derived quantization step and performing coding based on the quantization. The program further comprises:
An average value calculation step of calculating an average value of the number of bits whose allocation has been determined along with encoding of a frame in the frame group,
The calculated, the target value of the number of bits and the average value of the number of bits, performing a predetermined operation to specify as a new target value of the number of bits,
18. The program according to claim 17, wherein the quantization step calculation step derives the quantization step using a target value of the number of bits calculated in the calculation step. A computer-readable recording medium in which a program for an image encoding device that performs encoding on an image signal sequentially input in frame units is recorded,
The program is
A frame rate receiving step of receiving a frame rate indicating a cycle of performing the encoding,
A total bit number specifying step of specifying a total bit number that can be allocated as a whole to a frame group composed of a plurality of frames,
Based on the accepted frame rate and the number of already encoded frames, a frame number identifying step of identifying the number of unencoded frames in the frame group,
Based on the specified total number of bits and the specified number of frames, a target bit number calculation step of calculating a target value of the number of bits to be allocated to the next frame to be encoded,
Using the target value of the calculated number of bits, a quantization step deriving step of deriving a quantization step for the next frame to be encoded,
A coding step of performing quantization based on the derived quantization step and performing coding based on the quantization. A computer-readable recording medium in which a program for an image encoding device that performs encoding on an image signal sequentially input in frame units is recorded,
The program is
A frame rate receiving step of receiving a frame rate indicating a cycle of performing the encoding,
A total bit number specifying step of specifying a total bit number that can be allocated as a whole of a frame group including a plurality of frames;
Based on the accepted frame rate and the number of already encoded frames, a frame number identifying step of identifying the number of unencoded frames in the frame group,
Based on the specified total number of bits and the specified number of frames, a target bit number calculation step of calculating a target value of the number of bits to be allocated to the next frame to be encoded,
An average value calculation step of calculating an average value of the number of bits whose allocation has been determined along with encoding of a frame in the frame group,
An operation step of performing a predetermined operation on the calculated target value of the number of bits and the average value of the number of bits to specify the target value of the new number of bits; and Using a target value, a quantization step deriving step of deriving a quantization step for a frame to be encoded next,
A coding step of performing quantization based on the derived quantization step and performing coding based on the quantization. An image encoding device that performs encoding on an image signal sequentially input in frame units,
Encoding means for quantizing the frequency component of the image signal for each frame, and encoding based on the quantized result;
Quantization width calculation means for calculating a quantization width of a frame to be encoded next, based on the total number of bits and the number of bits already allocated to a frame group including a plurality of frames as a whole,
Frame rate calculation means for calculating an encoding frame rate indicating a cycle to be encoded for a frame to be encoded next, based on the calculated quantization width,
Based on the encoding frame rate, comprising an input control means for controlling the discard of the image signal input to the encoding means,
The image encoding apparatus according to claim 1, wherein the encoding unit quantizes the input image signal by the quantization width. The image encoding device further includes:
A transmission buffer that holds encoded data encoded by the encoding unit and outputs the retained encoded data to the outside by a fixed amount,
22. The image code according to claim 21, wherein the quantization width calculation means calculates the quantization width based on the amount of encoded data remaining unoutput in the transmission buffer. Device. 23. The image according to claim 22, wherein the quantization width calculation unit increases the quantization width of a frame to be encoded next when the code amount generated by the encoding exceeds a predetermined amount. Encoding device. The quantization width calculation means,
A comparison unit that compares the amount of code generated by the encoding with a predetermined threshold value;
As a result of the comparison, when the generated code amount exceeds the threshold value, a multiplication unit that multiplies the quantization width calculated for the immediately preceding input frame by a predetermined coefficient greater than 1.
A quantization width determination unit configured to determine a value closest to the prediction value from a predetermined value range as a prediction value of a quantization width using the multiplication result as a prediction value of a quantization width, and a quantization width of a frame to be encoded next. The image encoding device according to claim 23, wherein: 25. The image coding apparatus according to claim 24, wherein said quantization width calculation means further performs calculation such that when the code amount generated by said coding becomes equal to or less than a predetermined amount, said quantization width becomes smaller. The multiplying unit further multiplies the quantization width calculated for the immediately preceding input frame by a predetermined coefficient smaller than 1 when the generated code amount is equal to or smaller than the threshold value as a result of the comparison. 26. The image encoding apparatus according to claim 25, wherein: The multiplying unit further multiplies the quantization width calculated for the immediately preceding input frame by a predetermined coefficient greater than 1 when the generated code amount is larger than the threshold value as a result of the comparison. 26. The image encoding apparatus according to claim 25, wherein: The quantization width calculation means further includes:
A coefficient table holding unit that holds in advance a coefficient table indicating a value of a coefficient that increases the width of the quantization in association with the generated code amount, as the code amount generated by the coding increases,
A second multiplier for multiplying the quantization width calculated for the immediately preceding input frame by a coefficient corresponding to the generated code amount based on the coefficient table,
25. The quantization width determination unit according to claim 24, wherein the quantization width determination unit determines a quantization width of a frame to be encoded next, using a result of the multiplication by the second multiplication unit as a prediction value of the quantization width. Image coding device. The quantization width calculation means multiplies the quantization width used for quantization in the most recent frame by the coefficient to calculate a quantization width for a frame to be encoded next. 25. The image encoding device according to claim 24, wherein: 25. The frame rate calculation unit according to claim 24, wherein when the predicted value of the quantization width exceeds a predetermined threshold value, the frame rate calculation unit increases an encoding frame rate for a frame to be encoded next. Image coding device. When the predicted value of the quantization width exceeds the threshold value, the frame rate calculation unit multiplies the current encoding frame rate by a predetermined coefficient greater than 1 to generate a frame to be encoded next. 31. The image encoding apparatus according to claim 30, wherein said encoding frame rate is set to such an encoding frame rate. 31. The frame rate calculation unit according to claim 30, wherein, when the predicted value of the quantization width is equal to or smaller than the threshold value, the frame rate calculation unit further reduces an encoding frame rate of a frame to be encoded next. Image coding device. When the predicted value of the quantization width is equal to or smaller than the threshold value, the frame rate calculating means further multiplies the current encoding frame rate by a predetermined coefficient smaller than 1 to obtain a frame to be encoded next. 32. The image encoding apparatus according to claim 31, wherein the encoding frame rate is set to: When the determined width of the quantization exceeds a predetermined threshold, the frame rate calculation means increases the coding frame rate for the next frame to be coded. 25. The image coding apparatus according to claim 24, wherein a coding frame rate of the frame to be coded is reduced. The frame rate calculation means,
A quantization width averaging unit that calculates an average value of the determined quantization width,
When the average value of the quantization width exceeds a predetermined threshold, the frame rate calculation means increases the encoding frame rate for the next frame to be encoded. 25. The image coding apparatus according to claim 24, wherein a coding frame rate of a frame to be coded is reduced. When the code amount generated by the coding exceeds a predetermined threshold value, the frame rate calculation means further multiplies the calculated coding frame rate by a predetermined coefficient smaller than 1 to newly generate the frame rate. The image coding apparatus according to claim 21, wherein the calculated coding frame rate is used. When the code amount generated by the encoding is equal to or less than a predetermined threshold, the frame rate calculation unit further multiplies the calculated encoding frame rate by a predetermined coefficient larger than 1 to newly generate 22. The image coding apparatus according to claim 21, wherein the calculated coding frame rate is used. The input control means further comprises:
A frame rate averaging unit that calculates an average value of an encoding frame rate that is a calculation result of the frame rate calculation unit,
22. The image encoding apparatus according to claim 21, wherein the input control unit controls discarding of the input image signal based on the average value calculated by the frame rate averaging unit. A program for an image encoding device that encodes an image signal in frame units,
An encoding step of quantizing a frequency component of the image signal for each frame, and encoding based on the quantized result;
A quantization width calculation step of calculating a quantization width of a frame to be encoded next, based on the total number of bits related to the frame group and the number of allocated bits,
A frame rate calculating step of calculating an encoding frame rate indicating a cycle to be encoded for a frame to be encoded next, based on the calculated quantization width;
An input control step of controlling discarding of the image signal input to the encoding step, based on the encoding frame rate,
The program, wherein the encoding step quantizes the input image signal by the quantization width. A computer-readable recording medium that records a program for an image encoding device that encodes an image signal in frame units,
The program is
An encoding step of quantizing a frequency component of the image signal for each frame, and encoding based on the quantized result;
A quantization width calculation step of calculating a quantization width of a frame to be encoded next, based on the total number of bits related to the frame group and the number of allocated bits,
A frame rate calculating step of calculating an encoding frame rate indicating a cycle to be encoded for a frame to be encoded next, based on the calculated quantization width;
An input control step of controlling discarding of the image signal input to the encoding step, based on the encoding frame rate,
The recording medium, wherein the encoding step quantizes the input image signal with the quantization width. An image encoding method for encoding an image signal in frame units,
An encoding step of quantizing a frequency component of the image signal for each frame, and encoding based on the quantized result;
A quantization width calculation step of calculating a quantization width of a frame to be encoded next, based on the total number of bits related to the frame group and the number of allocated bits,
A frame rate calculating step of calculating an encoding frame rate indicating a cycle to be encoded for a frame to be encoded next, based on the calculated quantization width;
An input control step of controlling discarding of the image signal input to the encoding step, based on the encoding frame rate,
The image encoding method, wherein the encoding step quantizes the input image signal by the quantization width.

Images