JP4490046B2 - Image coding apparatus and image coding method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image encoding technique and an image decoding technique, and more particularly to an image encoding technique using a variable bit rate method or a variable frame rate method.
[0002]
[Prior art]
Conventionally, there is a variable bit rate (Variable Bit Rate) method as one of methods for controlling the bit rate in image coding technology such as MPEG (Moving Picture Expert Group) standard. This is mainly because local bits associated with the complexity of the image to be encoded in order to protect the restrictions such as the capacity of the buffer memory that stores the encoded data and the restrictions on the video recording time when recording on the recording medium. This is a control method in which the number of bits generated in a predetermined time length (for example, 1 second) is kept within a certain range while allowing a change in rate. In other words, generate more bits for encoding in scenes that are difficult to encode (higher bit rates), and reduce the number of bits to generate in scenes that are easier to encode (lower bit rates). Thus, the number of bits allocated according to the scene is controlled. That is, encoding is performed so that a high-quality image can be reproduced while keeping the number of generated bits within a certain time constant (that is, within the limits of buffer memory capacity and recording time). (For example, refer to Patent Document 1).
[0003]
The image coding apparatus using the conventional variable bit rate method will be described with reference to the image coding apparatus 100 in FIG. The image coding apparatus 100 includes an orthogonal transform unit 105, a quantization unit 106, a variable length coding unit 107, an inverse quantization unit 108, an inverse orthogonal transform unit 109, a frame memory 102, a motion detection unit 103, and a motion compensation unit 104. And a bit rate control unit 110.
[0004]
The orthogonal transform unit 105 performs discrete cosine transform (DCT) in units of macroblocks on the received encoded frame signal 101 (image signal data) to generate DCT coefficients and a quantization unit 106 Output to. Here, for an I (Intra coded) picture frame, DCT calculation is performed in the intra-picture coding mode. For a frame of a P (Predictive coded) picture, a DCT operation is performed in the forward predictive coding mode based on an I picture or P picture located in the past in time. For a B (Bidirectionally) picture frame, a DCT operation is performed in a bidirectional predictive coding mode based on an I picture or a P picture positioned before and after in time.
[0005]
The quantization unit 106 quantizes the DCT coefficients input from the orthogonal transform unit 105 in the quantization step (or may be a quantization parameter) received from the bit rate control unit 110 for each macroblock, and has a variable length. The data is output to the encoding unit 107 and the inverse quantization unit 108. The variable length coding unit 107 performs variable length coding and multiplexing on the quantized DCT coefficient and the like input from the quantization unit 106 and outputs the result to an output buffer (not shown).
[0006]
The inverse quantization unit 108 performs an inverse quantization operation on the quantized DCT coefficient received from the quantization unit 106 and outputs the result to the inverse orthogonal transform unit 109. The inverse orthogonal transform unit 109 performs an inverse orthogonal transform operation based on the inversely quantized DCT coefficient input from the inverse quantization unit 108 to reproduce image signal data, and outputs the image signal data to the frame memory 102.
[0007]
The frame memory 102 adds and stores the decoded image signal data of the I picture or P picture and the motion compensation data generated by the motion compensation unit 104. The motion detection unit 103 detects a motion vector from the reference image stored in the frame memory 102 and outputs data representing the motion vector to the motion compensation unit 104.
[0008]
The motion compensation unit 104 performs motion compensation data (reference image data) based on a reference image stored in the frame memory 102 for encoding a P picture or a B picture and data representing a motion vector input from the motion detection unit 103. ) Is generated. The bit rate control unit 110 receives the number of generated bits from the variable length coding unit 107, determines a quantization step based on the number of generated bits, and transmits this quantization step to the quantization unit 106.
[0009]
The overall control unit 140 is, for example, a microcomputer including a ROM, a RAM, and the like, and is a part that controls the entire image encoding apparatus 100. The overall control unit 140 controls each processing timing based on a control signal or the like.
[0010]
FIG. 33 is a block diagram illustrating a functional configuration of the bit rate control unit 110 in the conventional image encoding device 100. As shown in FIG. 33, the bit rate control unit 110 includes a frame group target bit number deriving unit 111, a next frame target bit number deriving unit 112, and a quantization step deriving unit 113.
[0011]
The frame group target bit number deriving unit 111 receives the generated bit number Nn131 from the variable length coding unit 107 and stores it in an internal memory (not shown). At this time, the frame group target bit number deriving unit 111 counts the number of times the generated bit number Nn131 is received (that is, the number of frames of the encoded frame). Further, the frame group target bit number deriving unit 111 calculates the number of bits that can be allocated to a frame that has not yet been encoded in units of frame groups, and transmits the calculated number of bits to the next frame target bit number deriving unit 112, as well as the actual encoding. The number of bits that can be allocated is sequentially updated using the number of generated bits Nn131 generated by the above. Here, the “frame group” refers to a group of frames that can be encoded in a predetermined time length.
[0012]
The next frame target bit number deriving unit 112 derives a target value for the number of bits to be allocated to the next frame based on the number of bits that can be allocated for each frame group received from the frame group target bit number deriving unit 111, and performs quantization. It transmits to the step derivation unit 113. The target value is calculated by dividing the number of bits that can be allocated in units of frames at that time by the number of remaining frames.
[0013]
The quantization step deriving unit 113 calculates a quantization step 141 (which may be a quantization parameter) based on the target value of the number of bits to be allocated to the next frame received from the next frame target bit number deriving unit 112 and performs quantum. To the conversion unit 106.
[0014]
FIG. 34 is a flowchart showing the flow of processing in the overall control unit 140 and the bit rate control unit 110 of the conventional image encoding device 100.
[0015]
First, when the generated bit number Nn131 is received by the bit rate control unit 110 (S1401), the overall control unit 140 determines whether or not the next frame to be encoded is the head of the frame group (S1402). In this case, when the next frame is the first frame of the frame group (S1402: Yes), the bit rate control unit 110 initializes the allocatable bit number Na, the allocation target period Ta, and the allocation target frame number Nt. (S1405 to 1407). Here, “NA” is an initial value of the number of bits assigned to each frame group. “TA” is a period of the entire frame group. “Rf” is an encoding frame rate of the image encoding apparatus 100.
[0016]
On the other hand, when the next frame is not the first frame of the frame group (S1402: No), the bit rate control unit 110 updates the allocatable bit number Na and the allocation target period Ta (S1403 to 1404).
[0017]
Next, the bit rate control unit 110 calculates the allocation bit number Nb based on the allocatable bit number Na and the allocation target frame number Nt (S1408), and updates (decrements) the allocation target frame number Nt (S1409). .
[0018]
Thereafter, the bit rate control unit 110 determines whether or not thinning is necessary for a frame to be encoded. If thinning is necessary, the bit rate control unit 110 notifies the overall control unit 140 of this (S1410: Yes). On the other hand, when it is not necessary to perform decimation (S1410: No), a quantization step is calculated and output to the quantization unit 106 (S1411).
[0019]
The overall control unit 140 and the bit rate control unit 110 repeat the above processing until the encoding processing ends (S1401 to 1412).
[0020]
FIG. 35 is a diagram showing a specific example of a method for calculating the number of allocated bits Nb in the conventional bit rate control unit 110. In this case, for the sake of convenience, the number of generated bits Nn131 and the number of assigned bits Nb are assumed to match.
[0021]
In the example of FIG. 35, image signal data (encoded frame signal 101) input at 30 [fps] is encoded at a frame rate of 15 [fps], and in principle is configured by 15 frames. One frame group is generated every second. In this example, the input encoded frame signal 101 is represented by “input image frame number 1701”, and the encoded frame signal 121 is represented by “frame group frame number 1702”. In FIG. 35, the column of the frame group frame number 1702 and the number of generated bits Nn 131 is “x” because the corresponding frame of the input image frame number 1701 is not coded but “thinned”. Represents.
[0022]
Furthermore, in FIG. 35, for example, the generated bit number Nn for the input image frame number “9” (that is, the frame group frame number “5”) is the total of the number of bits already assigned to the four frames is 1180 bits. Since the number of frames is “11”, ((3600-1180) / 11 = 220 bits).
[0023]
On the other hand, the number of generated bits for the input image frame number “13” (that is, the frame group frame number “6”) is the sum of the already allocated bits because the input image frame number “11” is “decimated”. Is 1400 bits, and the remaining number of frames is “9”, so the fractional part is rounded down ((3600-1400) / 9 = 244 bits).
[0024]
Note that “(14)” described in the column of the frame group frame number 1702 corresponding to the input image frame number “29” in FIG. 35 is the last frame when the input image frame number 1701 is “13”. This is an expected value of the group frame number 1702.
[0025]
As described above, the conventional image encoding apparatus 100 assigns bits to the next encoding target frame.
[0026]
[Patent Document 1]
Japanese Patent Laid-Open No. 2001-25015
[0027]
[Problems to be solved by the invention]
However, in the image coding apparatus 100 using the variable bit rate in the above-described prior art, when (complex) frames that are difficult to encode are continuous, the encoding step is accompanied even if the quantization step is changed to a large value. The number of generated bits increases (in this case, the bit rate value increases). Then, in order to keep the number of generated bits in a predetermined time length (one frame group) constant, the number of bits to be allocated is reduced for a frame after consecutive frames that are difficult to encode. For this reason, subsequent frames must be encoded with a large quantization step, and there is a first problem that the image quality of those frames is extremely deteriorated. The rate will be significantly smaller).
[0028]
FIG. 36 is a diagram for explaining the first problem of the conventional technology. In FIG. 36, 15 frames from the frame 1601 to the frame 1602 are defined as a frame group, and 3600 [bits] is assigned as a target value (as shown in FIG. 35 above) for each frame group. In this case, if scenes that are difficult to encode continue from time t1 and a large number of bits are assigned to the encoding of those frames, the number of bits that can be assigned to the latter half of the frame group is reduced. However, if the quantization step is set to a large value, the image quality is extremely deteriorated. In FIG. 36, the quantization step is particularly large between “t3 and t4”, and it can be estimated that the image quality is deteriorated accordingly.
[0029]
Furthermore, when the number of bits that can be allocated is finally reduced and the frame rate must be changed in the middle of the frame group (that is, the frame must be thinned out), the frame rate has changed even though the frame rate has changed. Since the number of bits allocated to one frame is determined based on the initial frame rate, a large number of bits cannot be allocated even though the frame rate is lowered. There is a second problem that it is necessary to roughen the image and further deteriorate the image quality.
[0030]
FIG. 37 is a diagram for explaining the second problem of the conventional technology. In the example of FIG. 37, scenes that are difficult to encode are continuous, and many bits are assigned to the encoding of those frames. As a result of gradually reducing the frame rate, the frame interval becomes large and eventually, It shows that the image becomes awkward and the quality of the image deteriorates.
[0031]
Therefore, the present invention has been made in view of the above problems, and even when scenes that are difficult to encode are continuous, image quality deterioration due to an extremely insufficient number of bits that can be allocated to the remaining frames. It is an object of the present invention to provide an image coding apparatus and method that can perform coding for high-quality images while also taking into account changes in the rate of frames to be coded.
[0032]
[Means for Solving the Problems]
In order to achieve the above object, an image encoding device according to the present invention is an image encoding device that encodes image signals sequentially input in units of frames, Encoding means for quantizing the frequency component of the image signal for each frame and encoding based on the quantized result; Frame group consisting of multiple frames In allocation Ru Total bit number specifying means for specifying the total number of bits, frame rate, In the frame group Frame number specifying means for specifying the number of unencoded frames in the frame group based on the number of already encoded frames, the specified total number of bits and the specified frame And a target bit number calculation means for calculating a target value of the number of bits to be allocated to the next frame to be encoded based on the number, and a frame to be encoded next using the calculated target number of bits A quantization step deriving means for deriving the quantization step; Based on the transmission buffer that holds the encoded data encoded by the encoding means and outputs to the outside by a certain amount, the amount of encoded data remaining in the transmission buffer, and the quantization step, Next, based on the frame rate determined by the frame rate control means, the frame rate control means for determining the frame rate indicating the cycle of the frame to be encoded, and the image signal input to the encoding means is controlled. Input control means And with The target bit number calculating means is the number of frames that are not yet encoded in the frame group calculated from the frame rate determined by the frame rate control means with the total bit number specified by the total bit number specifying means. The target value of the number of bits is calculated by dividing by The
[0033]
As a result, the number of bits to be allocated to the next frame to be encoded is calculated based on the total number of bits allocated to the entire frame group and the number of already allocated bits so that the frame group has a constant bit rate. Therefore, it is possible to avoid the case where the number of allocated bits is locally biased and to perform averaged bit allocation, and finally it is possible to prevent extreme deterioration in image quality.
[0034]
In order to achieve the above object, an image encoding device according to the present invention further includes a frame group in the frame group. Already encoded flame Occurrence based on the rate and the number of bits generated by it Average value calculating means for calculating an average value of the number of bits, and the calculated Occurrence The average number of bits On the other hand, the target value of the new number of bits obtained by multiplying by a predetermined coefficient is compared with the calculated target number of bits, and the larger one is set as the basic target value, and a preset lower limit value and the basic value are set. Compare the target value with the larger one Target value for new number of bits When And the quantization step deriving unit derives the quantization step using a target value of the number of bits specified by the calculating unit.
[0035]
Thus, the number of bits to be allocated to the next frame to be encoded is determined based on the number of unencoded frames calculated from the accepted frame rate and the number of assignable bits in the frame group. Coding according to the change in rate is possible.
[0036]
In order to achieve the above object, the image encoding apparatus according to the present invention further includes a storage unit that stores already encoded but untransmitted data, and a data amount stored in the storage unit. On the basis of this, a frame rate calculating means for calculating a frame rate relating to a frame to be encoded next is provided, wherein the frame rate receiving means receives the calculated frame rate.
[0037]
As a result, while maintaining a constant bit rate when observed for a predetermined time length, if there is sufficient space in the buffer, encoding can be performed with a higher quantization quality than before by encoding with a low quantization step. It is possible to suppress a sudden drop in the frame rate when a scene is shifted to a complicated scene (a lot of frames are skipped without being encoded), and encoding with smoother motion than before can be performed. Become.
[0038]
In order to achieve the above object, the present invention can be realized as an image encoding method using the characteristic means of the image encoding apparatus as steps, or as a program including these steps. it can. The program is not only stored in a ROM or the like included in the image encoding apparatus, but can also be distributed via a recording medium such as a CD-ROM or a transmission medium such as a communication network.
[0039]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments according to the present invention will be described in detail with reference to the drawings. In the following description, components that are the same as those in the conventional technology are denoted by the same reference numerals, and description thereof is omitted.
[0040]
(Embodiment 1)
FIG. 1 is a block diagram illustrating a functional configuration of an image encoding device 10 according to the first embodiment. This image encoding device 10 suppresses local bit rate fluctuations while keeping the bit rate constant when observed for a predetermined time length (for example, 1 second), thereby enabling higher quality images to be recorded. Enables encoding for reproduction.
[0041]
The image coding apparatus 10 includes a frame memory 102, a motion detection unit 103, a motion compensation unit 104, an orthogonal transform unit 105, a quantization unit 106, a variable length coding unit 107, an inverse quantization unit 108, an inverse orthogonal transform unit 109, The frame thinning unit 20, the rate control unit 30, and the overall control unit 40 are configured.
[0042]
The frame thinning unit 20 determines whether to encode / decode the frame of the encoded frame signal 101 input at a constant period, and sends the frame rate value Rf21 at that time to the rate control unit 30. Notice.
[0043]
Based on the number of generated bits received from the variable length coding unit 107 and the frame rate value Rf21 received from the frame thinning unit 20, the rate control unit 30 performs a quantization step (or a coding step in the next frame coding). Quantization parameter) is determined and transmitted to the quantization unit 106.
[0044]
The overall control unit 40 is, for example, a microcomputer including a ROM, a RAM, and the like, and controls the entire image encoding device 10. More specifically, the overall control unit 40 controls the processing timing of each unit based on a control signal (the broken line in the overall control unit 40 in FIG. 1 represents a control signal line). Further, the overall control unit 40 identifies the picture type of the encoded frame signal 101 input at a constant period (identification of I frame, B frame, and P frame), and sends the identification result to the rate control unit 30. Notice.
[0045]
FIG. 2 is a block diagram showing a detailed functional configuration of the rate control unit 30 in FIG. The rate control unit 30 includes a frame group target bit number deriving unit 31, a next frame target bit number deriving unit 32, an average frame bit number deriving unit 33, a calculating unit 34, a quantization step deriving unit 35, and a storage unit 36. .
[0046]
The frame group target bit number deriving unit 31 sequentially calculates and updates the target value of the total number of bits that can be allocated to all the remaining frames to be encoded in units of frame groups. More specifically, the frame group target bit number deriving unit 31 receives the generated bit number Nn 131 from the variable length coding unit 107 and stores it in the storage unit 36. Further, the frame group target bit number deriving unit 31 reads the total number of bits that can be allocated from the next frame to be encoded to the last frame of the frame group from the storage unit 36, and from the total number of bits, the above-described number of generated bits Nn 131 is subtracted, and the value is transmitted to the next frame target bit number deriving unit 32 and stored in the storage unit 36.
[0047]
The next frame target bit number deriving unit 32 determines the remaining frames to be encoded based on the frame rate value Rf21 received from the frame thinning unit 20 and the total allocatable bit number received from the frame group target bit number deriving unit 31. The target value of the number of bits to be allocated to the next frame is calculated in consideration of the number of frames.
[0048]
The average frame bit number deriving unit 33 calculates the average number of allocated bits based on the frame rate value Rf21 received from the frame thinning unit 20 and the number of bits allocated to the past frame stored in the storage unit 36. Calculate the value.
[0049]
The calculation unit 34 calculates a bit number obtained by multiplying the bit number calculated by the average frame bit number deriving unit 33 by a predetermined coefficient (for example, 0.8), and this and the next frame target bit number deriving unit 32 The calculated number of bits is compared with the target value, and the larger number of bits is selected. Further, the calculation unit 34 compares the selected number of bits with a preset lower limit value and selects a larger number of bits (that is, selects a number of bits that does not fall below the lower limit value). The data is transmitted to the quantization step deriving unit 35. The predetermined coefficient is not limited to “0.8”, and may be another value greater than 0 and less than or equal to 1.
[0050]
The quantization step deriving unit 35 derives the value of the quantization step so that the number of bits received from the calculation unit 34 is equal to the number of generated bits.
[0051]
The storage unit 36 stores the number of generated bits Nn 131 that is a result of encoding each frame received from the variable length encoding unit 107. Further, the storage unit 36 updates the frame group target bit number according to the control of the frame group target bit number deriving unit 31.
[0052]
FIG. 3 is a diagram illustrating general characteristics of the number of generated bits for each picture type in image coding. As shown in FIG. 3, the frame (P or B picture) of the inter-picture coding (P picture or B picture) is usually determined from the number of generated bits Nn in the frame of the intra picture coding (I picture) (I frames 210 to 250). The number of generated bits Nn in (B frame) is smaller. Therefore, by defining one frame group from the I frame to the next I frame, when defining a frame group based on other rules (for example, forming a frame group every 15 frames regardless of the picture type) In comparison with the case (1), the deviation of the number of generated bits in each frame group can be reduced. In the example of FIG. 3, for the sake of convenience, the time required for encoding each frame group is TA, and the period is the same.
[0053]
It should be noted that the effect of reducing the deviation in the number of bits in each frame group can be obtained only by the first frame of the frame group being the I frame or the last frame of the frame group being the I frame. .
[0054]
FIG. 4 is a diagram showing a specific example of the frame group defined in FIG. As shown in FIG. 4, the frame group 310 includes 15 frames from the intra-frame coding (I) frame 210 to the frame immediately before the intra-frame coding (I) frame 220. Similarly, each of the frame group 220 and the frame group 230 is composed of 15 frames. In this case, an initial value of the total number of bits that can be assigned to one frame group is determined in advance (for example, 3600 “bits” etc.) Note that the method of defining a frame group is not limited to FIG. .
[0055]
FIG. 5 shows a modification of the frame group defined in FIG. As shown in FIG. 5, a group of frames including a first frame set and a second frame set corresponding to the two frame groups 310 and 320 in FIG. 4 may be defined as one frame group 410. In this case, the initial value of the total number of bits that can be allocated to the frame group 410 is 7200 [bits] (3600 [bits for each of the first frame set and the second frame set)].
[0056]
FIG. 6 is a diagram for explaining the effect of configuring a frame group with two frame sets as shown in FIG. FIG. 6A shows the number of allocated bits of the next frame to be encoded including the number of allocatable bits in the next second frame set even when there is a scene that is difficult to encode in one frame group. Therefore, the change in the bit rate can be more relaxed. For example, 1000 bits are assigned as an initial value for the entire frame group 410, and 500 bits are temporarily assigned to the first frame set and 500 bits are temporarily assigned to the second frame set. Thereafter, even if 700 bits are used in the first frame set in actual encoding, the next frame group 420 has a total of 800 [bits] as the initial value (the remaining 300 [bits] of the previous frame group 410). + New second frame set (500 [bits]) is allocated.
[0057]
In FIG. 6B, even when the frame rate has to be changed because the encoding is further difficult, the bit rate is rapidly reduced to a small value by the same method as in FIG. That is, it is possible to avoid a sudden increase in the quantization step).
[0058]
FIG. 7 is a flowchart showing the flow of processing in the overall control unit 40 and the rate control unit 30 of the image encoding device 10.
[0059]
First, the rate control unit 30 receives the generated bit number Nn131 from the variable length coding unit 107 (S701), and receives the frame rate value Rf21 from the frame thinning unit 20 (S702).
[0060]
Next, the overall control unit 40 identifies whether or not the next frame is an I frame (S703). In this case, when the next frame is an I frame (S704: Yes), the rate control unit 30 initializes the allocatable bit number Na, the allocation target period Ta, and the allocation target frame number Nt (S708 to S710). .
[0061]
On the other hand, if the next frame is not an I frame (S704: No), the rate control unit 30 updates the allocatable bit number Na and the allocation target frame number Nt (S705, S706). Further, the rate control unit 30 rounds off the decimal part of the allocation target frame number Nt and converts this Nt into an integer (S707).
[0062]
Thereafter, the rate control unit 30 calculates the allocation bit number Nb based on the allocatable bit number Na and the allocation target frame number Nt (S711), and updates the allocation target period Ta (S712). In this case, the allocation target period Ta is updated in consideration of a case where the frame rate value Rf21 is changed in the middle of the same frame group.
[0063]
Thereby, the rate control unit 30 derives a quantization step based on the allocated bit number Nb, and transmits the quantization step to the variable length coding unit 107 (S713).
[0064]
The overall control unit 40 and the rate control unit 30 repeat the above processing until the encoding processing is completed (S701 to S714).
[0065]
FIG. 8 is an example illustrating a change in bit rate when encoding is performed using the image encoding device 10. When the change in the bit rate shown in FIG. 8 is seen, the number of bits that can be allocated is reduced because of the coefficient (for example, 0.8) in the calculation unit 34 as compared with the case of the prior art (FIG. 36 above). Therefore, the change in bit rate tends to slow down. Further, in FIG. 8, the arithmetic unit 34 selects the number of bits that does not fall below the lower limit value, and calculates the quantization step so that the number of bits after encoding becomes this number of bits. Furthermore, since the encoding is executed in accordance with the calculated quantization step and the generated bits are determined, it is shown that the bit rate is equal to or greater than the lower limit value.
[0066]
FIG. 9 is an example showing how the bit rate fluctuates when one frame group is composed of two frame sets as shown in FIG. As shown in FIG. 9, in the prior art (FIG. 36 above), the bit rate value is small during “t3−t4” where the bit rate value is extremely small (that is, the quantization step is extremely large). However, the effect of using a large number of bits in the first frame set appears in the first part of the second frame set (between “t4 and t6”). (That is, the quantization step cannot be reduced and the bit rate value remains small).
[0067]
FIG. 10 shows an example of how the bit rate changes when one frame group is composed of two frame sets and the encoding frame rate changes in the middle of the frame group. In FIG. 10, the frame rate is changed from 15 [fps] to 7.5 [fps], but the way of changing the bit rate is the same as the bit rate of FIG. 8 (that is, in the case of FIG. 10). In this case, it is assumed that the bit rate is the same because encoding is performed at a quantization step of ½ and the number of bits is doubled. In this case as well, between “t3 and t4” Image quality deterioration is suppressed).
[0068]
FIG. 11 is a diagram showing a specific example of a method for calculating the number of bits in the calculation unit 34 of FIG. In the example of FIG. 11, the encoded frame signal 101 input at 30 [fps] is encoded at a frame rate of 15 [fps]. Also in this example, similarly to FIG. 37, the input encoded frame signal 101 is represented by “input image frame number 1701”, and the encoded frame signal 121 is represented by “frame group frame number 1102”. In FIG. 11, the column of the frame group frame number 1102 and the number of generated bits Nn131 is “x” because the corresponding input image frame number 1701 is not coded but “thinned”. Represents.
[0069]
Further, in FIG. 11, for example, the target value of the number of generated bits for the input image frame number “9” (that is, the frame group frame number “5”) is 1180 [bits] with the total number of already assigned bits remaining. Since the number of frames is “11”, ((3600-1180) / 11 = 220 [bits]). In addition, the average number of generated bits (1180/4 = 295 [bits]) in the past frame in the average frame bit number deriving unit 33 is calculated, and the arithmetic unit 34 calculates a coefficient (for example, 0.8) and compare this with the target value of the number of bits described above to select the larger one. As a result, the number of generated bits for the frame group frame number “5” is 236 [bits].
[0070]
On the other hand, the number of generated bits for the input image frame number “13” (that is, the frame group frame number “6”) is “decimated” for the frame of the input image frame number “11” (thus, the frame rate value). Rf21 is changed to 7.5 [fps]), and the number of remaining frames (because the frame rate value Rf21 is calculated at 7.5 [fps]) is calculated based on “5”. Furthermore, since the fractional part is rounded down to the calculated value, (3600-1416) / 5 = 436 [bits]. In addition, the average number of generated bits in the past frame in the average frame bit number deriving unit 33 (1416/5 = 383 [bits]) is calculated, and the arithmetic unit 34 calculates a coefficient (for example, 0.8) and compare this with the target value of the number of bits to select the larger value. As a result, the number of generated bits for the input image frame number “13” is 436 [bits].
[0071]
When comparing the number of generated bits Nn of the frame group frame numbers “5” and “6” in FIG. 11 and FIG. 35 of the above prior art, both are generated in the image coding apparatus 10 (that is, FIG. 11 above). Since the number of bits is a larger value and a larger number of bits can be assigned, it is clear that encoding for a higher quality image than before is possible.
[0072]
Note that although two numbers are written in the frame group frame number column corresponding to the input image frame number “29” in FIG. 11, “(15)” indicates the time when the input image frame number is “9”. Is the expected value of the frame group frame number 1102 when the frame rate is 15 [fps], and “(10)” is a frame rate of 7.5 [fps] when the input image frame number is “13”. ] Is an expected value of the frame group frame number 1102.
[0073]
In the first embodiment, the frame thinning unit 20 is provided and the bit rate is controlled based on the frame rate value Rf21 transmitted from the frame thinning unit 20. The value may be obtained from other than the frame thinning unit.
[0074]
FIG. 12 is a block diagram showing a part of the functional configuration of an image encoding apparatus 50 configured not to provide the frame thinning unit 20 in FIG. 1 but to obtain the frame rate value Rf21 from other departments. It is. The image encoding device 50 is different from the image encoding device 10 in FIG. 1 only in that the frame thinning unit 20 is not provided and an overall control unit 60 is provided instead of the overall control unit 40.
[0075]
In addition, since the quantization step in the above embodiment uniquely corresponds to the quantization parameter, the quantization step may be replaced with the quantization parameter.
[0076]
As described above, according to the image coding apparatus according to the present embodiment, the number of bits to be allocated is determined while responding to changes in the frame rate of coding, so whether or not image coding is difficult. Therefore, it is possible to ensure a good image quality on average, and to prevent the movement from becoming extremely awkward or the image quality from being deteriorated.
[0077]
Furthermore, the bitstream encoded by the image encoding device shown in the first embodiment can be decoded by a general image decoding method. Also, in the stream that has been encoded according to the first embodiment, the quantization step of each frame varies according to the complexity of the image, but the variation is not extreme, and the intra-screen coded frame to the intra-screen It is possible to make the number of generated bits substantially constant in a frame group composed of frames up to immediately before the encoded frame.
[0078]
In the first embodiment, the period required for encoding each frame group is the same as “TA”. However, the period is not limited to the same, and the time required for encoding each frame group is different. It may be.
[0079]
In the above embodiment, only the average value of the number of bits calculated by the average frame bit number deriving unit is multiplied by a predetermined coefficient (for example, 0.8), and this is calculated by the next frame target bit number deriving unit. Although the configuration is such that the larger number of bits is selected by comparing with the set target value of the number of bits, it may be configured such that both are multiplied by a predetermined coefficient for comparison.
[0080]
(Embodiment 2)
FIG. 13 is a block diagram illustrating a functional configuration of the image encoding device 2100 according to the second embodiment. As shown in FIG. 5 of the first embodiment, the image encoding device 2100 controls quantization steps and frame thinning while considering a group of frames composed of twice as many frames as the conventional one. By doing so, it is possible to perform encoding with high image quality while suppressing rapid fluctuations in the bit rate and frame rate.
[0081]
The image coding apparatus 2100 includes a frame thinning unit 2101, a frame memory 102, a motion detection unit 103, a motion compensation unit 104, an orthogonal transform unit 105, a quantization unit 106, a variable length coding unit 107, a transmission buffer 2108, an inverse quantum A conversion unit 2109, an inverse orthogonal transform unit 2110, a subtractor 2111, an adder 2112, and a frame rate control unit 2113.
[0082]
The frame decimation unit 2101 performs decimation processing on each frame of the image signal input at the reference frame rate (for example, 30 Hz) according to the decimation information input from the frame rate control unit 2113, and the decimation is performed. The image signal is output to the motion detector 103 and the differentiator 2111. For example, when the value of the decimation information input from the frame rate control unit 2113 to the frame decimation unit 2101 is “1”, it indicates an instruction to perform encoding without decimation, and when the value is “0” It is assumed that an instruction to perform coding by thinning out one frame is shown. That is, when the decimation information is “1”, the frame decimation unit 2101 outputs the frame input as the image signal as it is to the motion detection unit 103 and the subtractor 2111 as an encoded frame. Conversely, when the thinning information is “0”, a process of thinning (for example, discarding) the input frame is performed.
[0083]
The frame memory 102 is realized by a RAM, a hard disk, or the like, and is a storage device for holding image data in units of frames. Furthermore, the frame memory 102 stores reference frame data serving as a reference image for the next inter-frame predictive coding frame. The motion detection unit 103 detects the motion of the encoded frame using the reference frame data stored in the frame memory 102 when the input encoded frame is a frame on which inter-frame prediction encoding is performed. The motion compensation unit 104 generates motion compensation data corresponding to the motion detected by the motion detection unit 103. This motion compensation data is data indicating a motion vector indicating a correspondence between a reference frame and an encoded frame, and a difference between image signals between blocks indicated by the motion vector.
[0084]
The subtractor 2111 obtains a difference between the motion vector of the frame to be encoded and the motion vector based on the motion compensation data generated by the motion compensation unit 104 when the frame to be encoded is a frame for which inter-frame prediction encoding is performed. . On the other hand, when the frame to be encoded is a frame for which intra-frame prediction encoding is performed, the differentiator 2111 performs prediction within the screen without performing processing for obtaining a difference from the reference frame for the input image signal. After that, the image signal is output to the orthogonal transform unit 105.
[0085]
The orthogonal transform unit 105 orthogonally transforms the motion compensation data generated in the motion compensation unit 104 into data representing a frequency component by DCT (Discrete Cosine Transform) transformation or the like, and outputs the transformation result to the quantization unit 106. The quantization unit 106 quantizes the orthogonally transformed data using the quantization step input from the frame rate control unit 2113, and the quantized result is sent to the variable length coding unit 107 and the inverse quantization unit 2109. Output. The variable length coding unit 107 performs variable length coding on the data quantized by the quantization unit 106 using a Huffman code or the like. The data subjected to the variable length encoding by the variable length encoding unit 107 is stored in the transmission buffer 2108 and output as an encoded bit stream.
[0086]
The transmission buffer 2108 is a first-in first-out (FIFO) memory realized by a RAM or the like. For example, as shown in FIG. 5 of the first embodiment, the transmission buffer 2108 stores the total number of bits allocated to a frame group composed of a plurality of frames, and the remaining data amount or free space inside. When the capacity is monitored and the variable-length encoded data is transmitted to the transmission buffer 2108, that is, when the encoding of the image signal for one frame is completed, the remaining data amount of the transmission buffer 2108 is The data is transmitted to the rate control unit 2113.
[0087]
The inverse quantization unit 2109 inversely quantizes the data quantized by the quantization unit 106 to generate a predicted image, and outputs the data to the inverse orthogonal transform unit 2110. The inverse orthogonal transform unit 2110 performs inverse orthogonal transform on the data inversely quantized by the inverse quantization unit 2109. The adder 2112 adds the data inversely quantized by the inverse orthogonal transform unit 2110 and the motion compensation data generated by the motion compensation unit 104 and outputs the result to the frame memory 102. The remaining data amount in the transmission buffer 2108 is input from the transmission buffer 2108 to the frame rate control unit 2113. The frame rate control unit 2113 calculates a quantization step corresponding to the remaining amount of data in the transmission buffer 2108, outputs the quantization step to the quantization unit 106, calculates a frame rate corresponding to the calculated quantization step, Decimation information indicating whether or not to decimate the current frame is determined and output to the frame decimation unit 2101. Here, the “quantization step” is a reference value (width) when quantizing the frequency component of each frame, for example, a natural number (“quantization parameter”) from “1” to “31”. "). This quantization parameter has a one-to-one correspondence with the quantization step, and is defined such that the greater the numerical value, the greater the quantization width. Hereinafter, this quantization parameter will be described as a “quantization step”.
[0088]
FIG. 14 is a block diagram showing a detailed functional configuration of the frame rate control unit 2113 in FIG. The frame rate control unit 2113 includes a quantization step prediction unit 2201, a frame rate calculation unit 2202, a quantization step determination unit 2203, a frame number calculation unit 2204, and a comparator 2205. The quantization step prediction unit 2201 performs a predetermined operation on the current quantization step in accordance with the remaining amount of data input from the transmission buffer 2108, calculates a quantization step prediction value, and calculates a frame rate calculation unit 2202 and It outputs to the quantization step determination part 2203. The quantization step determination unit 203 selects one natural number from “1” to “31” that is closest to the quantization step prediction value input from the quantization step prediction unit 201, and then selects the selected natural number for the next quantization Decide on a step. Specifically, when the input quantization step predicted value includes a value after the decimal point, rounding processing such as rounding off, rounding down or rounding up is performed. In this case, when the value is less than “1”, “1” is selected, and when it exceeds “31”, “31” is selected.
[0089]
Based on the remaining data amount input from the transmission buffer 2108 and the quantization step prediction value input from the quantization step prediction unit 201, the frame rate calculation unit 2202 performs the next encoding from the current frame rate. Calculate the optimal frame rate at. The frame number calculation unit 204 calculates the current frame rate based on the thinning information output from the comparator 205. The comparator 205 compares the optimum frame rate that is the output of the frame rate calculation unit 202 with the current frame rate that is the output of the frame number calculation unit 204, and if the optimum frame rate is greater than the current frame rate. , “1” indicating an instruction not to be thinned is output as the thinning information. On the other hand, when the optimum frame rate is smaller than the current frame rate, “0” indicating an instruction to perform decimation is output as decimation information.
[0090]
FIG. 15 is a block diagram for explaining the function of the quantization step prediction unit 201 in FIG. FIG. 15A is a diagram schematically illustrating a quantization step prediction method in the quantization step prediction unit 201. FIG. 15B is a diagram illustrating a prediction coefficient table 303 held in the quantization step prediction unit 201. The quantization step prediction unit 201 holds a current value memory 2302 illustrated in FIG. 15A and a prediction coefficient table 2303 illustrated in FIG. The current value memory 302 holds the initial value of the quantization step at the start of encoding, and holds the quantization step prediction value calculated by the quantization step prediction unit 201 after the start of encoding. The prediction coefficient table 303 stores the remaining data amount of the transmission buffer 2108 and the prediction coefficient in association with each other, and the quantization step prediction unit 201 predicts the prediction coefficient corresponding to the remaining data amount input from the transmission buffer 2108. Is multiplied by the current quantization step prediction value held in the current value memory 302 to calculate the quantization step prediction value of the next frame.
[0091]
As shown in FIG. 15A and FIG. 15B, the prediction coefficient is “1.2” when the remaining data amount in the transmission buffer 2108 is 50% or more and less than 80%, and 80% or more and less than 90%. In this case, the bit amount of the encoded data is reduced by increasing the value as the free capacity in the transmission buffer 2108 decreases, such as “1.5”, and “2” when 90% or more and 100% or less. Is set as follows. The prediction coefficient is set to “1” so that the quantization step is maintained as it is when the remaining data amount in the transmission buffer 2108 is 40% or more and less than 50%. Contrary to the above, the prediction coefficient is “0.8” when the remaining data amount of the transmission buffer 2108 is 30% or more and less than 40%, “0.5” when it is 20% or more and less than 30%, In the case of 0% or more and less than 20%, it is set as “0.4”. That is, the prediction coefficient is set so that the bit amount of data to be encoded increases as the free space in the transmission buffer 2108 increases (that is, the quantization step is reduced to finely quantize the quality of image quality). Set to enhance).
[0092]
For example, if the quantization step prediction value in the current value memory 302 is “7.5” and the remaining data amount in the transmission buffer 2108 is 75%, the quantization in the current value memory 302 is performed in the next frame. The step prediction value “7.5” is multiplied by the prediction coefficient “1.2”, and the quantization step prediction value for the frame becomes “9”. The calculated quantization step prediction value “9” is overwritten in the current value memory 2302 by the quantization step prediction unit 2201. Since the predicted quantization step value “9” is a natural number from “1” to “31”, the quantization step determination unit 2203 determines the quantization step as the quantization step. If the remaining amount of data in the transmission buffer 2108 is 70% in the next frame after the frame is quantized in the quantization step “9”, the quantization step predicted value “ 9 ”is multiplied by the prediction coefficient“ 1.2 ”, and the next quantization step prediction value becomes“ 10.8 ”. When the quantization step determination unit 203 determines the quantization step by rounding off the decimals, the next frame is quantized at the quantization step “11”. By setting the prediction coefficient in this way, when the remaining amount of data in the transmission buffer 2108 becomes 50% or more, the quantization step increases, so that the quality of the image to be encoded gradually becomes rough and encoded. The bit amount of data can be gradually reduced. On the other hand, when the remaining amount of data in the transmission buffer 2108 becomes less than 40%, the quantization step decreases, so the data bit amount gradually increases, but the image quality of the encoded image can be gradually improved. .
[0093]
FIG. 16 is a block diagram for explaining the function of the frame rate calculation unit 2202 in FIG. The frame rate calculation unit 202 is a unit that calculates the optimum frame rate based on the remaining data amount of the transmission buffer 2108 and the quantization step predicted value. The comparator 2401, the comparator 2402, the frame rate table 2403, the threshold memory 2404, a threshold memory 2405, and a free space calculation unit 2406. The threshold memory 2404 and the threshold memory 2405 are realized by a latch circuit or a RAM. The comparator 2401 receives the quantization step prediction value that is the output of the quantization step prediction unit 2201 and the threshold value B held in the threshold memory 2404. The comparator 2401 compares these two inputs. For example, when the predicted quantization step value exceeds the threshold B, the output D1 is “1”, and when the predicted quantization step value is equal to or less than the threshold B, the output D1 is “0”. Is output. The comparator 2402 receives the free capacity of the transmission buffer 2108 calculated by the free capacity calculation unit 2406 and the threshold value C held in the threshold memory 2405. The comparator 2402 compares these two inputs, for example, outputs “1” as the output D2 when the free capacity of the transmission buffer 2108 exceeds the threshold C, and outputs “0” as the output D2 when the free capacity is equal to or less than the threshold C. To do. The frame rate table 2403 is a table indicating the optimum frame rate corresponding to the combination of the output D1 of the comparator 2401 and the output D2 of the comparator 2402. The threshold memory 2404 holds a preset threshold B of the quantization step prediction value. The threshold memory 2405 holds a preset threshold C of the transmission buffer 2108. The free capacity calculation unit 406 calculates the current free capacity of the transmission buffer 2108 based on the total data capacity of the transmission buffer 2108 and the remaining data amount input from the transmission buffer 2108.
[0094]
Specifically, the frame rate calculation unit 2202 calculates the optimum frame rate for the encoding target frame according to the frame rate table 2403. For example, when the free capacity of the transmission buffer 2108 exceeds the threshold value C and the predicted quantization step value is equal to or less than the threshold value B, that is, when the transmission buffer 2108 has room despite the small quantization step value. The optimum frame rate is determined (for example, 30 Hz) so that the frame rate is maximized. Conversely, when the free capacity of the transmission buffer 2108 is equal to or smaller than the threshold C and the predicted quantization step value exceeds the threshold B, that is, the transmission buffer 2108 has a margin despite the increase in the quantization step. If not, the optimal frame rate is determined to be a minimum (for example, 10 Hz). Furthermore, when both the free capacity of the transmission buffer 2108 and the predicted quantization step value are larger than the threshold value or both are equal to or smaller than the threshold value, the optimal frame rate is determined so that the frame rate becomes an intermediate value.
[0095]
FIG. 17 is a diagram illustrating an example of each input signal and output signal of the frame rate calculation unit 2202 in FIG. FIG. 17A is a diagram illustrating an optimum frame rate that is an output signal of the frame rate calculation unit 2202. FIG. 17B is a diagram illustrating a quantization step prediction value that is one input signal of the frame rate calculation unit 202. FIG. 17C is a diagram illustrating the free capacity of the transmission buffer 2108 that is the other input signal of the frame rate calculation unit 202. More precisely, this free space is calculated by the free space calculation unit 2406 from the input remaining data amount. A curve L1 in FIG. 17B shows how the quantization step predicted value changes with time, and a curve L2 in FIG. 17C shows the time change in the free capacity of the transmission buffer 2108. Yes. As shown in FIG. 17B, the comparator 2401 outputs D1 = 0 while the quantized step predicted value is equal to or smaller than the threshold B, and when the quantized step predicted value exceeds the threshold B, the comparator 2401 outputs D1. = 1 is output. When the quantized step prediction value becomes equal to or smaller than the threshold value B again, the comparator 401 outputs D1 = 0. On the other hand, as shown in FIG. 17C, the comparator 2402 outputs D2 = 0 while the free capacity of the transmission buffer 2108 is equal to or less than the threshold C, and when the free capacity exceeds the threshold C, the comparator 2402 outputs D2 = 1 is output. Further, when the free space becomes equal to or less than the threshold value C again, D2 = 0 is output.
[0096]
The frame rate table 2403 held by the frame rate calculation unit 2202 includes four values “0” or “1” of the output D1 of the comparator 2401 and “0” or “1” of the output D2 of the comparator 2402. Four types of frame rates are described corresponding to the combinations of the streets. The frame rate calculation unit 2202 outputs 20 Hz as the optimum frame rate as shown in FIG. 17A while D1 = 0 (quantization step prediction value is small) and D2 = 0 (small free space). . Further, as shown in FIGS. 17B and 17C, when only the quantization step prediction value indicated by the curve L1 exceeds the threshold value B, that is, D1 = 1 (quantization step prediction). While the value is large) and D2 = 0 (the free space is small), the frame rate calculation unit 2202 outputs 15 Hz as the optimum frame rate. Subsequently, when both the quantization step predicted value and the free capacity exceed the threshold, that is, while D1 = 1 (the quantized step predicted value is large) and D2 = 1 (the free capacity is large), the frame rate calculation is performed. The unit 2202 outputs 20 Hz as the optimum frame rate in order to control the quantization step to be small. Further, when only the quantization step predicted value is equal to or less than the threshold value B, that is, while D1 = 0 (the quantization step predicted value is small) and D2 = 1 (the free capacity is large), the frame rate calculation unit 202 outputs 30 Hz as the optimum frame rate. As described above, the frame rate calculation unit 202 gradually increases or decreases the frame rate according to the quantization step prediction value and the free capacity of the transmission buffer 2108 (although there are four levels here).
[0097]
FIG. 18 is a diagram for explaining the operation of the frame number calculation unit 204 shown in FIG. FIG. 18A is a diagram illustrating an example of thinning information output from the comparator 2205. FIG. 18B is a block diagram illustrating a detailed configuration of the frame number calculation unit 2204. As illustrated in FIG. 18A, the thinning information output from the comparator 2205 is input to the frame number calculation unit 2204 at 30 Hz. This thinning information is a binary signal of “0” or “1”. “0” in the thinning information indicates that the frame has been thinned out, and “1” in the thinning information indicates that the frame has been encoded. The number-of-frames calculation unit 2204 in FIG. 18B calculates a current frame rate based on the thinning information output from the comparator 2205, and outputs a calculation result at a predetermined period (for example, 5 Hz). Or a circuit. The frame number calculation unit 2204 includes a thinning information memory 2601, a counter 2602, and a calculator 2603. The thinning information memory 2601 is a FIFO (first_in first_out) memory device that holds first-in first-out, and holds 30 bits of 30-bit thinning-out information input at 30 Hz from the current frame. The counter 2602 counts the number m of frames whose thinning information is “1” in the thinning information memory 2601 and the number n of frames whose thinning information is “0” in a predetermined cycle (for example, 5 Hz). The values of m and n are reset in a cycle (for example, after 1/5 second). The computing unit 2603 calculates “m / (m + n)” at the predetermined period and outputs the result as the current frame rate. As a result, when the frame number calculation unit 2204 calculates and outputs the current frame rate, for example, five times per second, it is possible to calculate an appropriate quantization step every 0.2 seconds.
[0098]
The comparator 2205 of the frame rate control unit 2113 compares the current frame rate calculated by the frame number calculation unit 2204 with the optimal frame rate calculated by the frame rate calculation unit 2202, and the optimal frame rate is determined to be the current frame rate. If the rate is greater than or equal to the rate, “1” indicating an encoding instruction is output as thinning information. On the other hand, if the optimum frame rate is less than the current frame rate, “0” indicating a thinning instruction is output as thinning information. As a result, the frame rate of the encoded frame becomes equal to the optimum frame rate calculated by the frame rate calculation unit 2202. That is, the present image coding apparatus 2100 controls the quantization step according to the free capacity of the transmission buffer 2108. Therefore, when the quantization step is controlled by further relaxing the time change of the free capacity of the transmission buffer 2108, That is, when encoding is performed by gradually increasing / decreasing the frame rate in accordance with the quantization step prediction value and the free capacity of the transmission buffer 2108, it is possible to avoid rapid image degradation.
[0099]
FIG. 19 is a diagram for describing an outline of an input image signal and a frame to be encoded in the image encoding device 2100. FIG. 19A is a diagram showing each frame of an image signal input to the image encoding device 2100. As shown in FIG. 19A, the image coding apparatus 2100 forms one frame group 2604 with two frame sets, as in the case of FIG. 5 of the first embodiment, and this frame group 2604 is One unit is used for determining the quantization step and the frame rate. FIG. 19B is a diagram showing each encoded frame encoded by the image encoding device 2100.
[0100]
As shown in FIG. 19A, an image signal input to the image encoding device 2100 includes a scene that is easy to encode and a scene that is difficult to encode. The generated bit amount of encoded data is reduced. On the other hand, in a scene where encoding is difficult, the generated bit amount of encoded data increases. For example, if a scene that is difficult to encode is input to the image encoding device 2100 at time a, the amount of generated bits due to the encoding of the first frame of the scene that is difficult to encode is large. The quantization step prediction unit 2201 increases the quantization step prediction value according to the free space. For example, the quantization step prediction value predicted by the quantization step prediction unit 2201 is increased to 1.5 times the quantization step prediction value corresponding to the immediately preceding frame. As a result, the amount of bits generated by encoding the next frame is suppressed. As a result, the amount of bits generated by encoding does not increase as much as when quantized using the original quantization step, and the free capacity of the transmission buffer 2108 does not drop rapidly. Therefore, even in the case where a large amount of frame thinning is required continuously in the past, the image coding apparatus 2100 can reduce the frame rate more gently.
[0101]
Still, when the free capacity of the transmission buffer 2108 decreases to become the threshold value C or less, the frame rate calculation unit 202 changes the optimal frame rate to a frame rate that is one step lower. For example, when encoding is performed at a frame rate of 30 Hz until time a, the frame rate calculation unit 2202 changes the optimal frame rate from the next frame to 20 Hz. Thus, even if the quantization step is increased and the frame rate is decreased, the capacity of the transmission buffer 2108 is not sufficient, the predicted quantization step value further increases, and the threshold B is exceeded. The unit 2202 changes the next optimum frame rate to a frame rate that is one step lower. For example, when encoding is performed at a frame rate of 20 Hz until time b, the frame rate calculation unit 2202 changes the next optimum frame rate to 15 Hz.
[0102]
As described above, the image encoding device 100 calculates the quantization step prediction value according to the generated bit amount by encoding even when a scene difficult to encode is input, and the quantization step prediction value is calculated. Since the optimum frame rate can be reflected, the frame rate can be reduced more gradually and gradually. On the other hand, in the input image signal, for example, even when the scene that is difficult to encode is switched to a scene that is easy to encode at time c, the image quality is as high as possible by reducing the quantization step first. If encoding is performed and there is still room in the transmission buffer (the scene is easy to encode), the frame rate is changed to a larger value, so the frame rate is increased more gradually and gradually. can do. For example, if a scene that is easy to encode at time c is input and the transmission buffer 2108 still has room even if the quantization step is reduced, the frame rate is changed to 20 Hz, which is one stage higher at time d. When the quantization step becomes sufficiently small at time e, the frame rate is changed to 30 Hz, which is one stage higher, so that the frame rate can be increased more gently. As a result, when the image signal encoded by the image encoding device 100 is decoded by the decoding device, even in a scene where encoding is difficult, the image does not suddenly become rough or suddenly awkward. A moving image can be reproduced. On the other hand, even when switching to a scene that is easy to encode, the shade of the image does not suddenly become smooth or the movement suddenly becomes smooth, so that the human eye does not feel unnaturalness. Images with high image quality can be reproduced.
[0103]
As described above, according to the image coding apparatus 2100 of the present embodiment, when the buffer has sufficient vacancy while maintaining a constant bit rate when observed for a predetermined time length (for example, 1 second). By encoding with a low quantization step, it is possible to encode with higher image quality than before, and a drastic drop in the frame rate when moving to a complicated scene (many frames are skipped without being encoded) Encoding can be performed with a smoother motion than in the prior art.
[0104]
In the above embodiment, the frame rate calculation unit 2202 calculates the optimum frame rate using the remaining data amount of the transmission buffer 2108 and the quantization step prediction value of the quantization step prediction unit 2201. However, the present invention is not limited to this and may be calculated using either one. Further, the frame rate calculation unit 2202 uses the frame rate table 2403 shown in FIG. 16 above to calculate the optimum frame rate of four stages of 30 Hz, 20 Hz, 15 Hz, and 10 Hz. It is not limited to a value, and it is not necessary to have four stages. Furthermore, there is no need to calculate the optimum frame rate using such a frame rate table. For example, when the free capacity of the transmission buffer is equal to or greater than the threshold C, the optimum frame rate is set to twice the current frame rate and the threshold C If it is less than 1, it may be calculated so as to be ½ times the current frame rate. Also, another threshold value D (C <D) is provided. When the free capacity of the transmission buffer is equal to or greater than the threshold value C, the optimum frame rate is set to twice the current frame rate. You may calculate so that it may be 1/2 times. By doing so, it is possible to prevent the optimum frame rate from being changed each time with respect to a slight change in the remaining amount of data in the transmission buffer in the vicinity of each of the threshold value C and the threshold value D. The optimum frame rate may be obtained by performing the same calculation for changes in the quantization step predicted value. Further, the optimum frame rate is not limited to a value such as 1/2 or 2 times the current frame rate. The same applies to the following embodiments.
[0105]
In the above embodiment, the quantization step prediction unit 2201 calculates the quantization step prediction value using the prediction coefficient table 2303 shown in FIG. 15 described above, but the quantization step prediction of the present invention has been described. The method is not limited to this, and it may be calculated using a predetermined function. Furthermore, the prediction coefficient of the quantization step in the prediction coefficient table 2303 is also set to the values “2”, “1.5”, “1.2”, “1”, “0.8”, “0.5”, and “0.4”. It is not limited. The same applies to the following embodiments.
[0106]
The frame number calculation unit 2204 according to the second embodiment includes a thinning information memory 2601 for storing thinning information for the past one second (that is, 30 frames). However, the present invention is not limited to this. Not. The calculation method of the current frame rate is not limited to the above calculation method, and other methods may be used. The same applies to the following embodiments.
[0107]
Further, instead of the remaining data amount in the transmission buffer, an average bit rate may be used. The same applies to the following embodiments.
[0108]
Hereinafter, frame rate control units 2800, 3000, and 3400 as modifications of the frame rate control unit 2113 will be described. In the following modified example, the description of the same configuration as the frame rate control unit 2113 is omitted, and different points will be mainly described.
[0109]
FIG. 20 is a block diagram illustrating a functional configuration of a frame rate control unit 2800 according to a modification. The frame rate control unit 2800 includes a quantization step prediction unit 2201, a quantization step determination unit 2203, a frame rate calculation unit 2202, a frame number calculation unit 2204, and a comparator 2205. The frame rate control unit 2800 is different from the frame rate control unit 2113 shown in FIG. 14 in that the quantization step prediction value of the quantization step prediction unit 2201 is input to the frame rate calculation unit 2202, instead of the quantization step. This is the point at which the quantization step determined by the determination unit 2203 is input.
[0110]
FIG. 21 is a diagram illustrating an example of each input signal and output signal of the frame rate calculation unit 2202 in FIG. FIG. 21A is a diagram illustrating an optimum frame rate that is an output signal of the frame rate calculation unit 2202. FIG. 21B is a diagram illustrating a quantization step which is one input signal of the frame rate calculation unit 2202. FIG. 21C is a diagram showing the free capacity of the transmission buffer 2108 that is the other input signal of the frame rate calculation unit 2202. The quantization step determination unit 2203 determines an integer value from “1” to “31” based on the quantization step predicted value obtained by the quantization step prediction unit 2201, and uses the determined value as the quantization step. Output. Accordingly, as shown in FIG. 21B, the quantization step input to the frame rate control unit 2800 is “31” at the maximum, unlike the quantization step prediction value.
[0111]
As described above, the frame rate calculation unit 2202 calculates the optimum frame rate based on the quantization step that is an output from the quantization step determination unit 2203 instead of the quantization step prediction value, thereby performing the actual encoding. The optimal frame rate can be calculated without greatly departing from the amount of generated bits.
[0112]
FIG. 22 is a block diagram illustrating a functional configuration of a frame rate control unit 3000 according to a modification. The frame rate control unit 3000 obtains an average quantization step of a frame encoded in the past from the quantization step obtained by the quantization step determination unit 2203, and determines the optimum of the frame rate calculation unit 2202 from the obtained average quantization step. The frame rate is calculated, and includes a quantization step prediction unit 2201, a quantization step determination unit 2203, an average quantization step calculation unit 3001, a frame rate calculation unit 3002, a frame number calculation unit 2204, and a comparator 2205. The average quantization step calculation unit 3001 calculates an average value of quantization steps used for encoding for the past one second. The frame rate calculation unit 3002 calculates the optimum frame rate from the average value of the quantization steps calculated by the average quantization step calculation unit 3001 and the free capacity of the transmission buffer 2108.
[0113]
FIG. 23 is a block diagram for explaining the function of the average quantization step calculation unit 3001 in FIG. The average quantization step calculation unit 3001 calculates the average value of quantization steps for frames encoded in the past, and includes a quantization step memory 3101, a thinning information memory 3102, an adder 3103, and a divider 3104. The quantization step memory 3101 is a FIFO that holds the value of the quantization step, which is the output of the quantization step determination unit 2203, for 30 frames from the current frame for 30 frames in a first-in first-out manner. The thinning-out information memory 3102 outputs the thinning-out information corresponding to each frame output from the comparator 2205 (that is, for example, 30 bits in total when there are 30 frames in one second) in a first-in first-out manner by going back one second from the current frame. FIFO to hold. The adder 3103 adds the results obtained by multiplying the thinning information and the quantization step corresponding to each frame (that is, sums the quantization steps of all frames encoded in the past one second), and divides. To the device 3104. The divider 3104 divides the sum of the quantization steps input from the adder 3103 by the number of frames encoded in the past 1 second, and calculates the average value of the quantization steps.
[0114]
FIG. 24 is a block diagram for explaining the function of the frame rate calculation unit 3002 in FIG. The frame rate calculation unit 3002 holds the frame rate table 2403 shown in FIG. 16, and further includes a hysteresis comparator 3201, a threshold memory 3202, a hysteresis comparator 3203, a threshold memory 3204, and a free capacity calculation unit 2406. The hysteresis comparator 3201 inputs threshold values α1 and β1 (β1 <α1) from the threshold memory 3202 to one input terminal, and when the average quantization step input to the other input terminal exceeds the threshold α1, The comparator outputs D1 = 1 until the quantization step becomes equal to or less than the threshold value β1, and outputs D1 = 0 until the average quantization step exceeds the threshold value β1 and exceeds the threshold value α1. The threshold memory 3202 is a latch circuit or memory that holds the thresholds α1 and β1. The hysteresis comparator 3203 inputs threshold values α2 and β2 (β2 <α2) from the threshold memory 3204 to one input terminal, and D2 = 1 when the free capacity of the transmission buffer 2108 input to the other input terminal exceeds α2. After that, D2 = 1 is output until the free capacity of the transmission buffer 2108 becomes β2 or less. The threshold memory 3204 is a latch circuit or memory that holds the thresholds α2 and β2.
[0115]
FIG. 25 is a diagram illustrating an example of each input signal and output signal of the frame rate calculation unit 3002 in FIG. FIG. 25A is a diagram illustrating an optimum frame rate that is an output signal of the frame rate calculation unit 3002. FIG. 25B is a diagram illustrating an average quantization step which is one input signal of the frame rate calculation unit 3002. FIG. 25 (c) is a diagram showing the free capacity of the transmission buffer 2108, which is the other input signal of the frame rate calculation unit 3002. As shown in FIG. 25B, the average quantization step takes a value from “1” to “31”, but it is not necessarily a natural number like the quantization step of each frame, and the degree of change is also gradual. It is getting smaller. The method by which the frame rate calculation unit 3002 calculates the optimum frame rate based on the values of the output D1 of the hysteresis comparator 3201 and the output D2 of the hysteresis comparator 3203 input in this way is as already described.
[0116]
As described above, the frame rate control unit 3000 controls the frame rate based on the average quantization step that is an output from the average quantization step calculation unit 3001 instead of the quantization step predicted value. It is possible to prevent the frame rate from being changed for each frame when the value fluctuates in the vicinity of the threshold value.
[0117]
In the above embodiment, an example has been described in which the average quantization step calculation unit 3001 calculates the average value of quantization steps in frames encoded in the past one second, but the present invention is not limited to this. Instead, the average value of the quantization step may be calculated for the past several seconds or the past several frames to several tens of frames. Also, the average value of past quantization steps may be calculated uniformly including unencoded frames.
[0118]
FIG. 26 is a block diagram illustrating a functional configuration of a frame rate control unit 3400 according to a modification. The frame rate calculation unit 3401 of the frame rate control unit 3400 includes a hysteresis comparator 3201 and a hysteresis comparator 3203 instead of the comparator 2401 and the comparator 2402, and, as with the frame rate calculation unit 202 in the first embodiment, the optimal frame The rate is calculated. Further, an average value of the optimum frame rates calculated in the past one second is obtained, and the obtained average frame rate is set as the optimum frame rate.
The frame rate control unit 3400 includes a quantization step prediction unit 2201, a quantization step determination unit 2203, a frame rate calculation unit 3401, a frame rate memory 3402, a frame number calculation unit 2204, and a comparator 2205. The frame rate calculation unit 3401 receives the output D1 of the hysteresis comparator 3201 and the output D2 of the hysteresis comparator 3203, and calculates the optimum frame rate according to the rules (truth table) of the frame rate table 2403. Further, the frame rate calculation unit 3401 calculates the average value from the calculation results of a plurality of past optimum frame rates.
[0119]
The frame rate memory 3402 is a FIFO that holds the optimal frame rate, which is the calculation result of the frame rate calculation unit 3401, in a first-in first-out manner for 30 frames that go back one second from the current frame.
[0120]
FIG. 27 is a block diagram showing detailed configurations of the frame rate calculation unit 3401 and the frame rate memory 3402 in FIG. The frame rate calculation unit 3401 holds a frame rate table 2403, and further includes a hysteresis comparator 3201, a threshold memory 3202, a hysteresis comparator 3203, a threshold memory 3204, a free space calculation unit 2406, an adder 3501, and a divider 3502. . The frame rate calculation unit 3401 receives the output D1 of the hysteresis comparator 3201 to which the quantization step prediction value is input from the quantization step prediction unit 2201 and the remaining data amount of the transmission buffer 2108 for each frame of the input image signal. The optimum frame rate is calculated from the frame rate table 2403 based on the output D2 of the hysteresis comparator 3203, and the calculated optimum frame rate is output to the frame rate memory 3402 at 30 Hz. An adder 3501 adds each optimum frame rate in the frame rate memory 3402. The divider 3502 divides the value output from the adder 3501 by “30”, refers to the frame rate table 2403, and selects the frame rate closest to the division result from the four optimum frame rates as the average optimum frame. Determine and output as a rate. As a result, even when the optimum frame rate changes, for example, from 30 Hz to 10 Hz, the average optimum frame rate always changes sequentially via an intermediate frame rate, such as 30 Hz → 20 Hz → 15 Hz → 10 Hz. Therefore, the frame rate control in encoding can be performed more smoothly.
[0121]
FIG. 28 is a diagram illustrating an example of each input signal and output signal of the frame rate calculation unit 3401 in FIG. FIG. 28A is a diagram showing an average optimum frame rate that is an output signal of the frame rate calculation unit 3401. FIG. 28B is a diagram illustrating a quantization step prediction value that is one input signal of the frame rate calculation unit 3401. FIG. 28C is a diagram showing the free capacity of the transmission buffer 2108 that is the other input signal of the frame rate calculation unit 3401. As shown in FIG. 28 (a), the average optimum frame rate is obtained by averaging the optimum frame rates of 30 frames going back from the current frame for one second and selecting the optimum frame rate closest to the value. A large change in the frame rate is suppressed, and as a result, an encoding result with smoother motion can be obtained.
[0122]
Note that although the present frame rate calculation unit 3401 has been described as calculating the average value of the optimum frame rate for 30 frames in the past 1 second, the present invention is not limited to this, and the range of the optimum frame rate for obtaining the average value is It may be the number of seconds or the number of frames in the past.
[0123]
(Embodiment 3)
FIG. 29 is a block diagram illustrating a functional configuration of an image encoding device 4100 according to the third embodiment. The image encoding device 4100 uses the variable bit rate method in the image encoding device 10 of the first embodiment and the variable frame rate method in the image encoding device 2100 of the second embodiment to perform more appropriate encoding. Is realized.
[0124]
As shown in FIG. 29, image coding apparatus 4100 has the same functional configuration as image coding apparatus 10 in Embodiment 1 above, except for transmission buffer 2108, frame rate control section 4113, and frame thinning section 4116. . In the following, the same components as those in the first embodiment or the second embodiment are denoted by the same reference numerals, and the description thereof is omitted.
[0125]
The frame rate control unit 4113 has only a function of generating decimation information among the functions of the frame rate control unit 2113 of the image coding apparatus 2100 of the second embodiment.
[0126]
The frame decimation unit 4116 receives the decimation information from the frame rate control unit 4113, determines the frame rate, and notifies the bit rate control unit 30 of the determined frame rate value at that time.
[0127]
FIG. 30 is a block diagram showing a detailed functional configuration of the frame rate control unit 4113 in FIG. As shown in FIG. 30, the frame rate control unit 4113 has a configuration in which the quantization step prediction unit 2201 and the quantization step determination unit 2203 are excluded from the frame rate control unit 2113 (see FIG. 14) in the second embodiment. It has become. Therefore, the frame rate control unit 4113 generates only thinning information based on the remaining data amount obtained from the transmission buffer 2108 and transmits this thinning information to the frame thinning unit 4116.
[0128]
As described above, the image encoding device 4100 according to the present embodiment controls the frame rate according to the remaining data amount of the transmission buffer, and based on the frame rate and the total number of bits allocated to the frame group, It is possible to realize more preferable bit allocation.
[0129]
(Embodiment 4)
In the following embodiment, the configuration of the image encoding device in the above first to third embodiments is realized as a step of an image encoding program, and functions equivalent to those of the image encoding device are realized on a general computer system. We will explain how to do this.
[0130]
FIG. 31 is an explanatory diagram of a case where a function equivalent to that of the image encoding apparatuses according to the first to third embodiments is realized on a general computer system using a flexible disk storing an image encoding program.
[0131]
FIG. 31B shows an appearance, a cross-sectional structure, and a flexible disk as viewed from the front of the flexible disk, and FIG. 31A shows an example of a physical format of the flexible disk that is a recording medium body.
[0132]
The flexible disk 1301 is built in a case 1302, and a plurality of tracks are formed concentrically on the surface of the disk from the outer periphery toward the inner periphery, and each track is divided into 16 sectors in the angular direction. Therefore, in the flexible disk storing the image encoding program, data as the image encoding program is recorded in an area allocated on the flexible disk 1301.
[0133]
FIG. 31C shows the configuration of an apparatus for recording and reproducing the image encoding program on the flexible disk 1301. When the image encoding program is recorded on the flexible disk 1301, the image encoding program data is written from the computer system 1304 via the flexible disk drive 1303. When the encoding or decoding device is constructed in the computer system by the program in the flexible disk 1301, the program is read from the flexible disk drive 1303 and transferred to the computer system 1304.
[0134]
In the above description, a flexible disk is used as the data recording medium. However, the above-described image encoding apparatus is similarly realized by using a medium capable of recording a program such as an optical disk, an IC card, and a ROM cassette. be able to.
[0135]
【The invention's effect】
As described above, according to the image encoding device of the present invention, when encoding is performed at a constant bit rate for a predetermined time length, the deviation in the number of generated bits in a frame group that is a collection of frames is reduced. Each time a frame group is formed and each frame in the frame group is encoded, the number of bits to be allocated to the next frame is calculated. Furthermore, by referring to the number of generated bits in the past frame, it is possible to derive the optimum number of allocated bits that can be encoded with high image quality for the next frame. That is, even when scenes that are difficult to encode are continuous, it is possible to avoid an extremely small number of bits allocated to subsequent scenes, and to improve the image quality as a whole.
[0136]
In addition, according to the image encoding device of the present invention, even when the rate of a frame to be encoded fluctuates (variable frame rate), the optimum frame rate using the current frame rate is used each time encoding is performed. Since the number of bits can be derived, it is possible to perform encoding capable of avoiding extreme deterioration in image quality while keeping the average bit rate in a predetermined time length constant.
[0137]
In addition, the bitstream encoded by the image encoding apparatus according to the present invention can be decoded by a general decoding apparatus, and can be decoded with high image quality without performing any special processing.
[0138]
Further, according to the image encoding device of the present invention, as a result of encoding the current frame, first, by adjusting the quantization resolution and adjusting the signal resolution of each pixel according to the amount of code generated. By controlling the generated code amount of the next frame and adjusting the generated code amount by adjusting the quantization width, the encoding frame rate can be adjusted. As a result, for example, by increasing the quantization width, it is possible to prevent the occurrence of a state in which frame thinning occurs even though the amount of generated code is reduced and the image quality deteriorates more than necessary. There is an effect that it is possible to control the generated code amount by encoding efficiently.
[0139]
In addition, the image encoding apparatus according to the present invention calculates the quantization width of the next encoded frame from the remaining code amount of the transmission buffer, for example, when there is a margin in the transmission buffer (that is, encoding is easy). For scenes), the quantization width is set to a small value, and conversely, if there is not enough room in the transmission buffer, the quantization width is set to a large value so that the bit rate for a sufficiently long time is kept constant and the image quality is as high as possible There is an effect that can be made.
[0140]
Furthermore, the image coding apparatus according to the present invention sets the coding frame rate when the predicted value of the calculated quantization width becomes a large value equal to or larger than the threshold (that is, in the case of a complicated scene that is difficult to code). Change to a smaller value to increase the frame decimation rate. However, since the amount of generated bits is suppressed by controlling the quantization width, there is an effect that high-quality encoding can be performed without causing a sudden change in the frame rate.
[0141]
Further, according to the image coding apparatus according to the present invention, the coding frame rate is controlled by using the average value of the quantization width controlled and fluctuated by the residual code amount of the transmission buffer, so that the coding of the current frame can be performed. When the code amount is extremely generated (the residual code amount of the transmission buffer is increased) and the quantization step of the next frame is largely controlled, it is suppressed that the frame rate is immediately changed. It is possible to prevent the movement from becoming awkward. At the same time, it is possible to suppress an abrupt change in the frame rate and to perform smooth motion encoding.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a functional configuration of an image encoding device according to a first embodiment.
2 is a block diagram showing a detailed functional configuration of a bit rate control unit in FIG. 1; FIG.
FIG. 3 is a diagram illustrating characteristics of the number of generated bits for each picture type in general image coding.
4 is a specific example of a frame group defined in FIG.
FIG. 5 is a modification of the frame group defined in FIG.
6A is a diagram for explaining an effect obtained by configuring a frame group with two frame sets as shown in FIG. 5; FIG.
(B) is a figure for demonstrating the effect when a frame group is comprised by two frame sets like the said FIG. 5, and the frame rate of encoding changes.
7 is a flowchart showing a flow of processing in the overall control unit and the bit rate control unit of the image coding apparatus according to Embodiment 1. FIG.
8A is a diagram illustrating a configuration example of a frame group when encoding is performed using the image encoding device according to Embodiment 1. FIG.
(B) is an example which shows the change of the bit rate when encoding is performed using the image encoding apparatus in Embodiment 1. FIG.
FIG. 9A is an example showing a state of each frame when one frame group is constituted by two frame sets.
(B) is an example showing how the bit rate fluctuates when one frame group is composed of two frame sets.
FIG. 10A is an example showing a state of each frame when one frame group is constituted by two frame sets and the encoding frame rate is changed.
(B) is an example showing how the bit rate fluctuates when one frame group is composed of two frame sets and the encoding frame rate changes.
FIG. 11 is a diagram illustrating a specific example of a method for calculating the number of bits in an arithmetic unit.
12 is a block diagram showing a part of a functional configuration of an image coding apparatus configured to obtain a frame rate value Rf from other departments without providing a frame thinning unit in FIG. 1;
FIG. 13 is a block diagram showing a functional configuration of an image coding apparatus according to Embodiment 2.
14 is a block diagram showing a detailed functional configuration of a rate control unit in FIG. 13;
15 is a block diagram for explaining a function of a quantization step prediction unit in FIG. 14;
16 is a block diagram for explaining a function of a frame rate calculation unit in FIG. 14;
17 is a diagram showing each input signal and output signal of the frame rate calculation unit in FIG. 16;
18 is a diagram for explaining the operation of a frame number calculation unit in FIG. 14;
FIG. 19 is a diagram illustrating an input image signal and an encoded frame of the image encoding device.
FIG. 20 is a block diagram illustrating a functional configuration of a frame rate control unit according to a modification.
FIG. 21 is a diagram showing each input signal and output signal of the frame rate calculation unit in FIG. 20;
FIG. 22 is a block diagram illustrating a functional configuration of a frame rate control unit according to a modified example.
23 is a block diagram showing a configuration of an average quantization step calculation unit in FIG.
24 is a block diagram for explaining a function of a frame rate calculation unit in FIG. 22. FIG.
25 is a diagram showing each input signal and output signal of the frame rate calculation unit in FIG. 22. FIG.
FIG. 26 is a block diagram illustrating a functional configuration of a frame rate control unit according to a modification.
27 is a block diagram for explaining functions of a frame rate calculation unit and a frame rate memory in FIG. 26. FIG.
FIG. 28 is a diagram showing each input signal and output signal of the frame rate calculation unit in FIG.
29 is a block diagram illustrating a functional configuration of an image encoding device according to Embodiment 3. FIG.
30 is a block diagram illustrating a functional configuration of a frame rate control unit in the image encoding device of FIG. 29. FIG.
FIG. 31 is a diagram for explaining a storage medium that can store a program for realizing the image coding apparatus according to the first to third embodiments by a general computer system.
FIG. 32 is a block diagram illustrating a functional configuration of an image encoding device using a conventional variable bit rate method.
FIG. 33 is a block diagram illustrating a functional configuration of a bit rate control unit in a conventional image encoding device.
FIG. 34 is a flowchart showing the flow of processing in the overall control unit and bit rate control unit of the conventional image encoding device.
FIG. 35 is a diagram showing a specific example of a method for calculating the number of allocated bits in a conventional bit rate control unit.
FIG. 36A is a configuration example of a frame group for explaining the first problem of the conventional technology.
(B) is an example of a bit rate for explaining the first problem of the prior art.
FIG. 37A is a configuration example of a frame group for explaining a second problem of the conventional technique.
(B) is an example of a bit rate for explaining the second problem of the prior art.
[Explanation of symbols]
10 Image encoding device
20 Frame thinning part
30 Rate control unit
31 Frame group target bit number deriving unit
32nd frame target bit number derivation unit
33 Average frame bit number derivation unit
34 Calculation unit
35 Quantization step derivation unit
36 Memory unit
40 Overall control unit
50 Image encoding device
60 Overall control unit
100 Image coding apparatus
101 Encoded frame signal
102 frame memory
103 detector
104 Compensator
105 Orthogonal transformation unit
106 Quantization unit
107 Variable length encoding unit
108 Inverse quantization unit
109 Inverse orthogonal transform unit
110 Bit rate control unit
111 Frame Group Target Bit Number Deriving Unit
112 Next frame target bit number derivation unit
113 Quantization step derivation unit
121 Encoded frame signal
140 Overall control unit
141 Quantization step
201 Quantization step prediction unit
202 Frame rate calculator
203 Quantization step determination unit
204 Frame number calculator
205 comparator
302 Current value memory
303 Prediction coefficient table
401 Comparator
406 Capacity calculation unit
1301 Flexible disk
1302 case
1303 Flexible disk drive
1304 Computer system
2100 Image coding apparatus
2101 Frame thinning part
2108 Transmission buffer
2109 Inverse quantization unit
2110 Inverse orthogonal transform unit
2111 Differencer
2112 Adder
2113 Frame rate control unit
2201 Quantization step prediction unit
2202 Frame rate calculator
2203 Quantization step determination unit
2204 Number of frames calculator
2205 comparator
2302 Current value memory
2303 Prediction coefficient table
2401 Comparator
2402 Comparator
2403 Frame rate table
2404 Threshold memory
2405 Threshold memory
2406 Capacity calculation unit
2601 Information memory
2602 counter
2603 calculator
2604 frames
2800 Frame rate controller
3000 Frame rate control unit
3001 Average quantization step calculator
3002 Frame rate calculator
3101 Quantization step memory
3102 Information memory
3103 Adder
3104 Divider
3201 Hysteresis comparator
3202 Threshold memory
3203 Hysteresis Comparator
3204 Threshold memory
3400 Frame rate control unit
3401 Frame rate calculator
3402 frame rate memory
3501 Adder
3502 Divider
4100 Image coding apparatus
4113 Frame rate control unit
4116 Frame thinning part

Claims (8)

An image encoding device that encodes image signals sequentially input in units of frames,
Encoding means for quantizing the frequency component of the image signal for each frame and encoding based on the quantized result;
And the total number of bits specifying means for specifying the total number of bits Ru assignment in a frame group including a plurality of frames,
Frame number specifying means for specifying the number of frames not yet encoded in the frame group based on the frame rate and the number of frames already encoded in the frame group;
Target bit number calculating means for calculating a target value of the number of bits to be allocated to the next frame to be encoded based on the specified total number of bits and the specified number of frames;
Quantization step deriving means for deriving a quantization step related to a frame to be encoded next using the calculated target number of bits;
A transmission buffer for holding the encoded data encoded by the encoding means and outputting the same to the outside by a fixed amount;
Frame rate control means for determining a frame rate indicating a period of a frame to be encoded next based on a data amount of encoded data remaining in the transmission buffer and the quantization step;
Based on the frame rate determined by the frame rate control means, the input control means for controlling the discarding of the image signal input to the encoding means ,
The target bit number calculation means is the number of frames that have not yet been encoded in the frame group calculated from the frame rate determined by the frame rate control means with the total bit number specified by the total bit number specification means. An image encoding apparatus characterized by calculating a target value of the number of bits by dividing the image. The image encoding device further includes:
An average value calculating means for calculating an average value of the number of generated bits based on the already encoded frame rate in the frame group and the number of bits generated thereby ;
The target value of the new number of bits obtained by multiplying the calculated average value of the number of generated bits by a predetermined coefficient is compared with the calculated target number of bits. A calculation unit that compares the lower limit value set in advance with the basic target value and sets the larger one as the target value for the new number of bits.
The image coding apparatus according to claim 1, wherein the quantization step deriving unit derives the quantization step using a target value of the number of bits specified by the computing unit. The total bit number specifying means includes:
The first frame of the frame group is a frame to be encoded in the screen, and a total bit number initialization unit that initializes the number of bits that can be allocated as the entire frame group ;
The image coding apparatus according to claim 2, further comprising: a total bit number updating unit that updates the total number of bits by subtracting the number of bits actually generated by encoding from the number of bits that can be allocated as a whole. The frame group is a frame set that consists of a plurality of frames is two or more combined configuration, when completing the encoding for one frame set, the frameset that has not yet been coded belonging to the group of frames , New frame sets are added sequentially to form a new frame group,
The total bit number specifying means further includes:
Adding the difference between the number of bits that have been distributed to each frame set obtained, the number of bits actually frameset number of bits generated is obtained coding is completed from the total number of bits allocated to the frame group And calculating the total number of bits of the new frame group and the number of assignable bits in each frame set,
The image encoding apparatus according to claim 1, wherein: The frame rate control means includes
Information obtained by calculating the free space of the buffer from the amount of encoded data remaining in the transmission buffer and comparing it with a predetermined threshold; and the quantization step obtained by the quantization step deriving means; The period of the next frame to be encoded is determined using information obtained by comparing with a predetermined threshold.
The image encoding apparatus according to claim 1, wherein: An image encoding method for encoding image signals sequentially input in units of frames,
An encoding step of quantizing the frequency component of the image signal for each frame and encoding based on the quantized result;
And the total number of bits specifying step of specifying the total number of bits Ru assignment in a frame group including a plurality of frames,
A number-of-frames identifying step for identifying a number of unencoded frames in the frame group based on the accepted frame rate and the number of frames already encoded in the frame group;
Based on said number of frames which have not yet been coded out of the total number of total bits identified by a specific number Step bits and frame group, the target number of bits for calculating the target value of the bit number allocated to a frame to be coded next A calculation step;
Using the calculated target number of bits, a quantization step derivation step for deriving a quantization step related to a frame to be encoded next;
The encoded data encoded by the encoding step is held, and then encoded based on the amount of encoded data remaining in the transmission buffer to be output to the outside by a predetermined amount and the quantization step. A frame rate control step for determining a frame period;
An input control step for controlling discarding of an image signal input to the encoding step based on the frame rate determined in the frame rate control step;
In the target bit number calculating step, the total bit number specified in the total bit number specifying step is the number of frames that are not yet encoded among the frame groups calculated from the frame rate determined in the frame rate control step. And a target value of the number of bits is calculated by dividing the image. A program for an image encoding device that encodes an image signal input in units of frames,
An encoding step of quantizing the frequency component of the image signal for each frame and encoding based on the quantized result;
And the total number of bits specifying step of specifying the total number of bits Ru assignment in a frame group including a plurality of frames,
A number-of-frames identifying step for identifying a number of unencoded frames in the frame group based on the accepted frame rate and the number of frames already encoded in the frame group;
Target bit number for calculating a target value of the number of bits to be allocated to the next frame to be encoded based on the total bit number specified in the total bit number specifying step and the number of frames not yet encoded in the frame group A calculation step;
Using the target value of the number of the calculated bit, the quantization step derivation step of deriving the quantization step according to the frame to be coded next,
The encoded data encoded by the encoding step is held, and then encoded based on the amount of encoded data remaining in the transmission buffer to be output to the outside by a predetermined amount and the quantization step. A frame rate control step for determining a frame period;
Based on the frame rate determined in the frame rate control step, the image encoding device executes the input control step for controlling the discarding of the image signal input to the encoding step,
In the target bit number calculating step, the total bit number specified in the total bit number specifying step is the number of frames that are not yet encoded among the frame group calculated from the frame rate determined in the frame rate control step. A program for calculating a target value of the number of bits by dividing . A computer-readable recording medium on which a program for an image encoding device for encoding image signals sequentially input in units of frames is recorded,
The program is
An encoding step of quantizing the frequency component of the image signal for each frame and encoding based on the quantized result;
And the total number of bits specifying step of specifying the total number of bits Ru assignment in a frame group including a plurality of frames,
Target bit number for calculating a target value of the number of bits to be allocated to the next frame to be encoded based on the total bit number specified in the total bit number specifying step and the number of frames not yet encoded in the frame group A calculation step;
Using the target value of the number of the calculated bit, the quantization step derivation step of deriving the quantization step according to the frame to be coded next,
The encoded data encoded by the encoding step is held, and then encoded based on the amount of encoded data remaining in the transmission buffer to be output to the outside by a predetermined amount and the quantization step. A frame rate control step for determining a frame period;
Based on the frame rate determined in the frame rate control step, the image encoding device executes the input control step for controlling the discarding of the image signal input to the encoding step,
In the target bit number calculating step, the total bit number specified in the total bit number specifying step is the number of frames that are not yet encoded among the frame group calculated from the frame rate determined in the frame rate control step. The target value of the number of bits is calculated by dividing the recording medium.

Images