US20120224779A1 - Image encoding device, method for encoding image, and computer-readable storage medium storing image encoding program - Google Patents
Image encoding device, method for encoding image, and computer-readable storage medium storing image encoding program Download PDFInfo
- Publication number
- US20120224779A1 US20120224779A1 US13/410,874 US201213410874A US2012224779A1 US 20120224779 A1 US20120224779 A1 US 20120224779A1 US 201213410874 A US201213410874 A US 201213410874A US 2012224779 A1 US2012224779 A1 US 2012224779A1
- Authority
- US
- United States
- Prior art keywords
- codes
- image
- data
- encoded
- variable
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/174—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a slice, e.g. a line of blocks or a group of blocks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
- H04N19/149—Data rate or code amount at the encoder output by estimating the code amount by means of a model, e.g. mathematical model or statistical model
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
Definitions
- the embodiments discussed herein are related to an image encoding device that encodes a moving image, a method for encoding an image, and a computer-readable storage medium storing an image encoding program.
- the “high-efficiency encoding” is an encoding process in which a certain data string is converted into another data string, thereby reducing the amount of data.
- Typical methods for encoding a moving image include International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group (MPEG)-2/MPEG-4 (hereinafter referred to as MPEG-2 and MPEG-4).
- ISO/IEC International Organization for Standardization/International Electrotechnical Commission
- MPEG-2 and MPEG-4 Moving Picture Experts Group
- An I-picture is a picture obtained by performing an encoding process closed within a frame without referring to an encoded image of another picture.
- a P-picture is a picture obtained by encoding a prediction error by performing forward prediction from a past picture.
- a B-picture is a picture obtained by encoding a prediction error by performing bidirectional prediction from past and future pictures.
- FIG. 1 is a diagram illustrating an example of a sequence having an IBBP structure. Numbers following Is, Ps, and Bs illustrated in FIG. 1 indicate the display order. In the example illustrated in FIG. 1 , the display order is B 0 , B 1 , I 2 , B 3 , B 4 , P 5 , and so on.
- the encoding order is I 2 , B 0 , B 1 , P 5 , B 2 , B 3 , and so on.
- a delay is caused by B-pictures.
- Encoded data is accumulated in a buffer and output to a decoding device. Since intra-frame encoding is performed for I-pictures, the entirety of the pictures is refreshed. Therefore, propagation of errors may be suppressed by the I-pictures.
- a pre-processing unit for example, a pre-processing unit, a predictive coding unit, a variable-length coding unit, and a multiplexing unit perform sequential processing (pipeline processing) for each picture.
- the pre-processing unit includes a picture rearranging unit that rearranges pictures in accordance with the picture types, and sequentially outputs frame images for each picture type and frame or the like.
- the predictive coding unit a difference from a predicted image is obtained and orthogonal transformation and quantization are performed.
- variable-length coding unit entropy coding is performed.
- the multiplexing unit data subjected to the entropy coding is multiplexed and a bit stream to be output is generated.
- FIG. 2 is a diagram illustrating an example of the sequential processing for each picture.
- each process is sequentially performed for each picture.
- a multiplexed bit stream is temporarily stored in a Coded Picture Buffer (CPB) and output to a decoder.
- CPB Coded Picture Buffer
- the CPB is a buffer that holds a bit stream before the bit stream is input to the decoder.
- the encoder controls the CPB such that a buffer on the decoder side does not fail. In a real-time processing system, delay caused by the CPB is the largest delay.
- the size of the CPB is increased, the quality of images improves, but the delay becomes larger. If the size of the CPB is decreased, the quality of images deteriorates, but the delay may be reduced. There is a trade-off between the size of the CPB and the quality of images. How the buffer delay may be reduced while maintaining the quality of images is being studied.
- CPB is the name of a buffer according to H.264/Advanced Video Coding (AVC) (hereinafter also referred to as H.264).
- AVC Advanced Video Coding
- a CPB corresponds to a Video Buffer Verifier (VBV).
- VBV Video Buffer Verifier
- a CPB is used, but in the case of MPEG, the CPB may be replaced with a VBV.
- a method has been disclosed in which, in order to reduce the delay in the real-time processing, a picture is divided into several slices and the slices are subjected to sequential processing.
- a slice the number of macroblocks in a slice
- the size of a slice is determined on the basis of the magnitude of a motion vector, the circuit capacity, and the number of codes written to the buffer.
- an image encoding device that encodes an image by performing sequential processing in units of a certain size
- the image encoding device includes, a variable-length coding unit that performs variable-length coding on data to be encoded having the certain size in the image; an estimating unit that estimates the number of codes at a time after the variable-length coding using the data to be encoded at a time before the variable-length coding is performed by the variable-length coding unit; and a slice controlling unit that controls division of the image into slices on the basis of a cumulative value of the estimated number of codes.
- FIG. 1 is a diagram illustrating an example of a sequence having an IBBP structure
- FIG. 2 is a diagram illustrating an example of sequential processing for each picture
- FIG. 3 is a diagram illustrating an example of sequential processing at a time when a picture is divided into four slices
- FIG. 4 is a diagram illustrating a relationship between the number of bits and encoder delay in a slice
- FIG. 5 is a diagram illustrating a relationship between sequential processing for each macroblock and the encoder delay
- FIGS. 6A and 6B are diagrams illustrating relationships between the number of bits generated and division into slices
- FIG. 7 is a diagram illustrating a problem (first problem) caused by the division into slices in the related art
- FIG. 8 is a diagram illustrating a problem (second problem) caused by the division into slices in the related art
- FIG. 9 is a block diagram illustrating an example of the configuration of an image encoding device according to a first embodiment
- FIG. 10 is a diagram illustrating division into slices according to the first embodiment
- FIG. 11 is a flowchart illustrating an example of an encoding process according to the first embodiment
- FIG. 12 is a flowchart illustrating an example of a process for controlling the division into slices according to the first embodiment
- FIG. 13 is a block diagram illustrating an example of the configuration of an image encoding device according to a second embodiment
- FIG. 14 is a diagram illustrating an example in which a cumulative value of the expected number of codes is corrected
- FIG. 15 is a flowchart illustrating an example of a process for controlling the division into slices according to the second embodiment
- FIG. 16 is a diagram illustrating an example of CPB modes
- FIG. 17 is a diagram illustrating examples of the number of divisions and a picture structure
- FIGS. 18A and 18B are diagrams illustrating other examples of the CPB modes.
- FIG. 19 is a diagram illustrating an example of the configuration of an image encoding device.
- FIG. 3 is a diagram illustrating an example of sequential processing at a time when a picture is divided into four slices. As illustrated in FIG. 3 , by performing the sequential processing for each slice, the delay may be smaller than that occurring in the sequential processing for each picture.
- FIG. 4 is a diagram illustrating a relationship between the number of bits and the encoder delay in a slice. As illustrated in FIG. 4 , almost all the number of bits for one picture may be generated in a certain slice. In the example illustrated in FIG. 4 , the number of bits almost equivalent to one picture is generated in slice 1 .
- the CPB has substantially the same size as a CPB at the time of the sequential processing for each picture. If the size of the CPB is reduced, the capacity of the CPB becomes insufficient, thereby losing a part of data. If a part of the data is lost, an error stream is caused, thereby making it difficult to normally decode the data.
- FIG. 5 is a diagram illustrating a relationship between the sequential processing for each macroblock and the encoder delay. As illustrated in FIG. 5 , in the sequential processing for each macroblock, although it is possible to further reduce the delay than in the case of the sequential processing for each slice, it is difficult to reduce the size of the CPB.
- the reason why it is difficult to reduce the size of the CPB is that, as described above, almost all the number of bits for one picture may be generated in one slice, and therefore the size of the CPB becomes the same as that of a CPB when the sequential processing is performed for each picture.
- the size of the CPB is the same as in the case of the sequential processing for each picture, the delay caused by the CPB is also the same.
- a method may be used in which the division into slices is performed on the basis of the number of bits generated as disclosed in the related art.
- the size of the CPB may be reduced by equalizing the size of the CPB.
- FIGS. 6A and 6B are diagrams illustrating relationships between the number of bits generated and the division into slices.
- the number of bits (the number of codes) generated in each slice is different.
- the number of bits generated is equalized, the size of each slice varies.
- the numbers of codes generated in slice 1 to slice 4 are the same, it is possible to reduce the size of the CPB.
- FIG. 7 is a diagram illustrating a problem (first problem) caused by division into slices in the related art.
- the sequential processing is performed for each macroblock (MB) and the number of codes generated is counted.
- the division into slices is performed such that a next MB belongs to a new slice.
- the slice structured mode of Annex K is not available.
- the image region of another slice is referred to in order to predict a motion vector, which is a violation of the standard.
- the predictive coding is performed again and data in the predictive coding unit is corrected.
- cost undesirably increases when the predictive coding is performed again.
- FIG. 8 is a diagram illustrating a problem (second problem) caused by the division into slices in the related art.
- the threshold value is assumed to be exceeded when the number of codes generated in MB 2 has been added.
- the division is performed such that MB 1 and MB 2 belong to different slices.
- MB 2 which has been subjected to the variable-length coding, is discarded and the process is begun again from MB 2 .
- a mechanism that holds previous MB information is provided and data that has already been written to the CPB is discarded.
- a function of correcting a head MB is provided in the pre-processing unit or the predictive coding unit.
- FIG. 9 is a block diagram illustrating an example of the configuration of an image encoding device 10 according to a first embodiment.
- the image encoding device 10 illustrated in FIG. 9 has a pre-processing unit 101 , a predictive coding unit 117 , a variable-length coding unit 114 , a multiplexing unit 115 , and a CPB 116 .
- the predictive coding unit 117 has a prediction error signal generating section 102 , a Discrete Cosine Transform (DCT) section 103 , a quantization section 104 , an inverse quantization section 105 , an inverse DCT section 106 , a decoded image generating section 107 , a frame buffer 108 , a motion estimating section 109 , a motion compensating section 110 , a rate controlling section 111 , a number of codes estimating section 112 , and a slice controlling section 113 .
- DCT Discrete Cosine Transform
- the pre-processing unit 101 the predictive coding unit 117 , the variable-length coding unit 114 , the multiplexing unit 115 , and the CPB 116 are subjected to the sequential processing for each MB.
- the pre-processing unit 101 includes a picture rearranging section that rearranges pictures in accordance with the picture types, and sequentially outputs frame images for each picture type and frame or the like. In addition, the pre-processing unit 101 may generate a reduced image and perform a process for investigating the direction of motion or the like.
- the prediction error signal generating section 102 obtains macroblock data (hereinafter also referred to as “MB data”), in which an image to be encoded included in input moving image data has been divided into 16 ⁇ 16 pixel blocks (MBs).
- the prediction error signal generating section 102 generates a prediction error signal on the basis of the MB data and MB data regarding a predicted image output from the motion compensating section 110 .
- the prediction error signal generating section 102 outputs the generated prediction error signal to the DCT section 103 .
- the DCT section 103 performs a DCT on the input prediction error signal in units of 8 ⁇ 8 or 4 ⁇ 4.
- an orthogonal transform such as a Hadamard transform may be performed.
- the DCT section 103 obtains data that has been divided into horizontal and vertical frequency components through the process of the orthogonal transform.
- the reason why the data is transformed into the frequency components is that most pieces of the data may be included in low-frequency components due to a spatial correlation of the image, thereby making it possible to reduce the amount of information.
- the quantization section 104 reduces the number of codes in the data (DCT coefficients) that has been subjected to the orthogonal transform by quantizing the data, and outputs the quantized data to the variable-length coding unit 114 and the inverse quantization section 105 .
- the quantization section 104 may output the quantized value to the number of codes estimating section 112 depending on the situation.
- the inverse quantization section 105 performs inverse quantization on the data output from the quantization section 104 .
- the inverse quantization section 105 outputs the data that has been subjected to the inverse quantization to the inverse DCT section 106 , the number of codes estimating section 112 , and the rate controlling section 111 .
- the inverse DCT section 106 performs an inverse DCT on the data that has been subjected to the inverse quantization in order to transform the frequency components into pixel components, and outputs the data at a time after the transform to the decoded image generating section 107 and the rate controlling section 111 .
- the decoded image generating section 107 adds MB data regarding an image that has been subjected to motion compensation in the motion compensating section 110 and the prediction error signal that has been subjected to the decoding process in the inverse DCT section 106 .
- a processed image that is the same as one on the decoding side may be generated on the encoding side.
- the image generated on the encoding side is called a “locally decoded image”.
- the decoded image generating section 107 outputs, to the frame buffer 108 , MB data regarding the locally decoded image that has been generated by adding the MB data and the prediction error signal.
- the MB data regarding the locally decoded image may be subjected to deblocking filtering.
- the locally decoded image may be a reference image.
- the frame buffer 108 stores the input MB data as data regarding a new reference image and outputs the data to the motion estimating section 109 .
- the motion estimating section 109 performs motion detection using the MB data regarding the image to be encoded and the MB data regarding the encoded reference image that has been obtained from the frame buffer 108 , in order to obtain an appropriate motion vector.
- a motion vector is a value that indicates a spatial deviation for each block obtained by using a block matching technique by which a position of the reference image that is most similar to the image to be encoded is detected for each block.
- the motion estimating section 109 obtains a difference vector and outputs an evaluation value corresponding to the code length of the motion vector in accordance with the size of the component.
- the motion estimating section 109 outputs the detected motion vector to the motion compensating section 110 and the number of codes estimating section 112 .
- the motion compensating section 110 performs motion compensation on the data regarding the reference image using the motion vector provided from the motion estimating section 109 .
- MB data as the reference image that has been subjected to the motion compensation is generated.
- the rate controlling section 111 controls a quantization scale value (qP value) on the basis of the obtained data, in order to control the bit rate.
- qP value quantization scale value
- the number of codes estimating section 112 estimates the number of codes in each encoding process unit (for example, an MB or the like) at a time after the variable-length coding using the data to be encoded at a time before the variable-length coding.
- Various methods relating to the estimation of the number of codes will be described hereinafter.
- the number of codes estimating section 112 estimates the number of codes using the DCT coefficients output from the inverse quantization section 105 .
- the number of codes estimating section 112 estimates the number of codes by using the rate-distortion function of AC coefficients included in the DCT coefficients and by determining, as the number of codes in an MB, the sum of the number of bits B ij that may be obtained as the square of each AC coefficient using the following expression (1).
- the rate-distortion function D may be represented by an expression (2) when the distribution of the AC coefficients is assumed to be the Gaussian distribution.
- the variation is originally represented by the square of ⁇ , but the number of codes estimating section 112 estimates that the square of each AC coefficient is a value close to the square of ⁇ , and substitutes each AC coefficient for ⁇ and a certain value for D in order to calculates B ij .
- the number of codes estimating section 112 outputs the value obtained by the expression (1) to the slice controlling section 113 .
- the number of codes estimating section 112 estimates the number of codes in an MB using the magnitude of a motion vector obtained from the motion estimating section 109 . Since the motion vector, too, is included in the data to be encoded, when the number of codes in other pieces of data is assumed to be the same, the number of codes in the MB may be estimated on the basis of the magnitude of the motion vector.
- the number of codes estimating section 112 estimates the number of codes using an estimation function F(x).
- the number of codes estimating section 112 obtains F(x) using any of the expressions (4) to (6) and estimates the value of F(x) to be the number of codes.
- the number of codes estimating section 112 outputs the estimated number of codes to the slice controlling section 113 .
- the number of codes estimating section 112 estimates the number of codes using the “level” and the “run” of a quantized value.
- the number of codes estimating section 112 estimates the number of codes on the basis of an estimation function G(y) using the “run” and the “level” of a DCT coefficient after quantization. For example, the number of codes estimating section 112 estimates “G(run)+G(level)” to be the number of codes.
- the number of codes estimating section 112 may estimate the number of codes using data at a time before the variable-length coding.
- the number of codes estimating section 112 may estimate the number of codes in an MB by combining the above-described numbers of codes.
- the number of codes estimating section 112 may obtain the number of codes by, for example, adding Bit obtained using the expression (1) and F(x) obtained using any of the expressions (4) to (6). Alternatively, the number of codes estimating section 112 may obtain the number of codes by, for example, adding F(x) obtained using any of the expressions (4) to (6) and G(y) obtained using any of the expression (7) to (9). Alternatively, the number of codes estimating section 112 may obtain the number of codes by, for example, adding Bit obtained using the expression (1) and G(y) obtained using any of the expressions (7) to (9). The estimated number of codes is called the “expected number of codes”.
- the slice controlling section 113 accumulates the number of codes obtained from the number of codes estimating section 112 and judges whether or not a cumulative value exceeds a threshold value. When the cumulative value has exceeded the threshold value, the division into slices is performed such that there is a slice boundary between an MB with which the threshold value has been exceeded and a next MB.
- the slice controlling section 113 After performing the division into slices, the slice controlling section 113 outputs the details of the division to the motion estimating section 109 , in order to cause the motion estimating section 109 not to refer to different slices and therefore maintain the independence of the slices.
- FIG. 10 is a diagram illustrating the division into slices according to the first embodiment. As illustrated in FIG. 10 , the division into slices is performed such that the expected number of codes becomes even in an image. If the expectation is correct, it is possible to perform the division into slices such that the number of codes generated becomes even.
- the expected number of codes in MB 0 is assumed to be 3
- the expected number of codes in MB 1 is assumed to be 2
- the expected number of codes in MB 2 is assumed to be 4.
- variable-length coding unit 114 performs variable-length coding on the data output from the quantization section 104 and outputs the data.
- the variable-length coding is a method in which a variable-length code is assigned in accordance with the frequency of appearance of a symbol.
- the variable-length coding unit 114 basically assigns, for example, a shorter code to a combination between coefficients whose frequency of appearance is higher and a longer code to a combination between coefficients whose frequency of appearance is lower. Thus, the overall code length is reduced.
- a variable-length code according to a method called “Context-Adaptive Variable-Length Coding (CAVLC)” or “Context-Adaptive Binary Arithmetic Coding (CABAC)” may be selected.
- the multiplexing unit 115 multiplexes the data that has been subjected to the variable-length coding in order to obtain a bit stream to be output.
- the CPB 116 stores the bit stream.
- the bit stream stored in the CPB 116 is output to the decoding device.
- FIG. 11 is a flowchart illustrating an example of an encoding process according to the first embodiment.
- the process illustrated in FIG. 11 is a process performed for each MB after the pre-processing.
- the predictive coding unit 117 performs predictive coding on an image for each MB.
- step S 102 the number of codes estimating section 112 and the slice controlling section 113 estimate the number of codes using data to be encoded at a time before variable-length coding, and control the division into slices on the basis of the estimated number of codes, respectively. This process will be described later with reference to FIG. 12 .
- step S 103 the variable-length coding unit 114 performs the variable-length coding on the data to be encoded.
- the steps S 102 and S 103 may be performed in parallel with each other.
- step S 104 the quantization section 104 performs multiplexing on the image including the data that has been subjected to the variable-length coding, in order to generate a bit stream to be output from the image encoding device 10 .
- step S 105 the CPB 116 stores the bit stream in order for a movie not to fail on the decoding side.
- FIG. 12 is a flowchart illustrating an example of a process for controlling the division into slices according to the first embodiment.
- the number of codes estimating section 112 estimates the number of codes (expected number of codes) in each MB at a time after the variable-length coding using the data to be encoded at a time before the variable-length coding.
- the expected number of codes in each MB that has been estimated is output to the slice controlling section 113 .
- step S 202 the slice controlling section 113 accumulates, using the following expression (10), the expected number of codes that has been obtained:
- sum_bit is a cumulative value of the expected number of codes and reset to 0 when processing of a picture begins.
- step S 203 the slice controlling section 113 judges whether or not the cumulative value sum_bit has exceeded a threshold value Th_bit. If the cumulative value has exceeded the threshold value (YES in step S 203 ), the process proceeds to step S 204 . If the cumulative value is equal to or smaller than the threshold value (NO in step S 203 ), processing of a next MB begins.
- the threshold value Th_bit may be obtained using, for example, the following expression (11):
- Th_bit Bit rate/(Picture rate ⁇ Number of divisions) Expression (11)
- the picture rate is the number of pictures per second, which is 60 in the field structure and 30 in the frame structure.
- step S 204 the slice controlling section 113 controls the division into slices such that an MB following the MB with which the threshold value has been exceeded belongs to a next slice. At this time, sum_bit is reset to 0.
- the buffer size As described above, according to the first embodiment, it is possible to reduce the buffer size while maintaining the independence of the slices, thereby reducing a delay. In addition, according to the first embodiment, since the buffer size suits the number of codes in each slice, even if the buffer size is reduced, data is not broken and therefore deterioration of the quality of images may be avoided.
- each picture is not necessarily divided into four slices, because the number of bits is different between the pictures. No problem is caused even if the number of slices obtained by dividing each picture is different between the pictures.
- the image encoding device 20 has a mechanism that feeds back an error between the expected number of codes and the number of codes generated. Therefore, it is possible to perform the division into slices such that the number of codes generated becomes more even.
- FIG. 13 is a block diagram illustrating an example of the configuration of the image encoding device 20 according to the second embodiment.
- the same reference numerals as in FIG. 9 are given to the same components as those illustrated in FIG. 9 , and therefore description of such components is omitted.
- a predictive coding unit 201 obtains an error in the number of codes from a variable-length coding unit 202 . Therefore, a number of codes estimating section 203 outputs the expected number of codes that has been estimated to the variable-length coding unit 202 .
- the variable-length coding unit 202 obtains an error between the expected number of codes that has been obtained and the number of codes at a time after the variable-length coding, and outputs the error to a slice controlling section 204 .
- the slice controlling section 204 corrects a cumulative value of the expected number of codes using the error obtained from the variable-length coding unit 202 .
- the slice controlling section 204 controls the division into slices using the corrected cumulative value.
- the variable-length coding unit 202 may output the number of codes at a time after the variable-length coding to the slice controlling section 204 and let the slice controlling section 204 obtain the error. In this case, the number of codes estimating section 203 does not output the expected number of codes to the variable-length coding unit 202 .
- FIG. 14 is a diagram illustrating an example in which the cumulative value of the expected number of codes is corrected.
- the expected number of codes in MB 0 is assumed to be 3
- the expected number of codes in MB 1 is assumed to be 2
- the expected number of codes in MB 2 is assumed to be 4.
- the actual number of codes in MB 0 is assumed to be 3
- the actual number of codes in MB 1 is assumed to be 4
- the actual number of codes in MB 2 is assumed to be 4. Therefore, there is an error of “+2” in MB 2 .
- the slice controlling section 204 After obtaining the error of 2 in MB 2 from the variable-length coding unit 202 , the slice controlling section 204 corrects the cumulative value when adding the expected number of codes in MB 3 .
- the slice controlling section 204 corrects the cumulative value by, for example, adding the error of 2 to the cumulative value of the expected number of codes of 9 at the point of MB 2 .
- FIG. 15 is a flowchart illustrating an example of a process for controlling the division into slices according to the second embodiment.
- step S 301 illustrated in FIG. 15 the number of codes estimating section 203 estimates the number of codes (expected number of codes) in each MB at a time after the variable-length coding using data to be encoding at a time before the variable-length coding.
- the expected number of codes in each MB that has been estimated is output to the slice controlling section 204 .
- step S 302 the slice controlling section 204 accumulates, using the following expression (10), the expected number of codes that has been obtained.
- sum_bit is a cumulative value of the expected number of codes and reset to 0 when processing of a picture begins.
- step S 303 the slice controlling section 204 corrects the cumulative value sum_bit using the error in the number of codes obtained from the variable-length coding unit 202 .
- step S 304 the slice controlling section 204 judges whether or not the corrected cumulative value sum_bit has exceeded the threshold value Th_bit. If the corrected cumulative value has exceeded the threshold value (YES in step S 304 ), the process proceeds to step S 305 . If the corrected cumulative value is equal to or smaller than the threshold value (NO in step S 304 ), processing of a next MB begins.
- step S 305 the slice controlling section 204 controls the division into slices such that an MB following the MB with which the threshold value has been exceeded belongs to a next slice. At this time, sum_bit is reset to 0.
- an image encoding device has a plurality of modes as CPB modes.
- the CPB modes include, for example, a high-definition mode, an intermediate mode, and a small-delay mode.
- the image encoding device sets the number of slices obtained as a result of the division and/or the group of picture (GOP) structure in accordance with the CPB mode.
- FIG. 16 is a diagram illustrating an example of the CPB modes. In the example illustrated in FIG. 16 , the number of divisions is 1 and the GOP structure is the IBBP structure in the high-definition mode. In this mode, the quality of images has priority over delay.
- the number of divisions is 2 and the GOP structure is the IPPP structure.
- delay may be reduced while deterioration of the quality of images is suppressed.
- the number of divisions is 4 and the GOP structure is the PPPP structure.
- Delay may be further reduced than in the intermediate mode while deterioration of the quality of images is suppressed.
- FIG. 17 is a diagram illustrating examples of the number of divisions and the GOP structure. As illustrated in FIG. 17 , when the number of divisions is 1, the image encoding device may be controlled such that processing is performed in the IBBP structure, in order to give the quality of images priority over delay.
- the buffer delay may be reduced by reducing the size of the buffer in the IPPP structure or the PPPP structure, in which pictures are not rearranged. Therefore, the GOP structure may be changed in accordance with the number of slices obtained as a result of the division.
- a slice controlling section obtains the threshold value Th_bit on the basis of the number of divisions.
- FIGS. 18A and 18B are diagrams illustrating other examples of the CPB modes.
- the number of divisions is the same and the GOP structure is changed. Therefore, differences in the GOP structure correspond to differences in the CPB modes.
- the CPB modes illustrated in FIG. 18B the GOP structure is the same and the number of divisions is changed. Therefore, differences in the number of divisions correspond to differences in the CPB modes.
- the size of the CPB is set to be larger than a capacity set in accordance with Th_bit.
- the size of the CPB may be set in consideration of the number of codes used by header information and the DC coefficient, in order to avoid data corruption due to an overflow in the buffer.
- a buffer capacity of at least 30 Mbit is used if the delay caused by the CPB is 1 second, or a buffer capacity of at least 15 Mbit is used if the delay caused by the CPB is 0.5 second.
- the image encoding device at least a CPB having a maximum bit rate is prepared, and a size of the CPB that is larger than this by a certain size may be set.
- the size of the CPB is 50 Mbit.
- a memory having a sufficient capacity may be installed in the image encoding device, and the size of the CPB may be determined in accordance with the delay caused by the CPB.
- the sequential processing is performed for each MB in the above-described embodiments, the sequential processing may be performed for each slice or picture.
- the above-described embodiments may adopt encoding methods in which division into slices is performed, such as MPEG-2 and MPEG-4.
- the process for encoding an image described in the above-described embodiments may be realized as a program to be executed by a computer.
- this program By installing this program from a server or the like and causing the computer to execute the program, the above-described process for encoding an image may be realized.
- the program is stored in a storage medium and the storage medium in which the program has been stored is read by a computer or a mobile terminal, in order to realize the above-described process for encoding an image.
- the storage medium any of various types of storage media may be used including storage media on which information is recorded optically, electrically, or magnetically, such as a compact disc read-only memory (CD-ROM), a flexible disk, and a magneto-optical disk, and semiconductor memories on which information is electrically recorded, such as a read-only memory (ROM) and a flash memory.
- the process for encoding an image described in each of the above embodiments may be realized by a single integrated circuit or a plurality of integrated circuits.
- FIG. 19 is a diagram illustrating an example of the configuration of an image encoding device 1000 .
- the image encoding device 1000 includes a control unit 1001 , a main storage unit 1002 , an auxiliary storage unit 1003 , a driving unit 1004 , a network interface unit 1006 , an input unit 1007 , and a display unit 1008 . These components are connected to one another through a bus in such a way as to enable transmission and reception of data.
- the control unit 1001 is a central processing unit (CPU) that controls each component and calculates and processes data in a computer.
- the control unit 1001 is an arithmetic unit that executes programs stored in the main storage unit 1002 and the auxiliary storage unit 1003 .
- the control unit 1001 receives data from the input unit 1007 and storage units, and outputs the data to the display unit 1008 and the storage units after calculating and processing the data.
- the main storage unit 1002 is a ROM or a random-access memory (RAM) and is a storage unit that stores or temporarily saves programs and data of an operating system (OS), which is basic software executed by the control unit 1001 , and application software.
- OS operating system
- the auxiliary storage unit 1003 is a hard disk drive (HDD) or the like and is a storage unit that stores data relating to the application software.
- HDD hard disk drive
- the driving unit 1004 reads a program from a storage medium 1005 , which is, for example, a flexible disk, and installs the program in a storage unit.
- a storage medium 1005 which is, for example, a flexible disk
- the driving unit 1004 also stores a certain program in the storage medium 1005 , and the program stored in the storage medium 1005 is then installed in the image encoding device 1000 through the driving unit 1004 .
- the installed certain program may be executed by the image encoding device 1000 .
- the network interface unit 1006 is an interface between a peripheral device having a communication function connected through a network, such as a local area network (LAN) or a wide area network (WAN), constructed by data transmission paths such as a wire circuit and/or a wireless link and the image encoding device 1000 .
- a network such as a local area network (LAN) or a wide area network (WAN)
- LAN local area network
- WAN wide area network
- the input unit 1007 includes a cursor key, a keyboard having numeric input keys, various function keys, and the like, a mouse and a trackpad for selecting a key on a display screen of the display unit 1008 , and the like.
- the input unit 1007 is a user interface for allowing a user to issue operation instructions to the control unit 1001 and to input data.
- the display unit 1008 is configured by including a cathode ray tube (CRT) and a liquid crystal display (LCD), and displays a screen according to display data input from the control unit 1001 .
- CTR cathode ray tube
- LCD liquid crystal display
- the process for encoding an image described in the above embodiments may be realized as a program to be executed by a computer.
- the above-described process for encoding an image may be realized by installing this program from a server or the like and causing the computer to execute the program.
- the program is stored in the storage medium 1005 and the storage medium 1005 in which the program has been stored is read by a computer or a mobile terminal, in order to realize the above-described process for encoding an image.
- the storage medium 1005 any of various types of storage media may be used including storage media on which information is recorded optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, and a magneto-optical disk, and semiconductor memories on which information is electrically recorded, such as a ROM and a flash memory.
- the process for encoding an image described in each of the above embodiments may be realized by a single integrated circuit or a plurality of integrated circuits.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Algebra (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Description
- This application is based upon and claims the benefit of priority of the prior Japanese Patent Application No. 2011-46999, filed on Mar. 3, 2011, the entire contents of which are incorporated herein by reference.
- The embodiments discussed herein are related to an image encoding device that encodes a moving image, a method for encoding an image, and a computer-readable storage medium storing an image encoding program.
- In these years, since moving image data has a large amount of data, the moving image data is subjected to high-efficiency encoding when transmitted from a transmission apparatus to a reception apparatus or when stored in a storage apparatus. The “high-efficiency encoding” is an encoding process in which a certain data string is converted into another data string, thereby reducing the amount of data.
- Typical methods for encoding a moving image include International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group (MPEG)-2/MPEG-4 (hereinafter referred to as MPEG-2 and MPEG-4).
- In MPEG-2, three types of pictures, namely I, P, and B, are defined. An I-picture is a picture obtained by performing an encoding process closed within a frame without referring to an encoded image of another picture. A P-picture is a picture obtained by encoding a prediction error by performing forward prediction from a past picture. A B-picture is a picture obtained by encoding a prediction error by performing bidirectional prediction from past and future pictures.
-
FIG. 1 is a diagram illustrating an example of a sequence having an IBBP structure. Numbers following Is, Ps, and Bs illustrated inFIG. 1 indicate the display order. In the example illustrated inFIG. 1 , the display order is B0, B1, I2, B3, B4, P5, and so on. - In
FIG. 1 , the encoding order is I2, B0, B1, P5, B2, B3, and so on. As illustrated inFIG. 1 , a delay is caused by B-pictures. Encoded data is accumulated in a buffer and output to a decoding device. Since intra-frame encoding is performed for I-pictures, the entirety of the pictures is refreshed. Therefore, propagation of errors may be suppressed by the I-pictures. - On the other hand, in a general moving image encoder, for example, a pre-processing unit, a predictive coding unit, a variable-length coding unit, and a multiplexing unit perform sequential processing (pipeline processing) for each picture. The pre-processing unit includes a picture rearranging unit that rearranges pictures in accordance with the picture types, and sequentially outputs frame images for each picture type and frame or the like.
- In the predictive coding unit, a difference from a predicted image is obtained and orthogonal transformation and quantization are performed. In the variable-length coding unit, entropy coding is performed. In the multiplexing unit, data subjected to the entropy coding is multiplexed and a bit stream to be output is generated.
-
FIG. 2 is a diagram illustrating an example of the sequential processing for each picture. In the example illustrated inFIG. 2 , each process is sequentially performed for each picture. A multiplexed bit stream is temporarily stored in a Coded Picture Buffer (CPB) and output to a decoder. - The CPB is a buffer that holds a bit stream before the bit stream is input to the decoder. The encoder controls the CPB such that a buffer on the decoder side does not fail. In a real-time processing system, delay caused by the CPB is the largest delay.
- If the size of the CPB is increased, the quality of images improves, but the delay becomes larger. If the size of the CPB is decreased, the quality of images deteriorates, but the delay may be reduced. There is a trade-off between the size of the CPB and the quality of images. How the buffer delay may be reduced while maintaining the quality of images is being studied.
- “CPB” is the name of a buffer according to H.264/Advanced Video Coding (AVC) (hereinafter also referred to as H.264). In MPEG, a CPB corresponds to a Video Buffer Verifier (VBV). In the following description, a CPB is used, but in the case of MPEG, the CPB may be replaced with a VBV.
- A method has been disclosed in which, in order to reduce the delay in the real-time processing, a picture is divided into several slices and the slices are subjected to sequential processing. For example, in Japanese Laid-open Patent Publication Nos. 2000-224584, 2010-34802, and 2010-62946, methods have been disclosed in which the size of a slice (the number of macroblocks in a slice) is determined on the basis of the magnitude of a motion vector, the circuit capacity, and the number of codes written to the buffer.
- In accordance with an aspect of the embodiments, an image encoding device that encodes an image by performing sequential processing in units of a certain size, the image encoding device includes, a variable-length coding unit that performs variable-length coding on data to be encoded having the certain size in the image; an estimating unit that estimates the number of codes at a time after the variable-length coding using the data to be encoded at a time before the variable-length coding is performed by the variable-length coding unit; and a slice controlling unit that controls division of the image into slices on the basis of a cumulative value of the estimated number of codes.
- The object and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the claims. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, as claimed.
- These and/or other aspects and advantages will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawing of which:
-
FIG. 1 is a diagram illustrating an example of a sequence having an IBBP structure; -
FIG. 2 is a diagram illustrating an example of sequential processing for each picture; -
FIG. 3 is a diagram illustrating an example of sequential processing at a time when a picture is divided into four slices; -
FIG. 4 is a diagram illustrating a relationship between the number of bits and encoder delay in a slice; -
FIG. 5 is a diagram illustrating a relationship between sequential processing for each macroblock and the encoder delay; -
FIGS. 6A and 6B are diagrams illustrating relationships between the number of bits generated and division into slices; -
FIG. 7 is a diagram illustrating a problem (first problem) caused by the division into slices in the related art; -
FIG. 8 is a diagram illustrating a problem (second problem) caused by the division into slices in the related art; -
FIG. 9 is a block diagram illustrating an example of the configuration of an image encoding device according to a first embodiment; -
FIG. 10 is a diagram illustrating division into slices according to the first embodiment; -
FIG. 11 is a flowchart illustrating an example of an encoding process according to the first embodiment; -
FIG. 12 is a flowchart illustrating an example of a process for controlling the division into slices according to the first embodiment; -
FIG. 13 is a block diagram illustrating an example of the configuration of an image encoding device according to a second embodiment; -
FIG. 14 is a diagram illustrating an example in which a cumulative value of the expected number of codes is corrected; -
FIG. 15 is a flowchart illustrating an example of a process for controlling the division into slices according to the second embodiment; -
FIG. 16 is a diagram illustrating an example of CPB modes; -
FIG. 17 is a diagram illustrating examples of the number of divisions and a picture structure; -
FIGS. 18A and 18B are diagrams illustrating other examples of the CPB modes; and -
FIG. 19 is a diagram illustrating an example of the configuration of an image encoding device. - As a result of a study conducted by the inventor, the following knowledge has been newly obtained. Suppose that an image is divided into certain sizes in order to reduce encoder delay.
FIG. 3 is a diagram illustrating an example of sequential processing at a time when a picture is divided into four slices. As illustrated inFIG. 3 , by performing the sequential processing for each slice, the delay may be smaller than that occurring in the sequential processing for each picture. - However, the number of bits generated in each slice is not constant.
FIG. 4 is a diagram illustrating a relationship between the number of bits and the encoder delay in a slice. As illustrated inFIG. 4 , almost all the number of bits for one picture may be generated in a certain slice. In the example illustrated inFIG. 4 , the number of bits almost equivalent to one picture is generated in slice1. - Therefore, in this case, it is difficult to significantly reduce the encoder delay caused by division into slices. This is because the CPB has substantially the same size as a CPB at the time of the sequential processing for each picture. If the size of the CPB is reduced, the capacity of the CPB becomes insufficient, thereby losing a part of data. If a part of the data is lost, an error stream is caused, thereby making it difficult to normally decode the data.
- Next, suppose that the sequential processing is performed for each macroblock.
FIG. 5 is a diagram illustrating a relationship between the sequential processing for each macroblock and the encoder delay. As illustrated inFIG. 5 , in the sequential processing for each macroblock, although it is possible to further reduce the delay than in the case of the sequential processing for each slice, it is difficult to reduce the size of the CPB. - The reason why it is difficult to reduce the size of the CPB is that, as described above, almost all the number of bits for one picture may be generated in one slice, and therefore the size of the CPB becomes the same as that of a CPB when the sequential processing is performed for each picture. When the size of the CPB is the same as in the case of the sequential processing for each picture, the delay caused by the CPB is also the same.
- When it is difficult to reduce the size of the CPB because a large part of the total number of bits is generated in a certain slice, a method may be used in which the division into slices is performed on the basis of the number of bits generated as disclosed in the related art. In this method, the size of the CPB may be reduced by equalizing the size of the CPB.
-
FIGS. 6A and 6B are diagrams illustrating relationships between the number of bits generated and the division into slices. As illustrated inFIG. 6A , when the division is performed such that the resultant slices have the same size, the number of bits (the number of codes) generated in each slice is different. As illustrated inFIG. 6B , when the number of bits generated is equalized, the size of each slice varies. As illustrated inFIG. 6B , since the numbers of codes generated in slice1 to slice4 are the same, it is possible to reduce the size of the CPB. - However, when division control is performed such that the number of bits generated in each slice becomes the same on the basis of the number of bits written to a buffer unit as in the related art, abnormal data that will be described hereinafter is undesirably generated.
-
FIG. 7 is a diagram illustrating a problem (first problem) caused by division into slices in the related art. As illustrated inFIG. 7 , for example, the sequential processing is performed for each macroblock (MB) and the number of codes generated is counted. When the cumulative value of the number of codes has exceeded a threshold value, the division into slices is performed such that a next MB belongs to a new slice. - As illustrated in
FIG. 7 , however, when the division is performed such that MB2 and MB3 belong to different slices, because the predictive coding has already been completed in MB3, the image region of MB2 might be being referred to, which makes it difficult to maintain the independence of the slices. In addition, when it is difficult to maintain the independence of the slices, it is also difficult to suppress propagation of errors, thereby causing a problem in that it is difficult to refresh each slice. - In the case of encoding according to H.264, it is difficult to judge that there is a slice boundary between MB2 and MB3 in the process of the predictive coding. Therefore, there may be a case in which deblocking filtering overlaps the slice boundary, thereby undesirably losing the independence of the slices.
- In the case of encoding according to H.263, because the slices are not independent from each other, the slice structured mode of Annex K is not available. The image region of another slice is referred to in order to predict a motion vector, which is a violation of the standard.
- In order to solve the above problem, the predictive coding is performed again and data in the predictive coding unit is corrected. However, cost undesirably increases when the predictive coding is performed again.
-
FIG. 8 is a diagram illustrating a problem (second problem) caused by the division into slices in the related art. In the example illustrated inFIG. 8 , the threshold value is assumed to be exceeded when the number of codes generated in MB2 has been added. In this case, the division is performed such that MB1 and MB2 belong to different slices. In order to maintain the independence of the slices, MB2, which has been subjected to the variable-length coding, is discarded and the process is begun again from MB2. - In order to perform the process again from MB2, a mechanism that holds previous MB information is provided and data that has already been written to the CPB is discarded. In addition, in the pre-processing unit or the predictive coding unit, a function of correcting a head MB is provided.
- Therefore, even in a case in which an image is divided into slices having the same size, it has been difficult to reduce the size of the buffer while suppressing deterioration of the quality of the image. In addition, even in a case in which an image is divided into slices such that the number of codes generated is the same in each slice, it has been difficult to maintain the independence of the slices and the process such as the predictive coding is performed again, which has made it difficult to perform efficient encoding.
- The embodiments will be described hereinafter with reference to the accompanying drawings.
-
FIG. 9 is a block diagram illustrating an example of the configuration of animage encoding device 10 according to a first embodiment. Theimage encoding device 10 illustrated inFIG. 9 has apre-processing unit 101, apredictive coding unit 117, a variable-length coding unit 114, amultiplexing unit 115, and aCPB 116. - The
predictive coding unit 117 has a prediction errorsignal generating section 102, a Discrete Cosine Transform (DCT)section 103, aquantization section 104, aninverse quantization section 105, aninverse DCT section 106, a decodedimage generating section 107, aframe buffer 108, amotion estimating section 109, amotion compensating section 110, arate controlling section 111, a number ofcodes estimating section 112, and aslice controlling section 113. - In the first embodiment, an example will be described in which the
pre-processing unit 101, thepredictive coding unit 117, the variable-length coding unit 114, themultiplexing unit 115, and theCPB 116 are subjected to the sequential processing for each MB. - The
pre-processing unit 101 includes a picture rearranging section that rearranges pictures in accordance with the picture types, and sequentially outputs frame images for each picture type and frame or the like. In addition, thepre-processing unit 101 may generate a reduced image and perform a process for investigating the direction of motion or the like. - The prediction error
signal generating section 102 obtains macroblock data (hereinafter also referred to as “MB data”), in which an image to be encoded included in input moving image data has been divided into 16×16 pixel blocks (MBs). The prediction errorsignal generating section 102 generates a prediction error signal on the basis of the MB data and MB data regarding a predicted image output from themotion compensating section 110. The prediction errorsignal generating section 102 outputs the generated prediction error signal to theDCT section 103. - The
DCT section 103 performs a DCT on the input prediction error signal in units of 8×8 or 4×4. Alternatively, instead of the DCT, an orthogonal transform such as a Hadamard transform may be performed. TheDCT section 103 obtains data that has been divided into horizontal and vertical frequency components through the process of the orthogonal transform. - The reason why the data is transformed into the frequency components is that most pieces of the data may be included in low-frequency components due to a spatial correlation of the image, thereby making it possible to reduce the amount of information.
- The
quantization section 104 reduces the number of codes in the data (DCT coefficients) that has been subjected to the orthogonal transform by quantizing the data, and outputs the quantized data to the variable-length coding unit 114 and theinverse quantization section 105. In addition, thequantization section 104 may output the quantized value to the number ofcodes estimating section 112 depending on the situation. - The
inverse quantization section 105 performs inverse quantization on the data output from thequantization section 104. Theinverse quantization section 105 outputs the data that has been subjected to the inverse quantization to theinverse DCT section 106, the number ofcodes estimating section 112, and therate controlling section 111. - The
inverse DCT section 106 performs an inverse DCT on the data that has been subjected to the inverse quantization in order to transform the frequency components into pixel components, and outputs the data at a time after the transform to the decodedimage generating section 107 and therate controlling section 111. By performing this process with theinverse DCT section 106, it is possible to obtain a signal that is substantially the same as the prediction error signal before the encoding. - The decoded
image generating section 107 adds MB data regarding an image that has been subjected to motion compensation in themotion compensating section 110 and the prediction error signal that has been subjected to the decoding process in theinverse DCT section 106. Thus, a processed image that is the same as one on the decoding side may be generated on the encoding side. - The image generated on the encoding side is called a “locally decoded image”. By generating a processed image on the encoding side that is the same as one on the decoding side, it is possible to perform differential encoding on subsequent pictures. The decoded
image generating section 107 outputs, to theframe buffer 108, MB data regarding the locally decoded image that has been generated by adding the MB data and the prediction error signal. The MB data regarding the locally decoded image may be subjected to deblocking filtering. The locally decoded image may be a reference image. - The
frame buffer 108 stores the input MB data as data regarding a new reference image and outputs the data to themotion estimating section 109. - The
motion estimating section 109 performs motion detection using the MB data regarding the image to be encoded and the MB data regarding the encoded reference image that has been obtained from theframe buffer 108, in order to obtain an appropriate motion vector. - A motion vector is a value that indicates a spatial deviation for each block obtained by using a block matching technique by which a position of the reference image that is most similar to the image to be encoded is detected for each block.
- In the motion detection, in general, for example, not only how large the sum of absolute differences of pixels is but also an evaluation value of a motion vector is added. When a motion vector is encoded, not the component itself but a difference vector relative to a motion vector of a nearby MB is encoded. Therefore, the
motion estimating section 109 obtains a difference vector and outputs an evaluation value corresponding to the code length of the motion vector in accordance with the size of the component. - The
motion estimating section 109 outputs the detected motion vector to themotion compensating section 110 and the number ofcodes estimating section 112. - The
motion compensating section 110 performs motion compensation on the data regarding the reference image using the motion vector provided from themotion estimating section 109. Thus, MB data as the reference image that has been subjected to the motion compensation is generated. - The
rate controlling section 111 controls a quantization scale value (qP value) on the basis of the obtained data, in order to control the bit rate. - The number of
codes estimating section 112 estimates the number of codes in each encoding process unit (for example, an MB or the like) at a time after the variable-length coding using the data to be encoded at a time before the variable-length coding. Various methods relating to the estimation of the number of codes will be described hereinafter. - A method in which a DCT coefficient is used will be described. The number of
codes estimating section 112 estimates the number of codes using the DCT coefficients output from theinverse quantization section 105. The number ofcodes estimating section 112 estimates the number of codes by using the rate-distortion function of AC coefficients included in the DCT coefficients and by determining, as the number of codes in an MB, the sum of the number of bits Bij that may be obtained as the square of each AC coefficient using the following expression (1). -
- The rate-distortion function D may be represented by an expression (2) when the distribution of the AC coefficients is assumed to be the Gaussian distribution.
-
D=σ y 2·2−2b Expression (2) - In the above expression (2), “σ” denotes variance and “b” denotes the number of bits.
- By converting the expression (2), an expression (3) is obtained.
-
- The variation is originally represented by the square of σ, but the number of
codes estimating section 112 estimates that the square of each AC coefficient is a value close to the square of σ, and substitutes each AC coefficient for σ and a certain value for D in order to calculates Bij. The number ofcodes estimating section 112 outputs the value obtained by the expression (1) to theslice controlling section 113. - A method in which a motion vector is used will be described. The number of
codes estimating section 112 estimates the number of codes in an MB using the magnitude of a motion vector obtained from themotion estimating section 109. Since the motion vector, too, is included in the data to be encoded, when the number of codes in other pieces of data is assumed to be the same, the number of codes in the MB may be estimated on the basis of the magnitude of the motion vector. - When the magnitude of the motion vector is assumed to be x, the number of
codes estimating section 112 estimates the number of codes using an estimation function F(x). -
F(x)=|x|×β1 (constant) Expression (4) -
F(x)=log|x|×β2 (constant) Expression (5) -
F(x)=|x|̂2×β3 (constant) Expression (6) - The number of
codes estimating section 112 obtains F(x) using any of the expressions (4) to (6) and estimates the value of F(x) to be the number of codes. The number ofcodes estimating section 112 outputs the estimated number of codes to theslice controlling section 113. - A method in which a quantized value is used will be described. The number of
codes estimating section 112 estimates the number of codes using the “level” and the “run” of a quantized value. The number ofcodes estimating section 112 estimates the number of codes on the basis of an estimation function G(y) using the “run” and the “level” of a DCT coefficient after quantization. For example, the number ofcodes estimating section 112 estimates “G(run)+G(level)” to be the number of codes. -
G(y)=|y|×γ1 (constant) Expression (7) -
G(y)=log|y|×γ2 (constant) Expression (8) -
G(y)=ŷ2×γ3 (constant) Expression (9) - Here, the “run” or the “level” is substituted for y.
- The above-described estimation methods are just examples, and the number of
codes estimating section 112 may estimate the number of codes using data at a time before the variable-length coding. In addition, the number ofcodes estimating section 112 may estimate the number of codes in an MB by combining the above-described numbers of codes. - The number of
codes estimating section 112 may obtain the number of codes by, for example, adding Bit obtained using the expression (1) and F(x) obtained using any of the expressions (4) to (6). Alternatively, the number ofcodes estimating section 112 may obtain the number of codes by, for example, adding F(x) obtained using any of the expressions (4) to (6) and G(y) obtained using any of the expression (7) to (9). Alternatively, the number ofcodes estimating section 112 may obtain the number of codes by, for example, adding Bit obtained using the expression (1) and G(y) obtained using any of the expressions (7) to (9). The estimated number of codes is called the “expected number of codes”. - The
slice controlling section 113 accumulates the number of codes obtained from the number ofcodes estimating section 112 and judges whether or not a cumulative value exceeds a threshold value. When the cumulative value has exceeded the threshold value, the division into slices is performed such that there is a slice boundary between an MB with which the threshold value has been exceeded and a next MB. - After performing the division into slices, the
slice controlling section 113 outputs the details of the division to themotion estimating section 109, in order to cause themotion estimating section 109 not to refer to different slices and therefore maintain the independence of the slices. -
FIG. 10 is a diagram illustrating the division into slices according to the first embodiment. As illustrated inFIG. 10 , the division into slices is performed such that the expected number of codes becomes even in an image. If the expectation is correct, it is possible to perform the division into slices such that the number of codes generated becomes even. - As illustrated in
FIG. 10 , for example, the expected number of codes in MB0 is assumed to be 3, the expected number of codes in MB1 is assumed to be 2, and the expected number of codes in MB2 is assumed to be 4. The threshold value is assumed to be 8. At this time, the cumulative value of the expected number of codes at the point of MB2 is 9 (=3+2+4), exceeding the threshold value. Therefore, theslice controlling section 113 controls the division into slices such that division is performed before MB3. - Thus, by making a judgment as to the division into slices with the
predictive coding unit 117, it is possible to maintain the independence of the slices. Since the slices are independent, it is possible to refresh the slices in order to suppress propagation of errors. - In addition, even when the encoding method is H.264, it is possible to avoid deblocking filtering between MB2 and MB3, which are separated from each other by a slice boundary. In addition, even when the encoding method is H.263, it is possible to avoid a violation of the standard.
- In
FIG. 9 , the variable-length coding unit 114 performs variable-length coding on the data output from thequantization section 104 and outputs the data. The variable-length coding is a method in which a variable-length code is assigned in accordance with the frequency of appearance of a symbol. - The variable-
length coding unit 114 basically assigns, for example, a shorter code to a combination between coefficients whose frequency of appearance is higher and a longer code to a combination between coefficients whose frequency of appearance is lower. Thus, the overall code length is reduced. For example, in H.264, a variable-length code according to a method called “Context-Adaptive Variable-Length Coding (CAVLC)” or “Context-Adaptive Binary Arithmetic Coding (CABAC)” may be selected. - The
multiplexing unit 115 multiplexes the data that has been subjected to the variable-length coding in order to obtain a bit stream to be output. - The
CPB 116 stores the bit stream. The bit stream stored in theCPB 116 is output to the decoding device. - Next, the operation of the
image encoding device 10 according to the first embodiment will be described.FIG. 11 is a flowchart illustrating an example of an encoding process according to the first embodiment. The process illustrated inFIG. 11 is a process performed for each MB after the pre-processing. In step S101, thepredictive coding unit 117 performs predictive coding on an image for each MB. - In step S102, the number of
codes estimating section 112 and theslice controlling section 113 estimate the number of codes using data to be encoded at a time before variable-length coding, and control the division into slices on the basis of the estimated number of codes, respectively. This process will be described later with reference toFIG. 12 . - In step S103, the variable-
length coding unit 114 performs the variable-length coding on the data to be encoded. The steps S102 and S103 may be performed in parallel with each other. - In step S104, the
quantization section 104 performs multiplexing on the image including the data that has been subjected to the variable-length coding, in order to generate a bit stream to be output from theimage encoding device 10. - In step S105, the
CPB 116 stores the bit stream in order for a movie not to fail on the decoding side. -
FIG. 12 is a flowchart illustrating an example of a process for controlling the division into slices according to the first embodiment. In step S201 illustrated inFIG. 12 , the number ofcodes estimating section 112 estimates the number of codes (expected number of codes) in each MB at a time after the variable-length coding using the data to be encoded at a time before the variable-length coding. The expected number of codes in each MB that has been estimated is output to theslice controlling section 113. - In step S202, the
slice controlling section 113 accumulates, using the following expression (10), the expected number of codes that has been obtained: -
sum_bit=Expected number of codes in MB Expression (10) - “sum_bit” is a cumulative value of the expected number of codes and reset to 0 when processing of a picture begins.
- In step S203, the
slice controlling section 113 judges whether or not the cumulative value sum_bit has exceeded a threshold value Th_bit. If the cumulative value has exceeded the threshold value (YES in step S203), the process proceeds to step S204. If the cumulative value is equal to or smaller than the threshold value (NO in step S203), processing of a next MB begins. - The threshold value Th_bit may be obtained using, for example, the following expression (11):
-
Th_bit=Bit rate/(Picture rate×Number of divisions) Expression (11) - The picture rate is the number of pictures per second, which is 60 in the field structure and 30 in the frame structure. When the bit rate is assumed to be 6 Mbps and the number of divisions is assumed to be 4, the threshold value is 25,000 bits from the expression Th_bit=6,000,000/(60×4)=25,000 (bit).
- In step S204, the
slice controlling section 113 controls the division into slices such that an MB following the MB with which the threshold value has been exceeded belongs to a next slice. At this time, sum_bit is reset to 0. - As described above, according to the first embodiment, it is possible to reduce the buffer size while maintaining the independence of the slices, thereby reducing a delay. In addition, according to the first embodiment, since the buffer size suits the number of codes in each slice, even if the buffer size is reduced, data is not broken and therefore deterioration of the quality of images may be avoided.
- It is to be noted that even if the number of divisions of each picture is set to 4, each picture is not necessarily divided into four slices, because the number of bits is different between the pictures. No problem is caused even if the number of slices obtained by dividing each picture is different between the pictures.
- Next, an
image encoding device 20 according to a second embodiment will be described. In the second embodiment, theimage encoding device 20 has a mechanism that feeds back an error between the expected number of codes and the number of codes generated. Therefore, it is possible to perform the division into slices such that the number of codes generated becomes more even. -
FIG. 13 is a block diagram illustrating an example of the configuration of theimage encoding device 20 according to the second embodiment. In the configuration illustrated inFIG. 13 , the same reference numerals as inFIG. 9 are given to the same components as those illustrated inFIG. 9 , and therefore description of such components is omitted. - A
predictive coding unit 201 obtains an error in the number of codes from a variable-length coding unit 202. Therefore, a number ofcodes estimating section 203 outputs the expected number of codes that has been estimated to the variable-length coding unit 202. The variable-length coding unit 202 obtains an error between the expected number of codes that has been obtained and the number of codes at a time after the variable-length coding, and outputs the error to aslice controlling section 204. - The
slice controlling section 204 corrects a cumulative value of the expected number of codes using the error obtained from the variable-length coding unit 202. Theslice controlling section 204 controls the division into slices using the corrected cumulative value. - The variable-
length coding unit 202 may output the number of codes at a time after the variable-length coding to theslice controlling section 204 and let theslice controlling section 204 obtain the error. In this case, the number ofcodes estimating section 203 does not output the expected number of codes to the variable-length coding unit 202. -
FIG. 14 is a diagram illustrating an example in which the cumulative value of the expected number of codes is corrected. In the example illustrated inFIG. 14 , the expected number of codes in MB0 is assumed to be 3, the expected number of codes in MB1 is assumed to be 2, and the expected number of codes in MB2 is assumed to be 4. In addition, the actual number of codes in MB0 is assumed to be 3, the actual number of codes in MB1 is assumed to be 4, and the actual number of codes in MB2 is assumed to be 4. Therefore, there is an error of “+2” in MB2. - After obtaining the error of 2 in MB2 from the variable-
length coding unit 202, theslice controlling section 204 corrects the cumulative value when adding the expected number of codes in MB3. Theslice controlling section 204 corrects the cumulative value by, for example, adding the error of 2 to the cumulative value of the expected number of codes of 9 at the point of MB2. - Thus, by correcting the cumulative value of the expected number of codes using the error between the actual number of codes and the expected number of codes, it is possible to perform the division into slices such that the actual number of codes becomes more even.
- Next, the operation of the
image encoding device 20 according to the second embodiment will be described. A process for encoding an MB is the same as that illustrated inFIG. 11 , and therefore description thereof is omitted.FIG. 15 is a flowchart illustrating an example of a process for controlling the division into slices according to the second embodiment. - In step S301 illustrated in
FIG. 15 , the number ofcodes estimating section 203 estimates the number of codes (expected number of codes) in each MB at a time after the variable-length coding using data to be encoding at a time before the variable-length coding. The expected number of codes in each MB that has been estimated is output to theslice controlling section 204. - In step S302, the
slice controlling section 204 accumulates, using the following expression (10), the expected number of codes that has been obtained. -
sum_bit=Expected number of codes in MB Expression (10) - “sum_bit” is a cumulative value of the expected number of codes and reset to 0 when processing of a picture begins.
- In step S303, the
slice controlling section 204 corrects the cumulative value sum_bit using the error in the number of codes obtained from the variable-length coding unit 202. - In step S304, the
slice controlling section 204 judges whether or not the corrected cumulative value sum_bit has exceeded the threshold value Th_bit. If the corrected cumulative value has exceeded the threshold value (YES in step S304), the process proceeds to step S305. If the corrected cumulative value is equal to or smaller than the threshold value (NO in step S304), processing of a next MB begins. - In step S305, the
slice controlling section 204 controls the division into slices such that an MB following the MB with which the threshold value has been exceeded belongs to a next slice. At this time, sum_bit is reset to 0. - As described above, according to the second embodiment, by correcting the cumulative value of the expected number of codes using the error between the actual number of codes and the expected number of codes, it is possible to perform the division into slices such that the actual number of codes becomes more even.
- Next, a modification will be described. For example, an image encoding device has a plurality of modes as CPB modes. The CPB modes include, for example, a high-definition mode, an intermediate mode, and a small-delay mode.
- In this case, the image encoding device sets the number of slices obtained as a result of the division and/or the group of picture (GOP) structure in accordance with the CPB mode.
FIG. 16 is a diagram illustrating an example of the CPB modes. In the example illustrated inFIG. 16 , the number of divisions is 1 and the GOP structure is the IBBP structure in the high-definition mode. In this mode, the quality of images has priority over delay. - In the intermediate mode, the number of divisions is 2 and the GOP structure is the IPPP structure. In this mode, delay may be reduced while deterioration of the quality of images is suppressed.
- In the small-delay mode, the number of divisions is 4 and the GOP structure is the PPPP structure. Delay may be further reduced than in the intermediate mode while deterioration of the quality of images is suppressed.
-
FIG. 17 is a diagram illustrating examples of the number of divisions and the GOP structure. As illustrated inFIG. 17 , when the number of divisions is 1, the image encoding device may be controlled such that processing is performed in the IBBP structure, in order to give the quality of images priority over delay. - When the number of divisions is 4, the buffer delay may be reduced by reducing the size of the buffer in the IPPP structure or the PPPP structure, in which pictures are not rearranged. Therefore, the GOP structure may be changed in accordance with the number of slices obtained as a result of the division. Once the number of slices obtained as a result of the division is determined in accordance with the CPB mode, a slice controlling section obtains the threshold value Th_bit on the basis of the number of divisions.
- There is no particular relationship between the number of divisions and the GOP structure, and therefore other corresponding relationships may be adopted.
FIGS. 18A and 18B are diagrams illustrating other examples of the CPB modes. In the CPB modes illustrated inFIG. 18A , the number of divisions is the same and the GOP structure is changed. Therefore, differences in the GOP structure correspond to differences in the CPB modes. In the CPB modes illustrated inFIG. 18B , the GOP structure is the same and the number of divisions is changed. Therefore, differences in the number of divisions correspond to differences in the CPB modes. - The size of the CPB is set to be larger than a capacity set in accordance with Th_bit. For example, the size of the CPB may be set in consideration of the number of codes used by header information and the DC coefficient, in order to avoid data corruption due to an overflow in the buffer.
- With respect to the capacity of the CPB, when the encoding is assumed to be performed at 30 Mbps, a buffer capacity of at least 30 Mbit is used if the delay caused by the CPB is 1 second, or a buffer capacity of at least 15 Mbit is used if the delay caused by the CPB is 0.5 second. In the image encoding device, at least a CPB having a maximum bit rate is prepared, and a size of the CPB that is larger than this by a certain size may be set.
- In an image encoding device having a maximum encoding rate of 100 Mbps, when the delay caused by the CPB is assumed to be 0.5 second, the size of the CPB is 50 Mbit. A memory having a sufficient capacity may be installed in the image encoding device, and the size of the CPB may be determined in accordance with the delay caused by the CPB.
- Although the sequential processing is performed for each MB in the above-described embodiments, the sequential processing may be performed for each slice or picture. In addition to the encoding methods such as H.264 and H263, the above-described embodiments may adopt encoding methods in which division into slices is performed, such as MPEG-2 and MPEG-4.
- The process for encoding an image described in the above-described embodiments may be realized as a program to be executed by a computer. By installing this program from a server or the like and causing the computer to execute the program, the above-described process for encoding an image may be realized.
- Alternatively, it is also possible that the program is stored in a storage medium and the storage medium in which the program has been stored is read by a computer or a mobile terminal, in order to realize the above-described process for encoding an image. As the storage medium, any of various types of storage media may be used including storage media on which information is recorded optically, electrically, or magnetically, such as a compact disc read-only memory (CD-ROM), a flexible disk, and a magneto-optical disk, and semiconductor memories on which information is electrically recorded, such as a read-only memory (ROM) and a flash memory. Alternatively, the process for encoding an image described in each of the above embodiments may be realized by a single integrated circuit or a plurality of integrated circuits.
-
FIG. 19 is a diagram illustrating an example of the configuration of animage encoding device 1000. As illustrated inFIG. 19 , theimage encoding device 1000 includes acontrol unit 1001, amain storage unit 1002, anauxiliary storage unit 1003, adriving unit 1004, anetwork interface unit 1006, aninput unit 1007, and adisplay unit 1008. These components are connected to one another through a bus in such a way as to enable transmission and reception of data. - The
control unit 1001 is a central processing unit (CPU) that controls each component and calculates and processes data in a computer. In addition, thecontrol unit 1001 is an arithmetic unit that executes programs stored in themain storage unit 1002 and theauxiliary storage unit 1003. Thecontrol unit 1001 receives data from theinput unit 1007 and storage units, and outputs the data to thedisplay unit 1008 and the storage units after calculating and processing the data. - The
main storage unit 1002 is a ROM or a random-access memory (RAM) and is a storage unit that stores or temporarily saves programs and data of an operating system (OS), which is basic software executed by thecontrol unit 1001, and application software. - The
auxiliary storage unit 1003 is a hard disk drive (HDD) or the like and is a storage unit that stores data relating to the application software. - The
driving unit 1004 reads a program from astorage medium 1005, which is, for example, a flexible disk, and installs the program in a storage unit. - The
driving unit 1004 also stores a certain program in thestorage medium 1005, and the program stored in thestorage medium 1005 is then installed in theimage encoding device 1000 through thedriving unit 1004. The installed certain program may be executed by theimage encoding device 1000. - The
network interface unit 1006 is an interface between a peripheral device having a communication function connected through a network, such as a local area network (LAN) or a wide area network (WAN), constructed by data transmission paths such as a wire circuit and/or a wireless link and theimage encoding device 1000. - The
input unit 1007 includes a cursor key, a keyboard having numeric input keys, various function keys, and the like, a mouse and a trackpad for selecting a key on a display screen of thedisplay unit 1008, and the like. In addition, theinput unit 1007 is a user interface for allowing a user to issue operation instructions to thecontrol unit 1001 and to input data. - The
display unit 1008 is configured by including a cathode ray tube (CRT) and a liquid crystal display (LCD), and displays a screen according to display data input from thecontrol unit 1001. - Thus, the process for encoding an image described in the above embodiments may be realized as a program to be executed by a computer. The above-described process for encoding an image may be realized by installing this program from a server or the like and causing the computer to execute the program.
- In addition, it is also possible that the program is stored in the
storage medium 1005 and thestorage medium 1005 in which the program has been stored is read by a computer or a mobile terminal, in order to realize the above-described process for encoding an image. As thestorage medium 1005, any of various types of storage media may be used including storage media on which information is recorded optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, and a magneto-optical disk, and semiconductor memories on which information is electrically recorded, such as a ROM and a flash memory. Alternatively, the process for encoding an image described in each of the above embodiments may be realized by a single integrated circuit or a plurality of integrated circuits. - All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the invention and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions, nor does the organization of such examples in the specification relate to a showing of the superiority and inferiority of the invention. Although the embodiments of the present invention have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.
Claims (20)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2011-046999 | 2011-03-03 | ||
JP2011046999A JP5741073B2 (en) | 2011-03-03 | 2011-03-03 | Image encoding apparatus, image encoding method, and image encoding program |
Publications (2)
Publication Number | Publication Date |
---|---|
US20120224779A1 true US20120224779A1 (en) | 2012-09-06 |
US8781241B2 US8781241B2 (en) | 2014-07-15 |
Family
ID=46753336
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/410,874 Active 2032-08-04 US8781241B2 (en) | 2011-03-03 | 2012-03-02 | Image encoding device encoding an image by performing sequential processing, method for encoding image, and computer-readable storage medium storing image encoding program |
Country Status (2)
Country | Link |
---|---|
US (1) | US8781241B2 (en) |
JP (1) | JP5741073B2 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8781241B2 (en) * | 2011-03-03 | 2014-07-15 | Fujitsu Limited | Image encoding device encoding an image by performing sequential processing, method for encoding image, and computer-readable storage medium storing image encoding program |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6438777B2 (en) * | 2015-01-30 | 2018-12-19 | ルネサスエレクトロニクス株式会社 | Image processing apparatus and semiconductor device |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6611624B1 (en) * | 1998-03-13 | 2003-08-26 | Cisco Systems, Inc. | System and method for frame accurate splicing of compressed bitstreams |
US6636565B1 (en) * | 1999-01-12 | 2003-10-21 | Lg Electronics Inc. | Method for concealing error |
US6891893B2 (en) * | 2000-04-21 | 2005-05-10 | Microsoft Corp. | Extensible multimedia application program interface and related methods |
US20060126744A1 (en) * | 2004-12-10 | 2006-06-15 | Liang Peng | Two pass architecture for H.264 CABAC decoding process |
US20100080478A1 (en) * | 2008-09-29 | 2010-04-01 | Shinichi Satou | Image coding apparatus and image coding method |
Family Cites Families (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3439361B2 (en) | 1999-02-04 | 2003-08-25 | 日本電気株式会社 | Image encoding device and moving image transmission system |
JP2005124041A (en) * | 2003-10-20 | 2005-05-12 | Nec Corp | Image coding device and method therefor |
KR100556340B1 (en) * | 2004-01-13 | 2006-03-03 | (주)씨앤에스 테크놀로지 | Image Coding System |
JP4031455B2 (en) * | 2004-03-29 | 2008-01-09 | 株式会社東芝 | Image encoding device |
JP2009038746A (en) * | 2007-08-03 | 2009-02-19 | Panasonic Corp | Image information encoding device |
CN101897190B (en) * | 2007-11-28 | 2012-06-20 | 松下电器产业株式会社 | Image encoding method and image encoding device |
JP2010034802A (en) | 2008-07-28 | 2010-02-12 | Canon Inc | Moving image encoding device and method of controlling the same |
JP5062110B2 (en) | 2008-09-04 | 2012-10-31 | 富士通株式会社 | Video transmission apparatus, video transmission method, and video transmission program |
JP5059728B2 (en) * | 2008-09-30 | 2012-10-31 | ルネサスエレクトロニクス株式会社 | Image coding apparatus and method |
JP2010109912A (en) * | 2008-10-31 | 2010-05-13 | Canon Inc | Image encoder and image encoding method |
JP5072893B2 (en) * | 2009-03-25 | 2012-11-14 | 株式会社東芝 | Image encoding method and image decoding method |
JP2010278730A (en) * | 2009-05-28 | 2010-12-09 | Mitsubishi Electric Corp | Moving image encoder and moving image encoding method |
JP5741073B2 (en) * | 2011-03-03 | 2015-07-01 | 富士通株式会社 | Image encoding apparatus, image encoding method, and image encoding program |
-
2011
- 2011-03-03 JP JP2011046999A patent/JP5741073B2/en active Active
-
2012
- 2012-03-02 US US13/410,874 patent/US8781241B2/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6611624B1 (en) * | 1998-03-13 | 2003-08-26 | Cisco Systems, Inc. | System and method for frame accurate splicing of compressed bitstreams |
US6636565B1 (en) * | 1999-01-12 | 2003-10-21 | Lg Electronics Inc. | Method for concealing error |
US6891893B2 (en) * | 2000-04-21 | 2005-05-10 | Microsoft Corp. | Extensible multimedia application program interface and related methods |
US20060126744A1 (en) * | 2004-12-10 | 2006-06-15 | Liang Peng | Two pass architecture for H.264 CABAC decoding process |
US20100080478A1 (en) * | 2008-09-29 | 2010-04-01 | Shinichi Satou | Image coding apparatus and image coding method |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8781241B2 (en) * | 2011-03-03 | 2014-07-15 | Fujitsu Limited | Image encoding device encoding an image by performing sequential processing, method for encoding image, and computer-readable storage medium storing image encoding program |
Also Published As
Publication number | Publication date |
---|---|
JP2012186569A (en) | 2012-09-27 |
US8781241B2 (en) | 2014-07-15 |
JP5741073B2 (en) | 2015-07-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2403248B1 (en) | Moving picture encoding device, moving picture encoding method, and moving picture encoding computer program | |
US6125147A (en) | Method and apparatus for reducing breathing artifacts in compressed video | |
US7372903B1 (en) | Apparatus and method for object based rate control in a coding system | |
US8559501B2 (en) | Method and apparatus for adaptively determining a bit budget for encoding video pictures | |
KR101391661B1 (en) | Video coding control method, video coding device and video coding program | |
US20070081589A1 (en) | Adaptive quantization controller and methods thereof | |
US9307241B2 (en) | Video encoding method and a video encoding apparatus using the same | |
US20100254450A1 (en) | Video coding method, video decoding method, video coding apparatus, video decoding apparatus, and corresponding program and integrated circuit | |
JP5133290B2 (en) | Video encoding apparatus and decoding apparatus | |
US20090238283A1 (en) | Method and apparatus for encoding and decoding image | |
US9313496B2 (en) | Video encoder and video encoding method as well as video decoder and video decoding method | |
US20130294502A1 (en) | Motion video encoding apparatus, motion video encoding method, and motion video encoding computer program | |
US20170099491A1 (en) | Apparatus, method and non-transitory medium storing program for encoding moving picture | |
JP7343817B2 (en) | Encoding device, encoding method, and encoding program | |
US20090034632A1 (en) | Intra-forecast mode selecting, moving picture coding method, and device and program using the same | |
US8194740B2 (en) | Apparatus and method for compression-encoding moving picture | |
US8781241B2 (en) | Image encoding device encoding an image by performing sequential processing, method for encoding image, and computer-readable storage medium storing image encoding program | |
US8891622B2 (en) | Motion picture coding apparatus, motion picture coding method and computer readable information recording medium | |
US7991048B2 (en) | Device and method for double-pass encoding of a video data stream | |
US20170099494A1 (en) | Apparatus, method and non-transitory medium storing program for encoding moving picture | |
US20130101022A1 (en) | Methods and devices for encoding and decoding at least one image implementing an estimation in the transform domain, and corresponding signal and computer program | |
US20130287117A1 (en) | Video image encoding device, video image encoding method, video image decoding device, and video image decoding method | |
US8126277B2 (en) | Image processing method, image processing apparatus and image pickup apparatus using the same | |
KR20130032807A (en) | Method and apparatus for encoding a moving picture | |
CN102202220B (en) | Encoding apparatus and control method for encoding apparatus |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: FUJITSU LIMITED, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MISUDA, YASUO;REEL/FRAME:027931/0337 Effective date: 20120105 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551) Year of fee payment: 4 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |