JP2019092075A - Picture encoder and control method and program thereof - Google Patents

Abstract

To allow for generation of encoded data which not only reduce access amount of memory, but also reduce image quality deterioration.SOLUTION: A picture encoder for encoding a dynamic picture image has a memory used as a work memory in encoding, an input part for inputting frames, constituting a dynamic picture image, in order, a first coding part for encoding inputted frames according to set compressibility ratio, and storing the coded data obtained by the encoding in the memory, a determination part for determining the compressibility ratio to be set in the first coding part, a decoding part for decoding the coded data, encoded by the first coding part and stored in the memory, and a second coding part for generating the coded data of the dynamic picture image by using the decoding part and the memory. The determination part determines the compressibility ratio to be set in the first coding part, according to the number of times when the frame, inputted in the input part, accesses the memory while being encoded in the second coding part.SELECTED DRAWING: Figure 1

Description

  The present invention relates to image compression coding technology.

  As a moving image compression coding method, H.264 is used. H.264 and H. 265 mag is known. In an encoding apparatus using these compression encoding methods, encoding processing is performed while reading and writing of original image data and reference image data to a memory. In recent years, the resolution of the imaging device has been increased and the frame rate has been increased. As a result, the amount of memory access per unit time has increased, which has been a factor in increasing the cost of the device. Therefore, there is a technique for reducing the amount of access to the memory by compressing and encoding the original image data and reducing the amount of data (for example, Patent Document 1).

Unexamined-Japanese-Patent No. 2006-303689

  In general, the memory access amount of the image coding unit differs depending on the picture type to be coded. When the picture to be encoded is a P or B picture of an inter prediction code, access to reference image data frequently occurs, so the amount of memory access per unit time is large. Compared with this, when the picture to be encoded is the I picture of the intra prediction code, the memory access amount is smaller compared to the P and B pictures.

  The target compression rate in the band compression unit described in Patent Document 1 is fixed, and pictures with a small amount of memory access and a large number of pictures are compressed at the same compression rate. As a result, in pictures with few memory accesses, compression is performed more than necessary, resulting in image quality deterioration.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique that not only controls the amount of access to the memory but also suppresses the deterioration of the image quality.

In order to solve this problem, for example, an image coding apparatus according to the present invention has the following configuration. That is,
An image coding apparatus for coding a moving image, comprising:
A memory used as a work memory in encoding processing,
Input means for sequentially inputting frames constituting a moving image;
First encoding means for encoding the input frame in accordance with the set compression rate and storing the encoded data obtained by the encoding in the memory;
Determining means for determining the compression rate to be set in the first encoding means;
Decoding means for decoding encoded data by the first encoding means stored in the memory;
And second encoding means for generating encoded data of the moving image using the decoding means and the memory.
The determining means is
When the second encoding means performs encoding, the number of times each frame refers to another frame to encode that frame, and the number of times the frame is referenced to encode another frame And determining the compression rate.

  According to the present invention, it is possible not only to reduce the amount of memory access more than before, but also to generate encoded data that reduces image quality deterioration.

FIG. 1 is a block diagram of an image coding apparatus according to an embodiment. FIG. 2 is a block diagram of a first image encoding unit in the embodiment. The block block diagram of the image decoding part in embodiment. H. The figure which shows the general GOP structure in H.264 system. FIG. 7 is a diagram showing the compression rate of the first image coding unit in the first embodiment. H. The figure which shows the general GOP structure in a H.265 system. The figure which shows the compression rate of the 1st image coding part in 2nd Embodiment. 6 is a flowchart illustrating an example of processing of a control unit 106.

  Hereinafter, embodiments according to the present invention will be described in detail with reference to the attached drawings.

First Embodiment
FIG. 1 is a block diagram of an image coding apparatus 100 according to the first embodiment. The present apparatus 100 includes a first encoding unit 101, a decoding unit 102, a second encoding unit 103, a memory interface unit 104, a memory unit 105, and a control unit 106.

  In FIG. 1, image data to be encoded from the outside is input to the image encoding device 100 via the input terminal 107. In the embodiment, a source for supplying image data to be encoded to the input terminal 107 is an imaging unit, but a storage medium storing image data to be encoded or an image to be encoded is stored. It may be a server on the network, regardless of its type. The memory interface unit 104 arbitrates access to the memory unit 105. Other configurations will be clarified from the following description.

  The first encoding unit 101 performs compression encoding to reduce the amount of input image data before storing the input image data in the memory unit 105. The first encoding unit 101 encodes the input image data in accordance with the compression rate set by the control unit 106 described later.

  FIG. 2 is a block diagram of the first encoding unit 101. As shown in FIG. The first encoding unit 101 will be described below with reference to FIG. Since the first encoding unit 101 in the embodiment encodes the image of the frame of interest in the moving image with a small amount of memory, it does not refer to other frames, and the pixel-based predictive differential encoding (DPCM) method is used. It shall be used. However, although more memory than this is required, other methods such as JPEG may be used.

  As shown in FIG. 2, the first encoding unit 101 includes a subtractor 201, a quantization unit 202, an inverse quantization unit 203, an adder 204, a prediction unit 205, a selector 206, a variable length encoding unit 207, multiplexing And a code amount control unit 209.

  The subtractor 201 subtracts a prediction value generated by a prediction unit 205 described later for each pixel of the input image data to calculate a prediction difference value (also referred to as a prediction error). The quantizing unit 202 quantizes the prediction difference value with a quantization parameter (hereinafter referred to as “QP”) instructed from the code amount control unit 209 described later. The smaller the QP, the finer the quantization step, so that although the image quality deterioration can be suppressed, the coding efficiency becomes lower. In addition, since the quantization step becomes coarser as the QP is larger, high encoding efficiency can be expected, but the image quality deterioration becomes larger accordingly.

  The inverse quantization unit 203 inversely quantizes the predicted difference value quantized by the quantization unit 202 using the same QP as the quantization unit 202. The adder 204 adds the inversely quantized prediction difference value and the prediction value to calculate a decoded value. The prediction unit 205 generates a prediction value using the decoding value. The predicted value is a pixel located in the vicinity of the pixel value to be encoded, and is generated using the decoded value of the already encoded pixel. In general, in the case of pixel-by-pixel coding, the image is coded in raster scan order, so the values of pixels adjacent to the left and above the pixel of interest to be coded may be used directly as prediction values or their averages. The value is the forecast value. Note that, when using the pixel on the left of the pixel of interest as the prediction value of the pixel of interest, the first encoding unit 101 stores or holds the decoded value of the pixel on the left adjacent encoded one before. If you have), you can do it. Incidentally, in the case of adopting JPEG, since it is necessary to encode in 8 × 8 pixels in block units, it is sufficient if there is a buffer memory for 8 lines in the image to be encoded.

  The selector 206 is a selector for selecting 0 as a predicted value when there is no decoded value at the beginning of a picture or line. When 0 is selected, it means that the subtractor 201 supplies the input pixel value to the quantization unit 202 as it is.

  The variable-length coding unit 207 outputs code data to the multiplexing unit 208 for each pixel obtained by coding the quantized prediction difference value according to a predetermined variable-length coding method, and also calculating its code length It is output to the code amount control unit 209. Here, the predetermined variable length coding method is, for example, a Huffman code. When the input value is 0, the variable-length coding unit 207 is assigned code data with the shortest code length, and as the absolute value of the input value is larger, code data with a longer code length is assigned. On the other hand, the predicted difference value is data having positive and negative values, and becomes a value near 0 in a flat part where the variation of the image data is small, and becomes a large difference value in an edge part where the variation is large. Generally, predicted difference data has a characteristic of Laplace distribution centering around 0. Therefore, since code data of a short code length is more frequently used in a general image, the data amount can be reduced. Further, as the quantization unit 202 performs quantization with a larger QP, the absolute value of the prediction difference value becomes smaller, and the code length can be made smaller. In the embodiment, the variable-length coding unit 207 performs Huffman coding. However, for example, Golomb coding may be used.

  The code amount control unit 209 accumulates the code lengths from the variable-length coding unit 207, and calculates the generated code amount from the head pixel of the image of interest (image of interest) to the pixel position of interest. Assuming that the pixel of interest is the i-th pixel of the image of interest, and the code length of the code obtained from the pixel of interest is L (i), the code amount control unit 209 calculates “ΣL (i)”. .

  Also, the code amount control unit 209 determines the target code amount T per image based on the compression rate instructed from the outside (control unit 106). Assuming that the uncompressed data amount of one image is A and the compression rate is R, the target code amount T of the image of interest can be expressed as T = A × R. Here, when the number of pixels included in one image is N, the target code length per pixel can be expressed as T / N = A × R / N. Therefore, the target code amount in the section from the first (first) pixel to the target (i-th) pixel can be expressed as “i × A × R / N”.

The code amount control unit 209 generates a target code amount “i × A × R / N” from the head pixel to the pixel of interest i at every predetermined number of pixels (for example, each pixel for one line) and a code that is actually generated. The code amount “ΣL (i)” of the above is compared, and the QP is calculated so that the generated code amount finally falls within the target code amount in the period of the encoding process of one image. Specifically, when the generated code amount generated up to the pixel of interest is larger than the target code amount (when LL (i)> i × A × R / N), the code amount control unit 209 sets the QP to that time. A positive value ΔQ set in advance is added, and control is performed so that the amount of generated code at a pixel following the target pixel is reduced. Also, when the generated code amount is less than or equal to the target code amount (in the case of Σ L (i) ≦ i × A × R / N), the code amount control unit 209 subtracts ΔQ from QP and generates the generated code at the subsequent pixel. Allow the quantity to be large. The code amount control unit 209 supplies the calculated (updated) QP to the quantization unit 202, the inverse quantization unit 203, and the multiplexing unit 208. In addition, in order to suppress the fluctuation of the QP, the QP may be controlled as follows by using a preset positive value ε.
If L L (i)-i x T / N> ε, then QP QP QP + ΔQ
If -ε <ΣL (i) -i × T / N ≦ ε, maintain QP (not changed)
In the case of LL (i) -i × T / N ≦ −ε, QP ← QP−ΔQ

  Multiplexing section 208 multiplexes the code data from variable length coding section 207 and the QP from code amount control section 209 in a predetermined format, and outputs the result as final encoded data. The encoded code data is stored in a predetermined area of the memory unit 105 via the memory interface unit 104.

  The decoding unit 102 reads out and decodes encoded data of the requested image (encoded data generated and stored by the first encoding unit 101) in response to a request from the second encoding unit 103, The signal is supplied to the second encoding unit 103.

  FIG. 3 is a block diagram of the decoding unit 102. As shown in FIG. In this embodiment, as in the first image coding unit 101, an example using a predictive differential coding (DPCM) method will be described. When the first image coding unit 101 uses another coding method such as JPEG, the same method as that method is used.

  As shown in FIG. 3, the decoding unit 102 includes a separation unit 301, a variable length decoding unit 302, an inverse quantization unit 303, an adder 304, a prediction unit 305, and a selector 306.

  The separation unit 301 separates the input code data into variable-length code data and QP according to a predetermined format. As described above in the first encoding unit 101, the QP is updated for each predetermined number of pixels. The variable-length decoding unit 302 decodes the variable-length code data supplied from the separation unit 301 in the same manner as the variable-length coding unit 207, and obtains a quantized prediction difference value. The inverse quantization unit 303 inversely quantizes the quantized prediction difference value using the QP separated by the separation unit 301 to obtain a prediction difference value. The adder 304 adds the prediction difference value from the inverse quantization unit 303 and a prediction value described later, and outputs the result as the pixel value of decoded image data to the outside.

  The prediction unit 305 generates a predicted value for the pixel of interest. The predicted values are generated using already decoded pixel locations. For example, pixels adjacent to the left side or upper side of the pixel to be decoded are used as they are, or an average value is used. The selector 306 is a selector for selecting 0 as a predicted value when there is no decoded value at the beginning of a picture or line.

  As described above, the decoding unit 102 supplies the image data obtained by decoding to the second encoding unit 103.

  The second encoding unit 103 functions as a main image encoding unit in the image encoding apparatus 100 for recording on a recording medium. Specifically, the second encoding unit 103 is an image encoding unit using an inter prediction encoding method. In the present embodiment, H. An example using the H.264 method will be described.

  The second encoding unit 103 receives the input image data as H.264. Coding according to the H.264 scheme. H. The H.264 system has picture types called I, P, and B pictures. The I picture is encoded by intra prediction coding. In addition, P pictures perform inter prediction coding using temporally previous pictures as reference images. In addition, B picture performs inter prediction coding using preceding and following pictures as reference images. The second encoding unit 103 writes out a reference image in the memory unit 105 or reads out the reference image stored in the memory unit 105 as necessary in the encoding process of the image of each frame. The encoded code data is output to the outside through the output terminal 108 and recorded on a recording medium (not shown).

  As described above, the second encoding unit 103 according to the embodiment uses H.264. Coding is performed according to the H.264 method. During this encoding process, the second encoding unit 103 performs the encoding process using the memory unit 105 as a work memory. Then, when the second encoding unit 103 encodes the image of the encoded data stored in the first encoding unit 101, the second encoding unit 103 decodes the encoded data by the decoding unit 102, and then encodes the image. It will be done.

  The control unit 106 includes a CPU and a memory storing a program to be executed by the CPU, and controls the entire apparatus. In particular, the control unit 106 controls the compression rate of the first encoding unit 101. The control unit 106 determines the compression rate of the first encoding unit 101 according to the encoding characteristic of the frame (picture) to be encoded by the second encoding unit 103. Hereinafter, the method of determining the compression rate of the first encoding unit 101 in the control unit 106 will be described in detail.

  First, in order to explain the encoding characteristics of each picture in the second encoding unit 103, the reference relationship of the pictures will be described using FIG. FIG. The general GOP (Group Of Pictures) structure in H.264 is shown in display order. Arrows 411 to 420 between the pictures indicate directions in which the pictures refer (the tip of the arrow is a reference image).

In the coding of the picture of interest, the number of other pictures read out for reference is referred to as “the number of times of reference”. In addition, in order to encode another picture, the number of times the target picture is read as a reference image is set as the “number of times of reference”. In this case, the “number of references” and the “number of references” of each picture are as follows.
In the case of the I picture 403, the number of times of reference is 0, and the number of times of reference is 5 (arrows 411 to 415).
In the case of the P picture 406, the number of references is one (arrow 413) and the number of references is five (arrows 416 to 420).
In the case of the B picture 404, the number of references is two (arrows 414 and 416) and the number of references is zero.

Further, in each picture, the number of times of reading the reference image from the memory unit 105 and the number of times of writing of the reference image to the memory unit 105 are as follows.
The I picture 403 is read 0 times and written 1 time.
The P picture 406 is read once and written once.
The B picture 404 is read twice, and written out 0 times.

  As described above, it can be seen that the number of times of reference, the number of times of reference, the number of times of reading a reference image, and the number of times of writing a reference image differ depending on the picture type, and the degree of influence on memory access differs. .

  It is known that the quality of a picture generally referred to affects the quality of the picture to be coded that refers to it. Therefore, it can be said that it is desirable to lower the compression ratio of the first encoding unit 101 so that the picture quality deterioration in the first encoding unit 101 does not occur as the picture is referenced more frequently.

  In addition, since the number of times the reference image is read out and the number of times the reference image is written out affect the memory access amount, the amount of code data of the first encoding unit 101 is further reduced as the number of times increases. It can be said that it is desirable to increase the compression rate of the encoding unit 101.

From these, the control unit 106 obtains “the number of times of reference”, “the number of times of reading a reference image”, and “the number of times of writing a reference image” in the second image coding unit 103, and sets these values. Based on the equation (1), the compression rate of the first image coding unit 101 is determined, and the first image coding unit 101 is instructed.
Rate = 1 / ((read_num * x) + (write_num * y)-(ref_num * z) + offset) (1)
Here, the meaning of each symbol is as follows.
rate: compression ratio of the first image encoding unit 101 read_num: number of times of reading reference image write_num: number of times of writing reference image ref_num: number of times of reference x, y, z: predetermined coefficient offset: predetermined constant Expression (1 An example of the compression rate calculated as x = 1, y = 1, z = 0.1 and offset = 1.5 is shown in FIG.

  Note that the control unit 106 may calculate Equation (1) every time the image of interest (frame) is encoded, but, for example, the image of interest may be any of I, P, and B pictures by the second encoding unit 103. It is also possible to perform processing with reference to the table shown in FIG.

  FIG. 8 is a flowchart illustrating an example of processing of the control unit 106 according to the embodiment. The processing procedure of the control unit 106 in the embodiment will be described below with reference to FIG.

  First, in step S801, the control unit 106 determines the compression ratio R based on the access amount when the second encoding unit 103 encodes the target image supplied to the first encoding unit. Note that although this determination process is periodic based on one GOP, in the case of the first embodiment, since the access amount depends on I, P, and B pictures, the compression rate is determined by the picture type. It is good.

  Next, in step S802, the control unit 106 sets the determined compression ratio R in the first encoding unit 101. Then, in step S 803, the control unit 106 causes the first encoding unit 101 to perform encoding processing, and causes the memory unit 105 to store the generated encoded data.

  The encoding process of the second encoding unit 103 operates as a separate thread from the first encoding unit 101. This is because the order of the pictures to be encoded by the second encoding unit 103 is different from the input order.

  As described above, as shown in FIG. 5, the compression rate of each picture is 1/2 for I picture, 1/3 for P picture, and 1 / 3.5 for B picture. I-pictures that are frequently referenced have low compression rates, and B-pictures with high memory accesses have high compression rates, and determine the appropriate compression rate that reflects the importance of image quality and the amount of memory access. While reducing the amount of memory access, it is possible to reduce the image quality deterioration.

Second Embodiment
Next, a second embodiment will be described. The difference between the first embodiment and the present second embodiment is that the coding method of the second image coding unit 103 is H.264. H.264 to H. The point is 265. The coding scheme of the second image coding unit 103 is described below as H.264. The operation of the control unit 106 in the case of adopting H.265 will be described. The other configurations and operations are the same as those of the first embodiment, and therefore the description thereof is omitted.

  H. In order to explain the coding characteristics of each picture of the H.265 scheme, the reference relationship of the pictures will be described with reference to FIG. FIG. A general GOP (Group Of Picture) structure at H.265 is shown in display order. The arrows between the pictures indicate the direction in which the pictures refer (the tip of the arrow is the reference image). As shown in FIG. In H.265, hierarchical coding is adopted, and the prediction structure between reference pictures is hierarchical. In addition, there are cases where a B picture in which a plurality of reference frame numbers for forward prediction can be specified is used and a P picture does not exist. Therefore, except for the first I picture, it has a GOP structure called B picture, and each B picture has a different reference relation.

  The control unit 106 acquires “the number of times of reference”, “the number of times of reading a reference image”, and “the number of times of writing a reference image” in the second image coding unit 103, and based on these values, for example, The compression rate of the first image coding unit 101 is determined as shown in equation (1), and the first image coding unit 101 is instructed.

  In each picture, the number of times of reference, the number of times of reference, the number of times of reading a reference image, the number of times of writing a reference image, and x = 0.5, y = 0.5, z in the equation (1) shown above An example of the compression ratio R when calculated as = 0.3 and offset = 1.7 is shown in FIG.

  The control unit 106 determines the compression rate in accordance with which of FIG. 7 the image supplied to the first encoding unit 101 corresponds to, and sets the compression ratio in the first encoding unit 101 to perform encoding. Let Therefore, the control process for the first encoding unit 101 by the control unit 106 in the second embodiment is the same as the flowchart in FIG. 8.

  As described above, according to the present embodiment, even with the same B picture as shown in FIG. 7, the B picture that is frequently referred to has a low compression rate, and the B picture with many memory accesses is high. Since the compression rate is used, the importance of image quality and the appropriate compression rate reflecting the memory access amount can be determined, and the memory access amount can be reduced and the image quality deterioration can be reduced.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. Can also be realized. It can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

DESCRIPTION OF SYMBOLS 100 ... Image coding apparatus, 101 ... 1st encoding part, 102 ... Decoding part, 103 ... 2nd encoding part, 104 ... Memory interface, 105 ... Memory part, 106 ... Control part

Claims (6)

An image coding apparatus for coding a moving image, comprising:
A memory used as a work memory in encoding processing,
Input means for sequentially inputting frames constituting a moving image;
First encoding means for encoding the input frame in accordance with the set compression rate and storing the encoded data obtained by the encoding in the memory;
Determining means for determining the compression rate to be set in the first encoding means;
Decoding means for decoding encoded data by the first encoding means stored in the memory;
And second encoding means for generating encoded data of the moving image using the decoding means and the memory.
The determining means is
When the second encoding means performs encoding, the number of times each frame refers to another frame to encode that frame, and the number of times the frame is referenced to encode another frame An image coding apparatus that determines the compression rate.   The image coding apparatus according to claim 1, wherein the determination means determines the compression rate to be lower as the number of times of reference is increased.   The image coding apparatus according to claim 1, wherein the determination unit determines the compression rate to be higher as the number of times of reference increases.   The image coding apparatus according to any one of claims 1 to 3, wherein the first coding means is pixel-based predictive coding. A control method of an image coding apparatus for coding a moving image using a memory used as a work memory in coding processing, comprising:
An input step of sequentially inputting frames constituting a moving image;
A first encoding step of encoding the input frame according to a set compression rate, and storing the encoded data obtained by the encoding in the memory;
A determining step of determining the compression rate to be set in the first encoding step;
A decoding step of decoding encoded data according to the first encoding step stored in the memory;
A second encoding step of generating encoded data of the moving image using the decoding step and the memory;
The determination step
When the second encoding means performs encoding, the number of times each frame refers to another frame to encode that frame, and the number of times the frame is referenced to encode another frame The control method of an image coding apparatus, wherein the compression rate is determined.   A program for causing a computer to execute each step of the method according to claim 5 by reading and executing the program.

Images