JP2000261799A - Variable rate moving image encoding device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To encode a moving image in a variable rate at once by encoding an image in the latest GOP section to obtain the value of image complexity of the latest GOP section for the image in the just preceding GOP section and the value of an average coding rate many sections, thereby obtaining the target coding rate, and the target coding quantity of the next GOP section, and encoding in the next GOP section. SOLUTION: An MPEG encoder part the moving image is a basic part performing MPEG coding processing. A target coding quantity allocating part allocates target coding quantity to a GOP unit and an image frame unit and performs sequential coding processing while changing the quantization scale of the MPEG encoder part in performing encoding processing in an image frame. Then, it performs feedback control so that actual generation coding quantity can be constricted to the target coding quantity. A target coding quantity calculating part decides a target coding rate, target coding quantity, the target coding quantity in every I, P and B image frame and the value of a quantization scale in each macroblock in an image frame for the next GOP section.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a DVD (Digital)
Moving images are stored on a digital storage medium such as a Versatile Disk (Digital Video Camera) or a digital video camera.
The present invention relates to a variable encoding rate (hereinafter referred to as a variable rate) moving image encoding apparatus for a digital storage medium, which is used when encoding and accumulating digital image data by a digital image compression method based on a grouping method. GOP (Group Of) within a predetermined maximum change range around a target average coding rate determined from the storage capacity of the
(Pictures) A moving image is encoded at a variable rate once by defining a target encoding rate for each section and performing control so as to change according to a predetermined algorithm.
The present invention relates to a variable-rate video encoding device capable of storing data in a storage medium.

Although the present invention has been described with a particular emphasis on application to storage media, the field of application is not limited to application to recording on storage media. Even in the field of live broadcasting, etc., even when the transmission band is restricted due to the restriction of transmission cost and the like, compared to the fixed rate coding method and the variable rate coding method by the conventional two-pass coding method, A high-quality moving image can be transmitted in real time at low transmission cost, and its application field is wide.

[0003]

2. Description of the Related Art As a method of digitally encoding a moving image and storing the moving image in a digital storage medium, current motion compensation and DCT are used.
(Discrete Cosine Transform) M based on conversion technology
The PEG method is used.

[0004] The MPEG system is an image compression system that removes the redundancy of a moving image by utilizing the properties inherent in a stored moving image and human visual characteristics. , It is possible to compress the data to the order of several tenths or less.

The processing function of the MPEG system shown in FIG.
An explanatory diagram summarizing the main processing functions used in the EG method and the main purpose of the processing is shown. In FIG. 12, each of the unique processing functions shown on the left is used to fulfill the main purpose of the processing shown on the right.

That is, in the MPEG system,. The two-dimensional spatial frequency spectrum component of the image must be easily concentrated in the low band.

[0007] An ordinary slowly changing moving image has a strong correlation in the time axis direction.

[0008] Random access requirements must be imposed on stored moving images as a requirement on the application.

[0009] The resolution sensitivity characteristic with respect to the spatial frequency of the human eye has a band-pass characteristic centered on 2 to several Hz / degree, and the resolution sensitivity characteristic with respect to the luminance component is 2 to 3 times the resolution sensitivity with respect to the color difference component. Have characteristics.

[0010] For images with a lot of movement, the sensitivity of human vision limits the response speed, so that the discrimination sensitivity rapidly deteriorates in both the amplitude axis direction and the spatial frequency resolution performance.

Utilizing the inherent properties such as described above, the following technology is introduced to perform redundancy compression processing of a moving image.

[0012] Generation of a predicted image by motion detection and motion compensation between adjacent image frames, and processing for removing redundancy in the amplitude direction by time-axis difference processing between image frames with strong correlation.

[0013] Extraction of two-dimensional spatial frequency spectrum components by DCT transform using the property of orthogonal transform and redundancy elimination processing by a quantization matrix that performs coarse quantization weighting on high-frequency components.

[0014] In addition to the above, by making the quantization step size (hereinafter referred to as quantization scale) in the amplitude axis variable, redundancy removal processing is performed by assigning a coarse quantization scale to a rapidly moving image or a complex image.

[0015] Redundancy removal processing by subsampling encoding for the color difference components of pixels.

[0016] Redundancy removal processing by variable-length coding using the property of statistical bias in which many consecutive 0s occur on the high frequency side when the quantized spectral components are arranged in order from low frequency components to high frequency components .

A GOP section formed by collecting a predetermined number of a plurality of image frames is divided into three types of image frames, I, P, and B. Intra-frame predictive coding, forward predictive coding, forward and backward bidirectional predictive codes Redundancy removal processing using a combination of optimization.

As described above, in the MPEG system, the redundancy removal processing of a moving image is performed while having a hierarchical processing structure. The MPEG system prediction processing structure and code sequence shown in FIG. An overall picture of the MPEG system is shown together with the finally obtained encoded sequence.

In FIG. 13, a picture is used as a general term for a frame indicating a screen of a progressive scanning system and a field indicating a screen for interlaced scanning. In the following description, the term "image frame" is used to refer to a screen. I do.

As shown in FIG. 13 (a), a prediction process between image frames in a GOP, in the MPEG system, in order to satisfy the requirements of random access, an area must always be expressed in units of a GOP which is the minimum time unit of random access. An I frame for performing intra prediction encoding is inserted, and using this I frame,
It is possible to create a predicted image frame of a P frame for performing forward prediction and a B frame for performing bidirectional prediction.

In addition, since a moving image to be processed often includes a rapidly changing image such as a commercial on a television, the generated code amount is also compared for the P and B image frames, and Switching between inter prediction coding and intra prediction coding can be selected.

In the MPEG system, as shown in FIG. 2 and described above, parameters for controlling redundancy elimination that determine the rate of moving picture coding include: A quantization scale that determines the coarseness of the code amount allocation in the pixel amplitude direction,. A quantization matrix for performing weighted quantization allocation using visual characteristics for two-dimensional spatial frequency spectrum components of pixels; An encoding codebook used when performing variable length encoding using the statistical bias of generated codes,
However, from the viewpoint of the easiness in realizing the circuit and the application effect,
In general, a method of making the quantization scale variable is used.

As is clear from the header configuration shown in the MPEG code sequence of FIG. 13B, this quantization scale is composed of four unit blocks of pixels which are a set of adjacent 8 × 8 pixels. 16 × 16 formed together
It can be made variable for each macroblock that is a set of pixels.

In practice, the quantization scale is designated by designating the quantization scale code corresponding to the quantization scale in the header part of each macroblock.
This relationship is shown in the explanatory diagram of the quantization scale and the quantization scale code in FIG.

In MPEG2, in order to extend the range of redundancy removal for a moving image such as a commercial which has a lot of movement,
In addition to the linear characteristic type 0 of MPEG1, the nonlinear characteristic type 1 can be selected. The quantization scale code QC and the quantization scale Q have a one-to-one correspondence, and coding and decoding are performed by specifying a quantization scale code.

That is, when the movement is severe and the change information of the image is generated frequently along the time axis direction, the quantization scale Q is increased, the display accuracy of the DCT coefficient is reduced, and the movement becomes gentle. As your vision follows better,
The quantization scale is reduced to increase the display accuracy of DCT coefficients. This is also the case when the image becomes a complicated picture and the two-dimensional spatial frequency component increases. Similarly, the quantization scale is coarsened using the characteristic that the discrimination sensitivity of the high-frequency component of vision deteriorates. Thus, it is possible to perform a code amount assignment operation by quantization while suppressing the generated code amount in the amplitude axis direction.

As a result, it is possible to remove redundancy using a human visual characteristic even for a moving image or a complex screen having a large amount of code and a small amount of time correlation between screen frames having a strong motion.

Next, a method of digitally encoding a moving image using the MPEG method and storing the moving image in a digital storage medium includes a coding method of a fixed rate in digital video and the like and a variable rate coding method for a DVD and the like at present. Different coding methods are used.

The fixed-rate coding is performed at a constant rate regardless of the momentary characteristics of a moving image, for example, a rapidly moving image, a slowly moving image close to stillness, a complicated image, a simple image, and the like. This is a method for encoding moving images.

Assuming that the allocated memory capacity of the digital storage medium is M bits, a fixed rate R0 bps corresponding to a predetermined time length T seconds is obtained from R0 = M / T (1). The digital video camera writes the output from the variable-length encoding unit of the MPEG encoder to the output buffer memory, and reads out the data from the output buffer memory at a constant rate determined in this way, based on the past data. In order to prevent the amount of buffer memory from overflowing or underflowing, the value of the quantization scale Q of the encoder is changed in macroblock units to control the amount of generated code.

As described above, in the fixed-rate coding system, the coding rate of a moving image is controlled, and a moving image having a predetermined time length is simultaneously stored in the assigned digital memory capacity by one coding. ing.

The fixed-rate encoding system can easily associate the recording time length of the digital moving image with the allocated capacity of the storage medium, and is easy to handle.

However, since encoding is performed at a constant encoding rate ignoring the nature of the image, even when the amount of code per unit time is large, such as when the movement is intense or a fine video appears, In order to forcibly perform encoding at a constant speed, there is a possibility that compression of redundancy may be performed more than necessary, and the quality of recorded moving images may be reduced when encoding using a storage medium with the same allocated capacity. However, it is inferior to digital moving pictures obtained by variable rate coding.

Therefore, in order to suppress the image quality deterioration, encoding is performed at an encoding speed several times or more as compared with the variable rate encoding. As a result, the medium capacity is required to be several times as large as that of the variable rate coding system, which is a factor of high cost.

Therefore, in the DVD system which requires recording of a moving image with high image quality for a long time, a variable rate encoding system is used.

FIG. 15 is a diagram showing the configuration of a conventional variable-rate coding system encoder, and FIG. 15 is a diagram showing the configuration of the conventional variable-rate coding system in the two-pass coding according to the variable-rate coding system shown in FIG. A diagram showing the relationship between the amount of generated codes in the second time is shown in the processing flow of the conventional variable rate coding method in FIG.

In the conventional variable rate coding apparatus, in order to control the amount of generated codes in accordance with the capacity of the storage medium, in the first coding, all the input images are MPEG-coded by using an appropriate constant quantization scale. Perform the conversion.

In the configuration diagram of FIG. 15, reference numeral 1 denotes a DCT transform unit for performing processing for obtaining a two-dimensional spatial frequency spectrum component using the property of orthogonal transform, and 2 denotes roughness in the amplitude direction according to a quantization scale and a quantization matrix. , A quantizing unit for removing redundancy by allocating a code amount according to the following equation, 3 an inverse quantizing unit for reproducing an original signal from an output of the quantizing unit, and 4 a DCT coefficient representing a two-dimensional spatial frequency spectrum component. An inverse DCT converter for reproducing an image based on the image data, 5 is a frame memory storing the predicted reproduction image information for the number of pictures required for the prediction processing in image frame units, and 6 is a difference memory between the previous and next image frames. Before taking the difference, the motion vector detection is performed in macroblock units to know in which direction and how much the input image to be compared has moved with respect to the comparison reference image in order to minimize the difference. A motion compensating unit for performing an operation of reading an image frame for comparison from a frame memory in accordance with a motion vector instruction from the motion detecting unit; 8, a subtractor for performing difference processing between image frames; Is an adder for adding a difference value serving as a prediction error to a prediction image to reproduce an input image, and 10 is a variable length code for performing a variable length coding process in pixel block units based on an output of the quantization unit. Kabe,
Reference numeral 11 denotes a quantization control unit that controls the quantization scale in units of macroblocks. Reference numeral 12 denotes an output buffer for reading an encoded output from the variable length encoding unit to an external storage medium at a constant speed. , When encoding the B image frame, if the difference processing expands the amplitude information more than the image itself according to the degree of the intensity of the motion of the image, the intra-frame encoding is performed similarly to the I frame, When the amplitude information is reduced by performing the processing, a difference switch between image frames is performed.
In the first encoding, the switch is switched to the buffer memory side during the second encoding by measuring the generated code amount by tilting the generated code amount to the side of the generated code amount measurement unit. Reference numeral 15 denotes the measurement of the generated code amount. The generated code amount measurement unit 16 performs a code amount memory that stores the generated code amount measurement result, and the code amount memory 17 calculates a second code amount based on the first code amount measurement result. 5 shows a target code amount calculation unit to be transmitted to a quantization control unit.

The input CO of the control terminal of the quantization controller is
NT operates the quantization control unit at a fixed quantization scale during the first code amount measurement period, and performs the following at the time of the second encoding.
This is a DC control signal for performing a switching instruction for operating the quantization control unit so as to control the quantization scale based on the code amount allocation result.

A block 100 constituted by the circuit elements 1 to 13 is a basic component of an ordinary MPEG encoder. However, the changeover switch 14 is on the output buffer side.

With the above circuit configuration, as shown in the processing flow of the conventional variable rate coding system in FIG. 17, first, the first coding process is performed while the quantization scale is kept at a constant value. . The operation of the MPEG encoder unit 100 is exactly the same as that of a normal MPEG encoder, using GOP as a group unit for encoding, detecting motion between I, P, and B image frames, motion compensation, difference prediction processing, and DCT code conversion. , Quantization, variable-length coding, header addition processing, and the like.

The generated code by the first encoding is output from the variable length coding unit 10 in byte units, and the generated code amount for each generated image frame is measured by the generated code amount measuring unit 15.
And once recorded in the code amount memory 16 for recording the code amount for each frame and the accumulated code generation amount.

During this time, the external control input CONT is applied to the control terminal of the quantization control unit 11 so that the quantization scale of the quantization unit takes a predetermined constant value.

After the first encoding of all the input image signals is completed, as shown in the two-pass encoding in the conventional variable rate encoding method of FIG. 16, the allocated capacity M of the storage medium and the measured total code stored are stored. Knowing the ratio to the amount McT, the target code amount mci 'to be assigned to each image frame at the time of the second encoding is calculated from the generated code amount mci for each image frame by the following equation.

Mci ′ = mci × M / Σmci = mci × M / McT (2) The value of the first fixed quantization scale is selected to be a small value so that M / McT is usually smaller than 1. ing.

In the second encoding, the switch 14 is set to the output buffer 12 side, and the control input of CONT to the control terminal is switched so that the quantization control unit 11 is set to the automatic control mode. From the new allocated code amount, the allocated code amount per macroblock is known, and the quantization control unit 1
Under the control of 1, the quantization scale is changed for each macroblock, and coding is performed so that the generated code amount becomes the allocated code amount.

The quantization control unit 11 performs the sequential code processing while correcting the error between the actually generated code amount of the second time and the target code amount.

The output buffer 12 is a buffer memory for controlling the queuing with the storage medium and adjusting the reading speed for performing reading at a constant speed equal to the average coding rate.

When an overflow or an underflow is likely to occur in the buffer memory due to an excessively large change in the amount of generated code due to the variable rate coding, the tendency is checked, and the quantization control unit 11 is instructed. Control over the buffer scale to prevent overflow and underflow of the buffer memory.

[0050]

SUMMARY OF THE INVENTION As explained above,
In order to perform variable rate encoding of an input image signal in accordance with the allocated capacity of a storage medium, it was necessary to perform encoding twice. For this reason, it was not possible to cope with real-time applications such as live broadcast of television, and it was troublesome.

Also, like a digital video camera,
For applications where immediate recording is required on the spot, there is no means for providing encoding at one time at a variable rate, and there is no choice but to use fixed rate encoding at a speed several times faster than variable rate encoding. Was used.

For this reason, recording for a long time was difficult, and the medium cost was high.

It is therefore an object of the present invention to provide a variable-rate moving picture encoding apparatus for digital storage media, which encodes a moving picture at a variable rate at one time and allows the picture to be stored in a storage medium.

[0054]

In the MPEG system, the amount of code generation can be controlled by controlling the quantization scale. However, if the quantization scale is controlled according to any algorithm, a predetermined storage allocation capacity can be controlled to a predetermined storage allocation capacity. It is important to be able to appropriately encode a moving image at a time in real time by a variable rate encoding method at one time.

According to the present invention, encoding is performed for the latest GOP section, and the GOP section immediately preceding the GOP
Know the value of the image complexity of the latest GOP section and the value of the multi-section average coding rate for the image of the section,
A target coding rate and a target code amount of the next GOP section around the target average coding rate are obtained, and the next GO
Encode the P section. As described above, variable rate coding is realized by determining the target coding rate for each GOP section and repeating the constant rate coding. The details will be described below.

The basic algorithm of the present invention is shown in the principle explanatory diagram of the present invention in FIG. 1, the basic processing procedure of the present invention is shown in the basic processing flow of the present invention in FIG. 2, and the image complexity is shown in FIG. In the graph, the average quantization scale of the image complexity used in the embodiment of the present invention, an exemplary diagram showing the relationship between the amount of generated code,
FIG. 4 is an explanatory diagram of the time transition of variable rate coding according to the present invention, showing an example of the temporal transition of the coding rate when the present invention is applied.

In the principle explanatory diagram of the present invention shown in FIG.
The EG encoder section is a basic part for performing MPEG encoding processing.
The target code amount allocating unit including the generated code amount measuring unit, the encoded data memory unit, the average value calculating unit, and the target code amount calculating unit is represented by G
A target code amount is assigned to each OP and each image frame,
At the time of the encoding process in the image frame, the quantization control unit of the MPEG encoder unit is controlled on a macroblock basis, the encoding process is sequentially performed while changing the quantization scale, and the actual generated code amount becomes the target code amount. Feedback control is performed so as to converge.

The generated code amount measuring unit calculates the GOP unit, I,
The amount of generated code is measured for each P and B frame and each macro block.

The coded data memory unit stores these data and the quantization scale value from the target code amount calculation unit for a required period.

The average calculator calculates the section image complexity of the immediately preceding GOP, the image complexity of each I, P, and B frame, the multi-section image complexity, and the multi-section average coding rate from these statistical data. .

The target code amount calculation unit calculates a target coding rate, a target code amount, a target code amount for each of I, P, and B image frames and an image frame for the next GOP section based on the equation (3). Are determined by comparing the target code amount and the generated code amount.

As shown in the basic processing flow of the present invention in FIG. 2, in the present invention, the target coding rate of the next GOP section, the GOP unit, A target code amount for each image is determined.

According to the target data, the next section is coded, new statistical data is obtained, and the next GOP
A target value such as a target coding rate for the section is determined. By repeating the above, encoding for the entire medium capacity is performed.

As the statistical data, for each macro block,
The values from the past to the present for the number of averaging sections of the generated code amount, quantization scale, and actual coding rate for each GOP unit for each image frame are used.

Further, the target coding rate R for each GOP section
Equation (3) is used as a calculation equation for determining.

R = R0 + Cr.times.Cc.times.R0 '(3) In the equation (3), R0 is a value obtained by dividing the value of the capacity of the storage medium obtained in (1) by the number of bits by the recording time expressed in seconds. This is the target average coding rate obtained by

The target average coding rate R0 is determined in this way for the storage medium, but in the case of a live broadcast of television or the like, the result of the variable rate coding is read from the buffer memory at a constant speed. It can also be used as an approximation of the transmission speed in that case.

In the equation (3), the second term is a variable rate correction term indicating how much the target coding rate R for each GOP section can be changed from the target average coding rate R0. .

The variable rate correction term is given in the form of a product of an average coding rate correction coefficient Cr, an image complexity change index Cc, and a corrected target average coding rate R0 '.

The average coding rate correction coefficient Cr is a multi-section average coding rate obtained by averaging the actual values of past coding rates with respect to a target average coding rate and a fixed number of sections up to the latest coding section. It is determined from three variables of the allowable variation width of the section average coding rate from the target average coding rate, and has a value of 1 when the multi-section average coding rate is equal to the value of the target coding rate. When the coding rate greatly deviates from the target average coding rate, the value gradually approaches 0,
The variable rate correction term is defined to have a function of asymptotically approaching zero.

The image complexity change index Cc is a section image complexity that defines the complexity of an image with respect to the latest coding section, and a result of the section image complexity for a fixed number of sections up to the latest coding section. Determined from the averaged multi-segment average image complexity, take the value of the rate of change of the inter-segment image complexity with respect to the multi-segment average complexity. Is defined, the value asymptotically approaches 0 and the variable rate correction term asymptotically approaches 0.

The modified target average coding rate R0 'has a value near the target average coding rate R0,
It has a one-to-one correspondence with RO, and has a value obtained by multiplying a correction coefficient having a positive differential coefficient with respect to a change in the target coding rate R0 by a target average coding rate. When the subjective evaluation is severely required at the target average code rate of a low bit rate, it has a role of adjusting the absolute value of the lower limit of the variable rate correction term to a smaller value so that the target coding rate does not become too low.

By defining the average coding rate correction coefficient Cr, the image complexity change index Cc, and the corrected target average coding rate R0 'of the variable rate correction term in the equation (3) as described above, the coding rate is calculated. Is maintained at a predetermined target average coding rate, and high-quality coding can be realized at all times even in the event of a sudden change in the image.

Examples of specific functional forms of the average coding rate correction coefficient Cr and the image complexity change index Cc will be described in the embodiments.

Here, as shown in the image complexity graph of FIG. 3, the image complexity which determines the complexity of the image is determined as the average value of the quantization scale for each macro block per image when the type of image is determined. The product of the obtained average quantization scale value and the generated code amount for each image frame is determined as in equation (4).

Image complexity ≒ Average quantization scale × Code generation code amount (4) The meaning of the image complexity defined by the expression (4) is that when the image is determined, the average quantization scale is increased and the code in the amplitude axis direction is increased. A formula that decreases the amount of generated code per image frame, and conversely, decreases the average quantization scale and increases the amount of generated code when the amount of code in the amplitude axis direction is increased. In this way, it means that the defined image complexity is almost uniquely determined by the image itself.

Therefore, utilizing the fact that an image has the property of equation (4), the difference value between the target code amount assigned per image frame by prediction and the generated code amount for each macroblock of the actual image frame is calculated. Thus, the encoding process can be performed so that the quantization scale value is sequentially controlled so that the generated code amount per image frame converges to the target code amount.

However, since the expression (4) is not an expression having mathematical rigor, it is applied to an actual image and feedback control is performed while observing the result of the actual encoding.

As described above, as a result of encoding the actual image for the next GOP section, an actual measurement value of the quantization scale is obtained, whereby the direction of the change in the complexity of the image can be obtained. ), It is possible to predict the target coding rate of the image for the next GOP section.

The section image complexity is the sum of the image complexity defined for each image frame for all the image frames in the target GOP section, and is a value proportional to the average value of the image complexity for the target section. Take.

As an expression format of the image complexity, there is also a definition formula based on the variance of the pixel value for each block, which is also used as a parameter expressing the complexity of the image. The property of the definition equation of equation (4), which facilitates one-to-one correspondence with the quantization scale, is used in equation (3) of the present invention.

According to equation (3), GO to be encoded
When the target coding rate in the section P is determined, the target coding rate is multiplied by the time length of the GOP section to determine the target code amount for the GOP section.
From the measured values of the image complexity, generated code amount, and generated code amount for each macroblock for each image frame, using the assignment formula shown in the embodiment, for each of the I, P, and B image frames in the next GOP section, And assign a target code amount.

On the basis of this target value, encoding is performed for each image frame and for each macroblock.

Since the actual image changes every moment, prediction is performed based on the coding result up to the immediately preceding GOP section, and even if the target code amount of the next GOP section is allocated, the image having a sharp motion Suddenly, a large difference occurs between the target code amount and the actual generated code amount.

In order to correct this difference and suppress the generated code amount to the target code amount and make the image complexity equal to the image complexity of the new image whose image complexity has been switched, for each image frame,
The value of the quantization scale is sequentially controlled in macroblock units, and finally, the sequential encoding process is performed so that the generated code amount per image frame becomes the allocated target code amount.

In this manner, encoding is performed on all image frames of the GOP.

By repeating the above-described cycle for each GOP section, it is possible to achieve encoding once with a variable encoding rate centering on the target average encoding rate.

FIG. 4 is a diagram illustrating the time transition of the coding rate according to the present invention, showing how the present invention is applied to perform variable rate coding centering on the target average coding rate R0 for each GOP section. Show.

Next, the embodiments will be described more specifically.

[0090]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Equations (5) and (6) are shown as examples of specific definition equations of the average rate correction coefficient Cr and the image complexity change index Cc in the equation (3) described in the previous section.

[0091] Here, RL is an allowable variation width of the multi-section average coding rate,
GCM (Global Complexity Measure) is defined by Expression (7), is a section image complexity obtained as a result of encoding over the immediately preceding GOP section, and includes an average value qj of a quantization scale Q for a macroblock for each image frame, and The total value of the product of the code generation amount Pj for each image frame is used for the image frame in the GOP section.

GCM = [Σ (qj × Pj)] (j = 1 to n) (7) n = total multi-section average image complexity GCMav of the number of image frames within the GOP time, multi-section average coding rate Rav Are the section image complexity GCM and the target coding rate for each section, respectively.
A value obtained by averaging the actually measured coding rate R ', which is an approximate value of R, over a predetermined number of GOP sections m up to the immediately preceding GOP,
It is obtained from equations (8) and (9).

GCMav = avg (GCM) (8) Rav = avg (R ′) (9) However, several hundreds are used as the number m of the sections of the GOP immediately before the averaging target.

When the target coding rate R for the next GOP is determined, the target code amount CD to be allocated to the GOP period length is obtained from the time length Tg of the GOP by equation (10).

CD = R × Tg (10) Here, the time length Tg is obtained from the image frame rate f at the time of reproducing the image and the number n of the image frames of the GOP from (11)
It is obtained by the formula.

Tg = n / f (11) The average coding rate correction coefficient Cr is equal to the target average coding rate.
R0, the multi-section average coding rate Rav, and the allowable variation width RL from the target average coding rate of the multi-section average coding rate are determined from three variables, and the multi-section average coding rate Rav is the average target coding rate. In this case, the value becomes 1 and the value gradually approaches 0 when the deviation from the target average coding rate R0 approaches the allowable variation range RL, indicating that the variable rate correction term has a function of approaching 0.

The image complexity change index Cc has a change ratio value of the section image complexity GCM with respect to the multi-section average image complexity GCMav.
Takes a responsive value according to the rate of change, and when the state of the section image complexity continues to be the same deviation value from the multi-section average image complexity, the value gradually approaches 0, and the variable rate correction term gradually approaches 0. have.

As a result, if the moving image suddenly changes to a complex image, the coding rate is immediately increased. Conversely, if the moving image is changed to a simple image, the coding rate is reduced. When the state in which the coding rate deviates from the target average coding rate continues for a certain period or more, it is possible to automatically return to the target average coding rate.

Thus, while maintaining the average value of the coding rate at a predetermined target average coding rate, it is possible to always realize high-quality moving picture coding even with a sudden change in the picture. Becomes possible.

Next, among GOPs to be encoded, I, P,
Code amount CDI, CDP, CDB for each image frame of B
Is calculated according to the equation (12). * 1 * 1: Comprehensive Multimedia Selection “MPEG” edited by the Institute of Television Engineers, where Np and Nb are uncoded P pictures and B pictures in the GOP section, and Xi, Xp and Xb are images of I, P and B pictures. As for the complexity, Kp and Kb are the ratios of the average quantization scale of the P and B pictures to the average quantization scale of the I picture.

Each time CD ′ is assigned a code amount to an I picture and the remaining code amount after encoding is assigned to P and B pictures, encoding is performed for each image frame. The remaining code amount after subtracting the encoded code amount.

Equation (12) indicates that the target code amount CD for the next GOP section obtained from equation (10) is allocated to each image frame in proportion to the generated code amount for each image frame in the immediately preceding GOP section. This means that encoding is performed while sequentially correcting the error of the encoding generation amount by encoding the remaining screen.

When the allocation of the target code amount for each image frame is determined, the quantization scale code qc (j) for each macroblock in the screen is calculated according to the equations (13) to (16).
Encoding for each image frame is performed while performing sequential control. * 1
First, prior to coding of the j-th macroblock in the I, P, and B image frames, the occupancy of the j-th virtual buffer for each image frame di (j), dp (j), db (j)
Is determined by equation (13).

[0104] Here, B (j) is the actual code generation bit amount from the beginning of the image frame to the j-th macroblock, and Mb is the number of macroblocks in the image frame.

Also, as an initial value of the virtual buffer at the time of encoding for the first (j = 1) macroblock for each image frame, However, r = 2 × R / f (15) is used.

At this time, the quantization scale code qc (j) for the j-th macroblock is obtained by equation (16).

Qc (j) = dx (j) × 31 / r (16) dx (j) = (di (j), dp (j), db (j)) (13) and (16) Each macroblock is quantized by a quantization scale code qc (j, while observing the encoding results of the macroblock up to immediately before, so that the target code amount per image frame matches the code amount generated by performing actual encoding. )
To perform coding by successively performing feedback control.

As is apparent from equation (13), when the difference between the cumulative target code amount distributed in proportion to the macroblock number and the actual cumulative code amount changes greatly,
Since it means that the image changes the image complexity in the direction of the change, the quantization scale code (accordingly, the quantization scale) is changed by the equation (16), and the accumulated code amount is accumulated. Control is performed so as to approach the target code amount.

As is apparent from the equation (13), the error between the accumulated code amount and the accumulated target code amount is reflected at the time of encoding the next macroblock, and the error of the quantized scale code is reduced. Has the effect of feedback control in the direction.

Further, r in equation (15) is called a reaction parameter. If this value is too large, the value of the virtual buffer becomes too large, and the speed of responding to changes in the complexity of the image becomes slow, and the target code If the difference between the amount and the generated code amount is too wide, and if it is too small, the speed of responding to changes in the complexity of the image will be fast, but excessive reaction will occur, and the code amount will be reduced too much, causing deterioration in image quality .

Therefore, an appropriate value exists for the value of the reaction parameter r, and the equation (15) is empirically determined as an optimum value. The right side of the expression (15) means a value twice as large as the average code amount obtained by allocating the target code amount in the GOP section to the image frames of the GOP on average.

The initial value of the equation (14) is also an empirical equation. If this initial value is too large or too small, (1
6) The value of the quantization scale code for the first macroblock determined from the expression is too large or too small, and the difference from the target code amount becomes large at the beginning of the macroblock.

Taking the I-frame as an example, as is clear from substituting the initial value of the virtual buffer in the expression (14) into the expression of the quantization scale code in the expression (16), the coding of the first macroblock of the I-frame is performed. Is set to the quantization scale code 10 and encoding is started.

As described above, if the target code amount for the GOP section is determined by the equations (3) to (10), (1)
According to the expressions (2) to (16), it is possible to control the quantization scale for each macroblock so that the encoding is performed for each image frame and the target code amount and the generated code amount are made to match.

In the illustrative explanation of the target coding rate determination procedure in FIG. 5, a procedure for obtaining the target coding rate R using these equations is schematically shown.

In FIG. 5, for example, when the encoding operation is continuously performed at the average encoding rate R0, the encoding is performed by repeating a certain cycle.

In this state, if the complexity is switched to an image 1.5 times the average complexity GCMav in the next GOP cycle, encoding is performed in a new cycle.

As is clear from FIG. 5, the multi-section average coding rate is always kept within the range of the allowable variation range RL with respect to the average target coding rate R0. Since the long-term average has a symmetric probability density distribution around the degree GCMav, the variable rate is encoded with a symmetric probability density distribution with respect to the target average coding rate R0.

Therefore, it is possible to encode and accumulate a moving image of approximately equal to a predetermined time length with high quality at a variable rate in a storage medium of a predetermined storage capacity.

As shown in FIG. 5, the target coding rate for the next GOP section is determined from the generated code amount and the measured data of the quantization scale according to equation (3), and the actual coding processing according to the target coding rate is performed. A cycle in which a new generated code amount and a value of the quantization scale are determined by the above is repeated.

The procedure of this side will be described in further detail. The procedure of target code amount allocation in FIG. 6, the detailed processing flow chart of the variable rate encoding processing method of the present invention in FIG. 7, and the image encoding in the image frame plane in FIG. It will be described according to the procedure.

FIGS. 6 and 7 show that the target encoding rate R is obtained from the equations (3) to (9), the target code amount CD is determined from the equation (10), and each image frame in the GOP is obtained from the equation (12). , The coding is performed in units of macroblocks within the image frame using the equations (13) to (16), and the
The entire processing procedure for performing encoding for all time in P is shown.

FIG. 8 is a diagram showing a comparison between the target code amount and the actually measured result of the generated code amount for each image frame based on the comparison result of the actual code amount and the generated code amount, according to the equations (13) to (16). The procedure for performing the conversion is shown below.

Further, in the example of transition of the transient response of the target coding rate and related parameters in FIG. 9, the target coding rate R for each GOP section, the variable rate correction coefficient Cr of the equation (4), and (5)
A calculation example using the number m of sections of the GOP of the image complexity change index Cc in the equation as a parameter is shown.

FIG. 9 shows a state in which the average image continues, and the value of the section image complexity GCM suddenly changes to the immediately preceding image.
This shows the temporal transition of the transient response of the target coding rate R, the variable rate correction coefficient Cr, and the image complexity change index Cc when the image changes to a 1.8-fold image and the state continues for a long time. When a moving image with an image complexity greatly deviated from the average image complexity continues for a long time, both the variable rate correction coefficient Cr and the image complexity change index Cc tend to converge to 0,
Due to the synergistic effect of the two terms of the variable speed correction coefficient Cr and the complexity change index Cc, the image complexity deviates from the average complexity of the image over a long period of time and deviates from the average target coding rate R0. Deter.

In the case where a complicated image continues, the time constant at which the deterrent effect works depends on the number m of sections in which the average of past GOPs is averaged. Converges faster and if m is larger,
The convergence speed to R0 becomes slow.

Normally, when a similar image is recorded for a long time, the average complexity GCMav is multiplied by the average complexity GCMav.
Has a general rule that the value of has a substantially symmetric occurrence probability density distribution.

In the present invention, utilizing the above general rules,
According to the equation (3), the multi-section average coding rate is suppressed within a range of an allowable change width RL with respect to the target average coding rate R0,
By repeating constant-rate encoding at a variable rate for each section of the GOP, an average target encoding rate determined by the medium capacity and the recording time length is within an allowable variable range.
Performing the encoding, exceptionally, if a complex image or a simple image subsequently tries to deviate from the average target encoding rate R0,
The effect of pulling back the variable rate correction term works to return the target coding rate to the average target coding rate R0. Enables variable rate coding and storage.

When the accumulated error between the actual coding result and the allocated capacity is increased beyond the allowable error range, the value of the average target coding rate R0 is appropriately corrected in the middle or the remaining prediction recording is performed. The influence of the occurrence of minute errors in the accumulation time is reduced by a method such as calculating and displaying the time Tr from the allocated accumulation capacity M and the accumulated code amount ΣCDi up to that time by Tr = (M−ΣCDi) / R0 (17). be able to.

FIG. 10 shows a first example of a circuit configuration based on the principle of the present invention in Example 1 of the encoder configuration using the variable rate coding method of the present invention.

In FIG. 10, the MPEG encoder section 100 has the same configuration as the configuration diagram of the conventional variable rate encoding encoder shown in FIG.

The block section 200 is a target code amount allocating section.

Reference numeral 15 'denotes a generated code amount measuring unit for measuring the generated code amount in units of macroblocks, image frames, and GOPs. Reference numeral 16' denotes the generated code amount data and the quantization control unit 11 in units of macroblocks. A coded data memory 18 for storing the value of the quantization scale obtained from
Based on the past generated code amount data stored in the coded data memory, the latest GOP section image complexity GCM is calculated by the formulas (6), (7), and (8). An average value calculator for calculating the section average complexity GCMav and the multi-section average coding rate Rav, and 17 ′ is a coded data memory 1
6 ′ and the multi-section average coding rate in the latest GOP obtained from the average value calculation unit 18
Rav and section image complexity GCM, multi-section average image complexity GCMa
From (v), the calculation of the target allocation code amount for the next GOP and each of the I, P, and B image frames in the GOP is (3) to (11).
This is a target code amount calculation unit that uses an expression.

In the configuration shown in FIG. 10, the result of encoding up to the latest GOP is obtained as an output of the variable length encoding unit 10 and is measured by the generated code amount measuring unit 15 '. 'Is stored in In the coded data memory 16 ', the quantization scale value qi is stored in macroblock units for each image frame in the previous GOP section.

The average value calculator 18 calculates the section image complexity GCM for the latest GOP from the equation (7) based on the value of the coded data memory, and calculates the multi-section average image complexity GC from the equation (8).
For Mav, a multi-segment average coding rate Rav is obtained from equation (9).

The target code amount calculation unit 17 ′ calculates (3) to (1)
The target code amount CD for the next GOP section is obtained from the equation 1),
The target code amount is further allocated to each of the I, P, and B image frames of the GOP by using Expression (12) in accordance with the actually measured generated code amount for each image frame.

In response to the code amount distribution result for each image frame, eleven quantization control units sequentially control the quantization scale for each macroblock in each image frame by using equations (13) to (16). While the input image signal
The encoding operation of the MPEG encoder unit of 0 is performed.

In practice, the code amount allocation for each GOP, image frame, macroblock, and block is shown in FIG.
The code amount obtained by subtracting the code amount of the header shown in the code sequence of the MPEG system in (b) in advance is allocated.

As a result of performing the encoding process over the entire screen, the value of the new quantization scale GCM becomes
It is obtained for the next GOP section.

On the basis of the value of the quantization scale and the actually generated code amount, the average value calculating unit 18 and the target code amount calculating unit 17 'determine a predetermined value based on the equations (3) to (10). Is calculated, and the value of the coding rate R for the GOP is determined.

A second embodiment of the present invention is shown in the second embodiment of the encoder configuration according to the variable rate coding system of the present invention shown in FIG. In this configuration, the configuration of the code amount allocating unit is configured to measure the generated code amount from an MPEG encoder that performs MPEG encoding at a fixed quantization scale.
The input image signal delayed by one GOP, which is the same as the processing time of the code amount allocating unit, is encoded again according to the target code amount allocation.

Although an extra MPEG encoder unit is required, the structure of the first embodiment in which the next GOP section is predicted from the code result obtained in the previous GOP section and coding is performed while correcting the prediction error , The accuracy of target code amount allocation can be improved.

In FIG. 11, the MPEG encoder section of 100
It has basically the same configuration and function as the MPEG encoder unit 100 of FIG.

The MPEG encoder section 100 'is a section for performing MPEG encoding at a fixed quantization scale, excluding the quantization control section from the basic MPEG encoder section of 100.

Block 200 'corresponds to block 2 in FIG.
A target code amount allocating unit having a rate control function similar to 00. Reference numeral 20 denotes an image frame memory portion having a capacity of 1 GOP, and 100 MPEG encoder portions
Using the result of the predictive coding of the target code amount allocating unit of 0 ',
It serves as a delay line for performing encoding processing on the GOP.

Referring to FIG. 11, 10
The MPEG encoder unit of 0 'performs an encoding process on the GOP section to be encoded while keeping the quantization scale constant, and based on the generated code amount, the image complexity and the section complexity are calculated. Degree, multi-section average image complexity, and multi-section average coding rate, and based on these values, delay by 1 GOP,
For the input image signal input to the 100 MPEG encoders, a target code amount for the GOP is obtained from equations (3) to (10).

The target code amount is proportionally distributed based on the actually measured code amount for each image frame.

Next, using the result, the frame memory 20 has already performed the encoding process in the 100 'MPEG encoder unit which is delayed by one GOP and is to be encoded, and MPEG encoding is performed on the obtained GOP while controlling the quantization control unit.

That is, based on the target code amount allocation result, the eleven quantization controllers are controlled based on the equations (13) to (16), and are input with a delay of one GOP while changing the quantization scale. In the same manner as in the first embodiment, the input image signal is subjected to encoding processing of 100 MPEG encoder units.

When the second embodiment is compared with the first embodiment, in the first embodiment, it is assumed that the image having the same tendency continues in the next GOP based on the complexity obtained by the encoding result of the previous GOP. Then, the code amount is allocated to the next GOP. Therefore,
When the variable rate is changed when there is a sudden change in the image, a tracking delay occurs.

On the other hand, in the second embodiment, the complexity of the GOP period to be encoded is measured, and the target rate and the target code amount of the GOP period are determined again based on the complexity. Therefore, the accuracy of the prediction is greatly improved, and the control of the variable rate can be more appropriately performed according to the complexity of the image.

However, in the configuration of the target code amount allocating section,
New MPEG encoder for pre-prediction encoding, 1G
An extra image frame memory for OP is required.

[0153]

According to the present invention, moving images can be recorded at a variable rate.
It is possible to realize a variable-rate video encoding device for digital storage media that can be encoded in one time and stored in a storage medium, reducing costs by reducing the required storage medium capacity, and broadcasting live TV. Even if it is used for real-time applications, its application effects such as reduction of transmission cost are high.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating the principle of the present invention.

FIG. 2 is a basic processing flow of the variable rate encoding system of the present invention.

FIG. 3 is a graph of image complexity.

FIG. 4 is an explanatory diagram of a time transition of variable rate coding according to the present invention.

FIG. 5 is an illustrative description of a procedure for determining a target coding rate.

FIG. 6 shows a procedure of target code amount allocation.

FIG. 7 is a detailed processing flow chart of the variable rate coding method of the present invention.

FIG. 8 shows a procedure of image coding in an image frame.

FIG. 9 is an example of transient response transition of a target coding rate and related parameters.

FIG. 10 is a first embodiment of an encoder configuration according to the variable rate encoding method of the present invention.

FIG. 11 is a second embodiment of the encoder configuration according to the variable rate encoding method of the present invention.

FIG. 12 shows a processing function of the MPEG system.

FIG. 13 shows a prediction processing structure and a code sequence of the MPEG system.

FIG. 14 shows a quantization scale and a quantization scale code.

FIG. 15 is a configuration diagram of an encoder using a conventional variable rate coding scheme.

FIG. 16 shows two-pass encoding using a conventional variable rate encoding method.

FIG. 17 is a processing flow of a conventional variable rate coding method.

[Description of Signs] 1, 1 ′ DCT Transformer 2, 2 ′ Quantizer 3, 3 ′ Inverse Quantizer 4, 4 ′ Inverse DCT Transformer 5, 5′20 Frame Memory 6, 6 ′ Motion Estimator 7 , 7 'Motion compensation unit 8, 8' Subtractor 9, 9 'Adder 10, 10' Variable length coding unit 11, 11 'Quantization control unit 12, 12' Output buffer 13, 13 ', 14 Changeover switch 15 , 15 ′, 15 ″ Generated code amount measurement unit 16 Code amount memory 16 ′, 16 ″ Encoded data memory 17, 17 ′ Target code amount calculation unit 18, 18 ′ Average value calculation unit 100, 100 ′ MPEG encoder unit 200, 200 'target code amount allocating unit

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5C057 AA01 AA09 BA19 CA01 CB05 CC04 CE04 EF05 EG06 EH07 EK03 EL01 EM04 EM09 EM13 EM16 GE08 GF03 GF04 GG05 GH05 GJ05 5C059 KK26 KK33 MA00 MA05 MA14 SS23 MC53 TA57 TB03 TC19 TC38 TD16 UA02

Claims (5)

[Claims] A moving picture encoder for controlling a generated code amount by making a quantization scale variable, a target average coding rate, and a latest code composed of a fixed number of image frames from the moving picture encoder. The target coding rate for the next coding section and the product of the target coding rate and the time length of the coding section in accordance with a predetermined formula based on the generated code amount for the coding section and the measured data of the quantization scale. A target code amount allocating unit for calculating a target code amount and allocating the target code amount to each image frame, defined by the following. By repeating coding at the allocated target coding rate for each coding section, A variable-rate video encoding apparatus, which encodes a video once at a variable rate around the target average encoding rate. 2. A variable rate correction term representing a deviation of the target average coding rate and a target coding rate for each coding section from the target average coding rate. The variable rate correction term is defined as a target average coding rate, a multi-section average coding rate obtained by averaging the actual values of past coding rates for a certain number of sections up to the latest coding section, The multi-section average coding rate is determined from three variables of the allowable variation width from the target average coding rate, and takes a value of 1 when the multi-section average coding rate is equal to the target average coding rate. When the value deviates greatly from the conversion rate toward the permissible variation range, the value gradually approaches 0, and the variable rate correction term has a function of approaching 0.
Average coding rate correction coefficient, section image complexity totaling image complexity defining the image complexity for each image frame over the coding section, section image complexity for a fixed number of sections up to the latest coding section Determined from the multi-segment average image complexity obtained by averaging the degrees, the change rate of the section image complexity with respect to the multi-segment average complexity is taken as a value, and the section image complexity has the same deviation value state as the multi-segment average complexity And an image complexity change index whose value asymptotically approaches 0 and ascends the variable rate correction term to 0, having a value near the target average coding rate, and one-to-one from the target average coding rate. The variable rate video encoding apparatus according to claim 1, wherein the variable rate video encoding apparatus is defined as a product of three parameters of a modified target average encoding rate determined by the following equation. 3. The image complexity according to claim 2, wherein the image complexity is determined as a value of a product of an in-screen average value of a quantization scale determined for each macroblock in the image frame and a generated code amount for the image frame. The variable-rate video encoding device according to claim 1, wherein: 4. The variable-rate video coding apparatus according to claim 1, wherein the target code amount allocating unit uses an MPEG coder as the video coder, and outputs an output from the variable length coding unit. A generated code amount measuring unit for measuring the generated code amount for each of the macroblock, the image frame, and the coding section; generated code amount data from the generated code amount measuring unit;
A coded data memory for storing a quantized scale value obtained for each macroblock from a quantization control unit of the PEG encoder; a quantized scale value stored in the coded data memory;
An average value calculation unit that calculates the section image complexity, the multi-section average image complexity, and the multi-section average coding rate according to claim 2 from a generated code amount. From the calculation data of the average value calculation unit, a target coding rate for the next coding section is calculated using the predetermined calculation formula in claim 2, and the target coding rate is calculated for each image frame for the latest coding section. A target code amount calculation unit that performs allocation calculation and determines a quantization scale value for each macroblock in each image frame, and performs an encoding process in which a generated code amount per image frame is set as a target code amount per image frame. A variable-rate video encoding device, which performs the encoding sequentially. 5. The variable-rate video encoding apparatus according to claim 1, wherein the target code amount allocating unit performs an MPEG encoding process on a fixed quantization scale.
A G encoder; a generated code amount measuring unit for measuring a generated code amount of a macro block, an image frame, and an encoded section for the latest encoding section from an output of a variable length encoding unit of the MPEG encoder; Coded data memory for storing the generated code amount data from the amount measuring unit, the target coding rate value from the target code amount calculation unit, the generated code amount stored in the coded data memory and the code for each past coding section An average value calculator for calculating the section image complexity of the coding section, the multi-section average image complexity for a fixed number of sections up to the section, and a multi-section average coding rate from the actualization rate actual value; A target coding rate and a target code amount for the section are calculated from the accumulated data of the data memory and the calculation data of the average value calculation unit using a predetermined calculation formula according to claim 1, and each image of the section is calculated. 2. A target code amount calculator for calculating a target code amount for a frame and determining a quantization scale value for each macroblock in an image frame, wherein the input to the MPEG encoder according to claim 1 is one code. A delay circuit for an encoding section to input an input image signal delayed by one encoding section, and performs an encoding process with a generated code amount per image frame as a target code amount per image frame. The variable-rate video encoding device according to claim 1, wherein