JPH10210475A - Image coder - Google Patents

Abstract

PROBLEM TO BE SOLVED: To warrant the quality of a proper service through the processing satisfying various restrictions by allowing the coder itself to always monitor a state of a processing unit, even during the execution and to control parameters autonomously. SOLUTION: A coder is provided with a buffer monitor/part 1006 that monitors a data amount accumulated in a buffer 1002 and outputs a control signal to decrease space resolution, time resolution, amplitude resolution of an image and fidelity for an input signal as an aim to suppress a generated code amount low, when a write-in margin to the buffer is less than a specified amount and with a generated code amount adjustment part 1005 that outputs a control signal to increase/decrease the space resolution, time resolution, amplitude resolution of an image and fidelity for an input signal as an aim at what a write-in margin to the buffer takes a prescribed value. When no output is provided from the buffer monitor part 1006, the coder is operated by a control signal from the generated code amount adjustment pat 1005, and when an output is provided from the buffer monitor part 1006, the coder is operated based on a control signal from the buffer monitor part 1006.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image encoding apparatus for encoding multimedia information for moving images and the like using a computer.

[0002]

2. Description of the Related Art Moving picture encoding / decoding methods include:
ISO / IEC MPEG1, MPEG2, ITU-T already standardized
H.263 and other systems exist. Also, there are G.723 and other methods for encoding and decoding audio. Transmission and control protocols for moving images and audio information are also specified in H.324 and H.245. Information multiplexing is also defined in H.223.

[0003] Each of these standardization technologies is a standardization organization, for example, for MPEG1, "International Standardization Organization" (In
ternational Standardisaiton Organization).
3 years ”(ISO / IEC 11172-2," Information Technology-Co
ding of Moving Pictures andAssociated Audio for Di
gital Storage Media atup to about 1.5Mbit / s ", 199
As in 3), and for H.263, ITU (International
Telecommunication Union), “ITU-T H.263 Draft Recommendation December 1995” (ITU-T
T H.263 Draft Recommendation: Video coding for lo
w bitrate communication, Dec. 1995), and so on for other standardized technologies.

[0004]

However, considering the case where audio video information is specifically encoded using these techniques, there are various available networks, system processing capabilities, etc. There are many control parameters such as size, frame rate, quantization step,
It is difficult to guarantee the appropriate quality of service by supporting the interoperability including the change of the encoding method on the system side while guaranteeing the real-time communication.

SUMMARY OF THE INVENTION It is an object of the present invention to ensure proper quality of service by always monitoring the status of a processing unit itself during execution, performing autonomous parameter control and performing processing satisfying various restrictions. And

[0006]

In order to solve the above-mentioned problems, the present invention monitors the amount of data accumulated in an output buffer, and generates a code only when the amount of room to be written to the buffer becomes equal to or less than a prescribed amount. Buffer monitoring means for outputting a control signal for lowering the spatial resolution, time resolution, amplitude resolution or fidelity to an input signal for the purpose of keeping the amount low, and the room for writing to the buffer takes a predetermined value. For the purpose of providing a spatial resolution of the image, time resolution, amplitude resolution or a generated code amount adjusting means for outputting a control signal to raise or lower the fidelity to the input signal, when there is no output from the buffer monitoring means, It operates according to the control signal output from the generated code amount adjusting means, and when there is an output from the buffer monitoring means, the control output from the buffer monitoring means. Characterized in that it operates on the basis of items.

Further, the present invention monitors an encoding calculation amount,
Control to reduce the spatial resolution, temporal resolution, amplitude resolution, or fidelity to the input signal in order to reduce the processing amount only when the difference between the encoding calculation amount and the maximum possible amount is less than the specified amount. A calculation monitoring means for outputting a signal; and a control signal for increasing or decreasing the spatial resolution, the time resolution, the amplitude resolution, or the fidelity to the input signal of the image for the purpose of the encoding calculation amount having a predetermined value. It comprises a generated code amount adjusting means, and operates when there is no output from the calculation monitoring means by a control signal output by the generated code amount adjusting means, and when there is an output from the buffer monitoring means, the buffer monitoring means It operates based on a control signal to be output.

Further, the present invention provides a low-order generated code amount adjusting means for outputting a control signal for raising or lowering the spatial resolution, temporal resolution, amplitude resolution, or fidelity to an input signal of an image,
Spatial resolution, temporal resolution, amplitude resolution, or
High-order generated code amount adjusting means for outputting a control signal for raising or lowering the fidelity to the input signal, and when there is no output from the high-order generated code amount adjusting means, based on the output of the low-order generated code amount adjusting means. If there is an output from the higher-order generated code amount adjusting means, the output of the higher-order generated code amount adjusting means operates so as to suppress the output of the lower-order generated code amount adjusting means for a certain period. Features.

The present invention also provides a computer load adjusting means for outputting a spatial resolution, a time resolution, an amplitude resolution, or a control signal for raising or lowering the fidelity to an input signal, a spatial resolution, a time resolution, and an amplitude of an image. A higher-order computer load adjusting means for outputting a control signal for raising or lowering the resolution or the fidelity to the input signal, and when there is no output from the higher-order computer load adjusting means, based on the output of the lower-order computer load adjusting means. And when there is an output from the higher-order computer load adjuster, the output of the higher-order computer load adjuster operates to suppress the output of the lower-order computer load adjuster for a certain period. .

The present invention also provides an arbitration means which has a plurality of the image encoding devices having different encoding characteristics, and generates an operation permission / inhibition signal for each of the plurality of image encoding devices. An image encoding apparatus having a predetermined encoding characteristic is operated using the arbitration means according to a possible generated code amount.

[0011]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows a system configuration diagram in a first embodiment of the present invention. In FIG. 1, reference numeral 11 denotes encoded information;
16 is a control signal, 101 is an image input, 102 is a code output, 103 is a control signal input, 1001 is an H.263 coding unit, 1002 is an output buffer, 1003 and 1004 are control switches, and 1005 is a generated code amount adjustment unit. , 1006 are buffer monitoring units. The image information input from the image input 101 is encoded by the H.263 encoding unit 1001 according to the H.263 encoding method, and the encoded information 11 is temporarily stored in the output buffer 1002 and then transmitted to the code output 102. Sent out.

[0013] The generated code amount adjustment unit 1005 receives the storage amount of the output buffer 1002, and determines the control signal based on "Experts Group on Very Low Bit Rate Visual Telephony" (ITU Telecommunication).
Standaridization SectorLBC-95Study Group 15, Work
ign Party 15/1 Expert's Group on Very Low Bitrate
"Video Codec Test Model TMN5" (Visusal Telephoney) (Title: VIDEO CODEC TEST MODE
L, TMN5 Source: Telenor Research Date: 31 January 19
95), the control switch 100 based on (method 1)
4 to the control signal 12.

When the control signal 16 is input from the control signal input 103, the control switch 1004 controls the generated code amount adjusting unit 1
The control signal from 005 is discarded for 100 milliseconds and the control signal 1
6 is sent to the control switch 1003. Control signal input 1
When there is no signal from the control switch 1004, the control switch 1004 outputs the signal from the generated code amount adjusting unit 1005 to the control switch 10
Send to 03. The buffer monitoring unit 1006 controls the output buffer 1
002 is received as the control signal 13, and if the stored amount exceeds 90% of the storable amount of the output buffer 1002, the spatial resolution, the time resolution, the amplitude resolution, or the fidelity to the input signal indicates the generated code amount. A control signal for changing in a decreasing direction is output, and a control signal 14 is sent to the control switch 1003.

When there is a control signal 14, the control switch 1003 discards the control signal from the control switch 1004 and changes the control signal 15 from the buffer monitoring unit 1006 to H.263.
When the control signal is sent to the encoding unit 1001 and there is no control signal from the buffer monitoring unit 1006, the control signal from the control switch 1004 is used as the control signal 15 as the H.263 encoding unit 1001.
Send to

When there is no output from the buffer monitoring means by the above mechanism, the operation is performed by the control signal output from the generated code amount adjusting means, and when there is an output from the buffer monitoring means, the buffer monitoring means outputs An image coding device that operates based on the control signal to be performed can be realized.

The encoding method of the H.263 encoding unit 1001 may be MPEG1 or the like. FIG. 2 shows a second embodiment of the present invention.
FIG. In FIG.
4 is a control signal, 201 is an image input, 202 is a code output,
203 is a control signal input; 2001 is an H.263 encoding unit;
002 is a calculation monitoring unit, 2003 and 2004 are control switches, and 2005 is a generated code amount adjustment unit.

The image information input from 201 is encoded by an H.263 encoding unit 2001 according to the H.263 encoding method, and the encoded information is sent to a code output 202.
The generated code amount adjustment unit 2005 measures the encoding processing time by setting a processing end flag in the encoding unit for each frame, and controls the control switch 2004 based on (method 2).
Send a control signal to The control signal input from the control signal input 203 is sent to the control switch 2004, and the control signal from the generated code amount adjustment unit 2005 is discarded for 100 milliseconds. During that time, the signal from the control signal input 203 is transmitted to the control switch 2003. When there is no signal from the control signal input 203, a signal from the generated code amount adjustment unit 2005 is sent to the control switch 2003.

The calculation monitoring unit 2002 has an H.263 encoding unit 20
01 is received every 10 milliseconds as the control signal 23, and if the amount of encoding calculation exceeds 90% of the maximum achievable amount of the H.263 encoding unit 2001, the spatial resolution and the temporal resolution , A control signal for changing the amplitude resolution or the fidelity to the input signal in a direction in which the amount of calculation processing decreases, and sends the control signal 24 to the control switch 2003.

The control switch 2003 is connected to the calculation monitoring unit 200
If there is a control signal from the control monitor 2, the control signal from the control switch 2004 is discarded, and the control signal from the calculation monitoring unit 2002 is sent to the H.263 encoding unit 2001.
02 when there is no control signal from the control switch 200
4 is sent to the H.263 encoding unit 2001.

When there is no output from the calculation monitoring means by the above mechanism, the operation is performed by the control signal output from the generated code amount adjusting means. When there is an output from the buffer monitoring means, the buffer monitoring means outputs. An image encoding device that operates based on a control signal can be realized.

The encoding of the H.263 encoding unit 2001 is as follows.
As in the case of the H.263 encoding unit 1001, it may be MPEG1 or the like.

FIG. 3 shows a system configuration diagram according to a third embodiment of the present invention. In FIG. 3, reference numeral 301 denotes an image input;
Is a code output, 303 is a control signal input, 304 is an image coding device according to the first and second embodiments, 3001 is a voice input,
3002 is a multiplexing unit, 3003 is a multiplexed code output,
Reference numeral 04 denotes a higher-order generated code amount adjustment unit. 301, 302,
Reference numeral 303 corresponds to 101, 102, and 103 in the first embodiment and 201, 202, and 203 in the second embodiment, respectively.

Image information input from an image input 301 is encoded by an image encoding device 304 according to the encoding adjustment method described in the first and second embodiments, and the encoded information is sent to a code output 302. You. The code output 302 sends the coded information to the multiplexing unit 3002. Multiplexing unit 3002
Multiplexes the voice information input from the voice input 3001 and the coded information from the code output 302 according to H.223, and outputs the multiplexed code information to a multiplexed code output 3003.

In the ITU-T rate control technique, the frame frequency fluctuates greatly due to frequent frame skipping. On the other hand, a document "MPEG-4
Video verification model version 5.
0 ”(ISO / IEC JTC1 / SC29 / WG11 Document MPEG96 / 1469so
urce: Video group Title: MPEG-4 Video Verification
There is a rate control technology described in Model Version 5.0).

The above published document (ISO / IEC JTC1 / SC29 / WG11)
The described rate control technique is for the purpose of accumulation coding, and the control is performed on the assumption that the number of frames to be coded and the total amount of bits required for coding are known.

Since the present embodiment is intended for real-time communication coding, the control signal is output in accordance with (method 3) in the higher-order generated code amount adjusting section 3004, input. The control signal input to the control signal input 303 controls the image encoding device 304 according to the first and second embodiments.

With the above mechanism, a rate control technique having different characteristics can be easily added as a higher-order control technique.
When there is no output from the higher-order generated code amount adjusting means, the operation is performed based on the output of the lower-order generated code amount adjusting means. An image coding apparatus can be realized in which the output of the code amount adjusting means operates so as to suppress the output of the low-order generated code amount adjusting means.

FIG. 4 shows a system configuration diagram according to the fourth embodiment of the present invention. 4, reference numerals 41 and 42 denote control signals, 4
01, 411, and 4001 are image inputs, 402, 412,
4004 is a code output, 4002 and 4003 are selection units,
005 is an arbitration unit, and 4006 is a user request. 40
Reference numerals 1 and 402 correspond to 101 and 102 in the first embodiment and 201 and 202 in the second embodiment, respectively. 41
Reference numerals 1 and 412 correspond to 101 and 102 in the first embodiment and 201 and 202 in the second embodiment, respectively. However, the coding devices 404 and 414 have the input image size of 352 × 288 for the coding device 404 and the coding device 4
It is assumed that separate settings such as 176 × 144 are made in FIG.

Encoding device 404 upon user request 4006
When the user is requested to select any one of the image input 401 and the code output 402, the request is input to the arbitration unit 4005 and the control signals 41 and 42 for selecting either the image input 401 or the code output 402 are output to the selection units 4002 and 4003.

The selector 4002 connects the image input 4001 to either the image input 401 or 411 according to the control signal 42. The selector 4003 connects either the code output 402 or 412 to the code output 4004 according to the control signal 41.

The image input from the image input 4001 is input to either the image input 401 or 411 in accordance with the selection unit 4002. When the image is input to the image input 401, the coding device 404 operates and the code is output to the code output 402. When the image is input to the image input 411, the encoding device 414 operates to output a code to the code output 412.

The output from code output 402 or 412 is received by selector 4003 and outputs a code to code output 4004.

With the above arrangement, the arbitration means for generating a signal for permitting or prohibiting the operation of each of the plurality of image encoding apparatuses and the predetermined number of arbitration means using the arbitration means according to the possible generated code amount. An image encoding device that operates an image encoding device having encoding characteristics can be realized.

(Method 1) The variables in the method 1 are defined as follows.

[0036]

(Equation 1)

: Average quantization step of previous frame

[0038]

(Equation 2)

: Amount of bits consumed in the previous frame

[0040]

(Equation 3)

: Target bit amount per frame mb: current macroblock number MB: total number of macroblocks

[0042]

(Equation 4)

: Bit amount consumed up to the current macroblock mb for frame t R: Specified bit rate FR: Frame rate of input image

[0044]

(Equation 5)

: Target frame rate f0: parameter for determining the relationship between the target frame rate and the average quantization step of the previous frame (default value 10) In the first frame, the quantization step is encoded by 16

[0046]

(Equation 6)

It is assumed that In the next P frame, at each macroblock line,

[0048]

(Equation 7)

Then, the process is changed to the quantization step. And skip the frame,

[0050]

(Equation 8)

Set according to the following. That is, buffer_conte
Frames are skipped until nt is greater than 3 × R / FR.

For the subsequent frames, (Equation 3) and (Equation 5) are:

[0053]

(Equation 9)

Is calculated according to (Method 2) The variables in Method 1 are defined as follows.

[0055]

(Equation 10)

: Average quantization step of previous frame

[0057]

[Equation 11]

: Encoding processing time of the previous frame

[0059]

(Equation 12)

: Target encoding processing time per frame mb: current macroblock number MB: total number of macroblocks

[0061]

(Equation 13)

: Encoding processing time required up to the current macroblock mb for frame t R: specified bit rate FR: frame rate of input image

[0063]

[Equation 14]

: Target frame rate f0: parameter for determining the relationship between the target frame rate and the average quantization step of the previous frame (default value 10) In the first frame, the quantization step is encoded by 16,

[0065]

(Equation 15)

Assume that: In the next P frame, at each macroblock line,

[0067]

(Equation 16)

According to the above, it is changed to a quantization step. And skip the frame,

[0069]

[Equation 17]

Set according to the following. That is, buffer_content is
Frames are skipped until greater than 3 × R / FR. (Equation 12) and (Equation 14) for the subsequent frames
Is

[0071]

(Equation 18)

Is calculated according to (Method 3) Step 1. Calculation of target bit amount (T) per frame Before the encoding process of the frame, the target code amount of the frame is calculated.

T = Max (R / FR, R / Fc) * (1-w) + S * w R: specified bit rate FR: input frame rate (default 30 frame / sec) w: weighting factor (default 0.05) S : Bit amount required for the previous frame Fc: Frame rate for the past 1 second (Initial value is specified; for example, 10 fps) Next, VBV_fullness: Buffer occupancy (Initial value is VBV_size
/ 2) VBV_size: Updates the target bit amount T according to the buffer amount.

T = T × (a + 2b) (2a + b) a: VBV_fullness b: VBV_size−VBV_fullness Further, the following variation is added to prevent buffer overflow and underflow.

If (VBV_fullness-R / Fc + T> 0.9 × VBV_size) Overflow prevention T = Max (R / FR, R / Fc-VBV_fullness + 0.9 × VBV_size) else if (VBV_fullness-R / Fc + T <0.1 × VBV_size) T = R / Fc-VBV_fullness + 0.9 × VBV_size) Step 2: Calculation of Qstep Tp: Bit amount per frame in the past (here, past 1 second) Qp: During the quantization step of each frame It is considered that the quadratic expression of holds.

Tp = x2 × (1 / Qp) × (1 / Qp) + x1 × (1 / Qp) Then, coefficients x2 and x1 are obtained by regression analysis.

Thus, the quantization step Q can be estimated from the past statistical properties and the target bit amount for the current frame.

[0078] Step 3: Determination of frame skip (after encoding) Rc: Bit amount required for the frame

[0079]

As described above, according to the present invention, a proper service can be provided by monitoring the status of the processing apparatus itself during execution, performing autonomous parameter control, and performing processing satisfying various restrictions. To ensure the quality of the effect is great.

[Brief description of the drawings]

FIG. 1 is a system configuration diagram according to a first embodiment of the present invention;

FIG. 2 is a system configuration diagram according to a second embodiment of the present invention;

FIG. 3 is a system configuration diagram according to a third embodiment of the present invention;

FIG. 4 is a system configuration diagram according to a fourth embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 11 Encoding information 12-16 Control signal 101 Image input 102 Code output 103 Control signal input 21-24 Control signal 41 Control signal 42 Control signal 201 Image input 202 Code output 203 Control signal input 301 Image input 302 Code output 303 Control signal input 304 Image coding device 401 Image input 402 Code output 411 Image input 412 Code output 1001 H.263 coding unit 1002 Output buffer 1003 Control switch 1004 Control switch 1005 Generated code amount adjustment unit 1006 Buffer monitoring unit 2001 H.263 coding unit 2002 Calculation monitoring unit 2003 Control switch 2004 Control switch 2005 Generated code amount adjustment unit 3001 Voice input 3002 Multiplexing unit 3003 Multiplexed code output 3004 Higher-order higher-order generated code amount adjustment unit 4001 Image input 4 02 selector 4003 selecting section 4004 codes the output 4005 arbitration unit 4006 user request

Claims (5)

[Claims] 1. An amount of data stored in an output buffer is monitored.
Outputs a control signal for lowering the spatial resolution, time resolution, amplitude resolution, or fidelity to an input signal of an image for the purpose of holding down the generated code amount only when the room for writing to the buffer becomes equal to or less than a prescribed amount. Buffer monitoring means, and a generated code amount for outputting a control signal that raises or lowers the spatial resolution, time resolution, amplitude resolution, or fidelity to an input signal for the purpose of taking a predetermined value into the buffer for writing into the buffer. An adjusting unit, which operates according to a control signal output from the generated code amount adjusting unit when there is no output from the buffer monitoring unit, and controls the buffer monitoring unit to output when there is an output from the buffer monitoring unit. An image encoding device that operates based on a signal. 2. The method according to claim 1, further comprising the steps of monitoring an encoding calculation amount, and reducing a processing amount only when a difference between the encoding calculation amount and a maximum possible amount is equal to or less than a predetermined amount. , An amplitude resolution or a calculation monitoring means for outputting a control signal for lowering the fidelity to the input signal, and a spatial resolution of the image, a time resolution, an amplitude resolution or for the purpose of the encoding calculation amount taking a predetermined value, The apparatus further includes a generated code amount adjusting unit that outputs a control signal that raises or lowers the fidelity to an input signal. When there is no output from the calculation monitoring unit, the unit operates by a control signal output by the generated code amount adjusting unit, and the buffer An image encoding apparatus characterized in that it operates based on a control signal output from the buffer monitoring means when there is an output from the monitoring means. 3. A low-order generated code amount adjusting means for outputting a spatial resolution, a time resolution, an amplitude resolution or a control signal for raising or lowering the fidelity to an input signal, and a spatial resolution, a time resolution, and an amplitude resolution of the image. Or
High-order generated code amount adjusting means for outputting a control signal that raises or lowers the fidelity to the input signal, and when there is no output from the high-order generated code amount adjusting means, the output of the low-order generated code amount adjusting means is provided. , And when there is an output from the higher-order generated code amount adjusting means,
An image coding apparatus, wherein the output of the higher-order generated code amount adjusting means operates so as to suppress the output of the lower-order generated code amount adjusting means for a certain period. 4. A computer load adjusting means for outputting a spatial resolution, a time resolution, an amplitude resolution or a control signal for raising or lowering the fidelity to an input signal; and a spatial resolution, a time resolution, an amplitude resolution or
A higher-order computer load adjusting unit that outputs a control signal that raises or lowers the fidelity to the input signal, and when there is no output from the higher-order computer load adjusting unit, based on the output of the lower-order computer load adjusting unit. Operate, and when there is an output from the higher-order computer load adjusting means,
An image coding apparatus, wherein an output of the higher-order computer load adjusting means operates so as to suppress an output of the lower-order computer load adjusting means for a certain period. 5. An arbitration system comprising a plurality of image encoding apparatuses according to claim 1 having different encoding characteristics, wherein said plurality of image encoding apparatuses are operated to generate an operation permission signal and an operation inhibition signal. Means for operating an image coding apparatus having predetermined coding characteristics using the arbitration means according to means and a possible generated code amount.