JP3486871B2 - High-performance code compression system for video information - Google Patents

Description

Detailed Description of the Invention

[0001]

The present invention relates are required to the television telephone or the like, relates to high-performance code compression system of the moving image information to the bit rate reduction. That is, in the image information, a highly important part such as a face including the movement of the lips associated with talking in a videophone and a less important part such as a background other than a human face are distinguished, and the information is processed. Weighted to improve information transmission efficiency, and compressed with a minimum amount of image information transmitted to accommodate conventional telephone lines with limited transmission capacity, facial expressions close to natural conversation, especially lips suitable for speaking It discloses a high performance code compression system of the motion i.e. moving image information so as to realize lip synchronization.

[0002]

2. Description of the Related Art Images seen on conventional videophones are
It is an image that is degraded by thinning out image information so that it can be stored within the allowable range of the amount of information that can be transmitted under the restriction of a telephone line. It is a still image that transmits intermittent changes rather than a moving image on a television. It was like a series of pictures, so to speak, the photograph of the face of the talker on the phone was delivered a little later than the voice of the phone by electronic photograph, that is, FAX. that is,
25 frames per second (PAL, SECAM television in Europe and Russia) or 30 frames per second (Japan-US countries) (NTSC TV) The TV image to be sent was dropped, and the original moving picture function of the TV was greatly reduced to a compromise. In addition, since the image has much more information than voice even if the majority of frames are thinned out, it takes time to process and transmit the image, and the image is received with a delay, the movement of the lips does not match the talking.
On the other hand, if you force the audio to be synchronized (delayed) so that it matches the image received late, the response of the conversation will be delayed, which is very likely to be seen in satellite relay on TV. It becomes a messy conversation.

[0003]

However, it is not always necessary to target the image quality of a high-definition movie for theatrical screening in an image used in a videophone or the like. If the motion is seen, the parts other than the face with the movement of the lips are the same as those of the original videophone without pursuing the reproduction of the motion with high fidelity up to a movie of 24 frames per second or a TV of 25 or 30 frames per second. The purpose can be achieved.

On the screen of the videophone, in consideration of the original purpose of the videophone, consideration is given to the importance of each specific area including the outline of the face of the subject in the screen,
Even if the signal processing of each part is weighted based on it and compressed to the necessary minimum amount of information, the original purpose of the videophone, that is, the limitation of the amount of information transmission in the existing telephone line network
Even within the range, an image showing the movement of the talker's lips and talking can be transmitted.
Sound that is reproduced in real time
There has been a demand for a high-performance code compression system for moving image information that guarantees a collaborative image quality and that does not impair the atmosphere of a videophone conference, which complies with the specifications of a simple configuration .

The problem to be solved by the present invention is that
Only the part of the face that stands out easily
Code for which preferential information processing is performed as
In a compression system, conventional electric power with a limited transmission capacity is used.
Minimal amount of image information transmission compressed to accommodate speech lines
So, facial expressions that are close to natural conversation, especially lip movements that are suitable for speaking
That is, the high quality of the moving image information that realizes lip synchronization
Providing a functional code compression system, that is, a videophone
Is to improve performance while remaining extremely simple and inexpensive.
It Therefore, specifically, information compression rate and noise
Relationship, handling of the difference image information of B frame,
Improvement of rate control mechanism of information compression processing in encoder
That is .

[0006] reasons required improvement in the rate control mechanism is a two-fold, one is for the purpose of corresponding to the restriction information transmission capacity of the transmission path, reproducing a moving image in second is the decoder side This is for averaging the bit lengths of the image frames as much as possible in order to make the speed of movement constant.

Therefore, in the conventional rate control system, the International Telecommunications Union ITU (International Te
Video Codec Test Mode published by LeCommunication Union)
1, Near-Term, Version 11 (hereinafter,
(Referred to as TMN-11). There are several types of methods adopted in the moving image information compression software program based on the H.263 plus standard.

However, in the conventional rate control system, a delay time and a frame drop that occur until a camera input image is output as a decoded image via the encoder, the transmission path and the decoder are eliminated. It does not have a function to strictly control the delay time to minimize it, and this delay time is a problem. , And the subject of lip synchronization in which the movement of the mouth and the voice do not match. Moreover, in order to average the bit lengths of the respective image frames with high accuracy, a very complicated calculation is required, and further the delay time is unavoidable due to the calculation processing.

Therefore, in the present invention, the information compression rate and the
Relation of noise, handling of difference image information of the B frame
In addition, in order to average the bit lengths of the image frames with high accuracy, a very complicated calculation, which was conventionally indispensable, is replaced with a simple calculation so that the calculation process described above can be performed. It is an object of the present invention to provide a system capable of realizing the lip synchronization by reducing the required delay time.

[0010]

To achieve SUMMARY OF for the] said object, the inventions of claims 1, stores the image data and Okumemo <br/> Li (1), of the moving image during the recognition process A specific area that is movable on the screen, that is, a window (21) to which priority information processing is applied, and all the images forming the window (21) are divided into rectangular small blocks, and search data for each small block is divided. Sequential processing means for sequentially inputting the reference data and the reference data from the memory (1) to execute window parallel processing, and a motion vector search unit (10) for searching a motion vector associated with the motion of the moving image of the small block. And the position of the face window (21) of the next frame is estimated by using the motion vector searched by the motion vector search unit (10), and the window (21) is made to follow the motion of the main subject. Moving with a position control program, the
A high-performance code compression system for image information, including a motion prediction function unit that inputs a current frame image, a previous reference frame image, and a subsequent reference frame image to perform motion prediction, motion compensation, and prediction method determination, and a motion prediction function thereof. All pixel value zeroization function unit for inputting the difference information between the prediction image output from the unit and the current frame image and forcibly setting all pixel values of the difference information to zero, and the all pixel value zeroization function A code generation unit that inputs the zeroed all-pixel information output from the unit, encodes the zeroed all-pixel information while predicting the next motion of the moving image by the prediction method determined by the motion prediction function unit , Equipped with
And a decoding unit that receives and decodes moving image information that is code-compressed and transmitted by the encoder and transmitted via a transmission path, and a decoding output from the decoding unit. An inverse quantization unit that inputs a signal and inversely quantizes it, and a discrete cosine inverse transformation unit that restores a differential image by inputting the inverse quantized signal output from the inverse quantization unit and inversely transforming the discrete cosine A B-frame processing, and an addition unit that adds the restored difference image and the prediction image predicted by the prediction method and outputs a decoded image.
Quantize the luminance signal and the color difference signal by rounding off only in the case of an intra macroblock that directly encodes the current macroblock image without using the prediction image , and if it is other than the intra macroblock. to reduce noise quantized chrominance signals are quantized by rounding, the color difference signal while applying the same quantization level into a luminance signal and a color difference signal by devaluation luminance signal to noise
And a noise reduction means for multiplying the sum of the inverse quantization correction value (p) and the rounding correction parameter (f) by the quantization level (Q) and the quantization level correction value (s). ,
The value obtained by subtracting the value obtained by subtracting the absolute value (| C |) of the original data composed of frequency components that have been discrete cosine transformed by the value obtained by multiplying the quantization level (Q) and the quantization level correction value (s). Quantizing means for making an integer obtained by rounding down a numerical value the absolute value (| L |) of quantized data , and
As the quantizing means, the correction parameter (f) is set to 0 so that the luminance signal and the color difference signal are quantized by rounding only in the case of an intra macroblock in which the current macroblock image is directly encoded without using the prediction image. a first quantizing means for setting the 5, to quantize the devaluation the luminance signal in the case of other than the intra-macro block, the set correction parameter (f) to zero, the color difference signals are due to rounding to quantize the correction parameter high code compression of video information to the second quantizing means for setting the (f) the 0,5, comprising the sheet
It is a stem .

According to the operation of the invention of claim 1,
In the B-frame coding method for compressing the information of the current frame, the difference information itself is not transmitted as the difference information with the image information of the reference frames before and after, and the difference calculation type, ie, the forward direction We have established a B-frame information compression method that sends only prediction, backward prediction or bidirectional prediction. By doing so, when the motion of the moving image is noticeable and intense, the amount of information in the difference image also increases, but in order to minimize the amount of information in the difference image, all pixel values are forced to zero. By doing so, the image of the transmission source can be restored at the transmission destination with the least amount of information.

Also, by changing the quantization method in the encoder for the luminance signal and the two color difference signals, it is possible to reduce the noise in the color difference signals when decoding is performed at the same quantization level in the decoder. it is that having established the quantizer to improve visual image quality of the image. By doing so, the noise reduction effect of the color difference signals can be efficiently exhibited with a simpler structure than the conventional one. Therefore, due to the nature of the human eye, there is also the effect that visually higher image quality can be obtained.
There is .

The invention according to claim 2 is characterized in that the comparing means compares the bit amount of the encoded moving image information with the communication buffer residual bit amount, and the comparison result of the comparing means.
Control means for controlling the target bit amount of the frame so that the residual bit amount of the communication buffer is not exhausted, and the control result by the control means is used, and the camera input image is the encoder,
To minimize the delay time and drop frame generated until the transmission path and via said decoder is output as the decoded image, in the frame-level rate control, and calculation means of the target bit amount of the frame 2. A high-performance code compression system for moving image information according to claim 1, further comprising:
System .

The invention according to claim 3 is the first calculation for calculating the quantization level applied to the first macroblock of the frame by using the weighted average of the quantization levels of the macroblocks of the previous frame. Means for finely adjusting the quantization level applied to the second and subsequent macroblocks, using the target bit amount, the actual code amount up to the current macroblock, and the quantization level applied to the first macroblock. a second calculating means for calculating a high-performance code compression system of video information according to claim 2, comprising the.

The operation of the invention according to claims 2 to 3
More, the coding of the image, while minimizing transfer and a delay time required for the encoding of the information, to minimize the dropped frames number, determine the optimum target code bit amount for each frame (hereinafter, This process is referred to as frame level rate control), and the quantization level is optimally adjusted in each subsequent macro block (hereinafter, this process is referred to as macro block level rate control). This method exhibits excellent controllability despite the small amount of calculation. By doing so, in order to average the bit lengths of the image frames with high accuracy, a very complicated calculation, which has been indispensable in the past, is replaced with a simple calculation, thereby performing the calculation process described above. Therefore, the delay time can be reduced and lip synchronization can be realized.

According to a fourth aspect of the present invention, a memory (1) for storing image frame information and a hardware module that operates independently of each other are coupled via a data bus (2), and the memory is stored. A system in which a centralized control device (3) for controlling a data flow and an operation schedule between (1) and each of the hardware modules is coupled to each of the hardware modules via a control bus (4).
4. An encoder and / or a decoder configured according to the architecture, comprising :
It is a high-performance code compression system for moving picture information described in 1 .

By doing so, hardware modules having a small circuit scale, which have flexibility and high speed and operate with low power consumption, are bus-coupled, and the data between the operation schedule and the memory data can be obtained. With the AGU module that controls the flow of, the optimum system architecture for video compression has been established.

[0018] The invention according to claim 5, memory compatible address configuration block scheme an image is divided into a plurality of blocks for processing the information of the coordinate units of the block (1) comprising an area memory (1) If, coordinate units of the address generation for <br/> accessing the memory (1) the macroblock, block, possible to instruction <br/>及beauty execution control of each hardware module in pixels 5. A centralized control device (3) having a ROM (9) storing the instruction program according to claim 3 or 4.
It is a high-performance code compression system for moving picture information described in 1.

As a processor for performing centralized control of the whole operation such as starting and stopping the operation of each module and controlling the data input / output to the memory (1), the address generation for the memory address is performed in macro block units, in block units. It is possible in units and pixels. By doing so, the independence of the design of each hardware module of the multiple hardware modules can be obtained, the design constraint conditions are significantly reduced, and the multiple designers can share each of them. By designing, the time required to design the entire system can be shortened.

[0020] The invention according to claim 6, in the array processing means for simultaneously processed in parallel by multiple processors, focusing on a particular pixel, by moving vertically and horizontally all over the narrow window (21), the window (21) A predetermined calculation process is executed once for each changing position of the specific pixel to which pixels included in various positions belong, and the predetermined calculation process is performed on the window (21) every time the position changes. as an alternative means of predetermined calculation process to start over several times by moving vertically and horizontally, and stores the image data with your Kume memory (1),
Supply and its memory (1) or llama black buffer for data format conversion to sequentially input the data block for each macro-block (39), the data of a plurality <br/> ports outputted from the buffer (39) Connected in a parallel array
Memory composed of multiple processor elements (101 to 132)
Shared array configuration and its processor elements (101-101)
The arithmetic means of ultra-high-speed parallel arithmetic data 132), the motion vector search circuit comprising an adder for searching for a motion vector representing whether those macroblocks of the current frame has been moved from any position of the previous frame (44) a high-performance code compression system of the moving image information according to any one of claims 1 to 5, comprising the.

With the operation of the invention according to claim 6, among the respective hardware modules, a plurality of processors required for the motion vector search circuit requiring 80% or more of the total processing steps can be efficiently operated. The configuration is established. By doing so, the search data and the reference data can be sequentially input from the external memory (1) and the window parallel processing can be executed without lowering the high parallel efficiency.

Next, the claims7The invention according to
The input data of (1) is changed by the input selector switch (66).
Two sets of registers (62) and (63) that are switched alternately
It is continuously input through and is not 64 pixels in the vertical 8 horizontal 8 convenience.Ru
MaIt becomes the data of the black block, and it is written every macro block.
Reduce the data transfer rate by entering the following
, Two sets of data that convert serial data into parallel data
Data format conversion means (55) and its data format conversion means
Two-dimensional discrete cosine of the parallel data via (55)
The processor that does the conversionelement(101-132) and its
ProcessorelementTwo-dimensional discrete cos from (101-132)
Input the sine transform output and quantize itThe memory
Output data to store data sequentially in (1)
That the quantization module (12) (58) is provided.
CharacterizingClaims 1-6The moving image information according to any one of items
It is a high-performance code compression system for information.

[0023] Then, the next most computational ChiaKing Ru away in modules requiring cosine transform and quantization circuit of the motion vector, in these inverse quantization circuit and an inverse discrete cosine transform module, which is a reverse operation , The data stored in the external memory is sequentially read, processed in parallel, and written without impairing the high-speed operation. By doing so, a means for sequentially inputting pixel data from the external memory and performing parallel processing without reducing the high data transfer rate and the parallel efficiency was established.

[0024] The invention according to claim 8, facilities and Okumemo Li (1) stores the image data, a specific region or preferential information processing movable on the screen of the moving image during the recognition process Window (21) and all the images forming the window (21) are divided into rectangular small blocks, and search data and reference data for each small block are sequentially input from the memory (1). Sequential processing means for executing window parallel processing, and a motion vector search unit (1) for searching a motion vector associated with the motion of the moving image of the small block.
0) and the motion vector searched by the motion vector search unit (10) to estimate the position of the window (21) of the face of the next frame, and follow the window (21) to the motion of the main subject. A high-performance code compression system for moving image information that includes a window position control program
There, the camera input image mark-decoder, to minimize the delay time and drop frame generated until the output becomes a decoded image through a transmission path and a decoder, a frame-level rate control As a program , the communication buffer residual bit amount (W) is set to 0, and the process of encoding the first input screen as one frame (S61) and determining whether the bit amount (B) of the current frame is a positive value If the bit amount (B) is a positive value, the number of screens per output image frame (C) is divided by the number of screens per second (F) of the output image (C / F) Seconds is the encoding process of the current frame Time, and the amount of bits that can be transmitted per second (R) in this encoding processing time (C / F) seconds
And the amount of bits (U) transmitted during this period multiplied by a predetermined value (R
C / F), if the bit amount (B) is 0, it is determined that the current frame is skipped, and the encoding processing time (C / F) seconds is the screen of the input image per second. The same (1 / G) seconds as the input screen cycle which is the reciprocal of the number (G) is set, and the bit amount (U) transmitted during this input screen cycle (1 / G) seconds is set to (R / G). Processing (S62),
A value obtained by adding the bit amount (B) of the current frame to the bit amount (W) remaining in the communication buffer and the bit amount (U) transmitted from the communication buffer during the encoding processing time (C / F) seconds of the current frame. The communication buffer residual bit amount (W)
The process of updating the value (S63), the value (D + 1) obtained by adding 1 to the guaranteed maximum frame delay time (D), and the bit amount (R) that can be transmitted per second and the input screen cycle (1 / G) second multiplied by (R / G) · (D + 1), the value obtained by subtracting 1 from the number of screens per output image frame (C) (C-
A value (R / F) · (C) obtained by multiplying 1) by the transmittable bit amount (R) and dividing by the number of screens per second (F) of the output image.
-1) is set to obtain a variable (L), and the minimum bit amount for using 100% of the bandwidth of the communication path set to the predetermined value (R / C / F) from the variable (L) (E)
And the maximum number of reproducible screens per second (H) of the decoder and the number of screens per second of the output image. (F) and a comparison process (S64) in which the target bit amount (T) of the next frame is calculated at the same time, and the result of the comparison process (S64) is the communication buffer residual bit amount (W) or Number of screens per second of output image (F)
Is smaller (W <LE), the target bit amount (T) of the next frame is set by subtracting the communication buffer residual bit amount (W) from the allocated bit amount (K) per frame. , If the target bit amount (T) is a positive value,
As a normal process, the next (G / F-1) frames are skipped, the subsequent 1 or 2 input screens are encoded,
If the target bit amount (T) value is 0 or negative, the next input screen is skipped without being encoded.
Processing for setting the bit amount (B) of the current frame to zero (S65)
And a comparison step of comparing the bit amount (B) of the current frame with the communication buffer residual bit amount (W) (S62) (S6).
3) and the step (S64) (S65) of controlling the target bit amount (T) of the next frame so that the communication buffer residual bit amount (W) is not exhausted according to the comparison result of the comparison step (S62) (S63). ), And is a high-performance code compression system for moving image information.

The invention according to claim 9 is the frame
Each consisting of vertical 8 horizontal 8 convenience 64 pixels after level rate control
To optimize the quantization level in macroblocks
As a program constituting the macroblock level rate control mechanism, sign bit quantity indicative in order, before the initial value of the quantization level applied to the first macroblock state examining the current frame of frame (Q), before If the first calculation means (S1) for calculating using the average value (Qa) of the quantization level of the coded macroblock of the frame and the target bit amount (T) of the current frame are positive values, the actual If the frame is encoded, otherwise go to the stage (S6) where the processing of all macroblocks is completed (S2)
When a first half of the coding compression processing video information in each macroblock (i) of said all current frame
Image of the decoded I frame or P frame immediately before the frame
The motion of each macroblock obtained by the motion estimator based on
Generates a motion-compensated prediction image with a vector
Processing of performing a discrete cosine transform on the difference image between the predicted image of the current image and the image of the current input frame, and quantizing the frequency component subjected to the discrete cosine transform using the quantization level (q) (S3), and quantization. This is the latter half of the process of updating the level (q) to an appropriate value (S4, S7 to 11) and the coding and compressing process of the moving image information in each macroblock (i). Is updated to a value including the code of the macroblock (i), dequantized to generate a discrete cosine restored image (S5), and the processing of all the macroblocks is finished, and the code of the current frame is After completion, replacing with the initial value (Q ') of the quantization level of the previous frame, the bit amount (B') of the previous frame, and the target bit amount (T ') of the previous frame, and preparing for the processing of the next frame (S6) The condition that the block is not coded is that it is not an intra macroblock and that the quantized frequency component and the motion vector component are all zero, and if it is not coded, the quantization level (q) is not updated. , The determination of whether each macroblock (i) is coded (S7), and the four variables that are indicators of the update calculation of the quantization level (q) when the macroblock (i) is coded A process of calculating the predicted value (d) of the bit amount, the remaining bit predicted value (h), the remaining bit allowable value (a), and the remaining bit target value (e) of a certain macroblock (i) (S8). a), so that the quantization level when the current of the quantization level (q) is greater than the initial value of the first macroblock to apply quantization level (Q) (q) is unlikely more increased and A parameter (b1) is a bias use,
A process (S9) of obtaining a parameter (b2) that is a bias that acts so that the quantization level (q) is less likely to become smaller than the initial value (Q), and the parameter (b
Multiplying 1) by a constant (g) of 0 or more, a constant (f) of 1 or more
The parameter (c1) and the parameter (b2) to which is added is multiplied by a constant (g) of 0 or more to obtain a parameter (c2) of which a constant (f) of 1 or more is obtained (S10), and an updated value in the forward direction. Under the condition that (q1) is a positive value that does not reach the maximum quantization level (qmax) and the update value (q2) in the negative direction does not reach the minimum quantization level (qmin), the current quantization is performed. Level (q) is the initial value (Q)
If the first condition is true that the remaining bit allowable value (a) is smaller than the remaining bit predicted value (h), the quantization level (q) is updated to the update value (q).
The value obtained by adding 1) is used as the value of the variable (q ′), and when false, the second condition is evaluated, and the product of the remaining bit allowable value (a) and the parameter (c1) is the remaining bit. Target value (e)
If the second condition that is smaller than is true, the value obtained by adding q1 to the quantization level (q) is set as the value of the variable (q ′), and if false, the third condition is evaluated, and evaluates the remaining bit target value (e) a parameter (c2) if the third condition the product is made smaller than the remaining bit tolerance (a) is true for the fourth condition, when the false the Quantization level (q)
Is the value of the variable (q '), and the sum of the communication buffer residual bit amount (W) and the current frame bit amount (B) is the input screen frequency.
If the fourth condition that is smaller than the bit amount (U) transmitted during the period (1 / G) seconds is true, the quantization level (q) is added to the update value (q2), and otherwise. Is a process of updating the actual quantization level (q) with the quantization level (q) as it is and false, using the quantization level (q) as the value of the variable (q ′) (S11), And a value clipped by a function (CQ) that stores the value of the variable (q ′) calculated by the evaluation of the first to fourth conditions within an allowable limit is set as the updated value of the quantization level (q). The high-performance code compression system for moving image information according to any one of claims 2 to 8 , wherein
Mu .

In the operation of the invention according to claims 8 to 9 ,
As a result, the frame level rate control and the subsequent macroblock level rate control can exhibit excellent control capability despite a small amount of calculation. Like this ,
Before Symbol the bit length of each image frame in order to average with high precision, very conventionally it has been essential complex calculations,
By replacing the calculation with a simple calculation, the delay time for the calculation process can be reduced and the lip synchronization can be realized.

Further, the communication buffer residual bit amount (W), the current frame bit amount (B), the number of screens per output image frame (C), the number of screens per second of the output image (F), the current frame , The target bit amount of the next frame (T), the transmittable bit amount (R) per second, Number of screens per second of image (G), number of screens of output image per second (F), maximum number of reproducible screens of decoder per second (H), maximum guaranteed frame delay time (D) ), The variable (L) to the predetermined value (R · C /
Minimum bit amount for the band to use 100% of the communication path is set to F) (E), allocation bit amount per frame (K) is approximated to all integer values, the frame level rate
It becomes possible to process the bets control performed on all integer operations,
To facilitate the realization of the hardware.

[0028]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, referring to FIGS.
An embodiment according to the present invention will be described. FIG. 1 is an explanatory view of the window. The face window 21 and the background image 22 are identified and weighted to the image information. The face window 21 is thick, and conversely, the background image 22 intentionally reduces the image quality and the amount of information. Is reduced. Also, the face window 21
Is continuously updated to the latest position by a mechanism that follows the swinging movement of the face.

On the screen of the moving image undergoing the recognition process, all the images constituting the movable specific area, that is, the window 21 of the face to which the information processing is preferentially performed are divided into rectangular small blocks and the sequential processing is performed. Under the method, the position of the face window of the next frame is estimated by using the motion vector associated with the motion of the moving image of the block, and the face window 21 is made to follow the motion of the main subject.

At the beginning of communication, a fixed-shape face window 21 is set at the center position, and among the motion vectors of macroblocks inside the window 21, the average value of non-zero macroblocks is calculated, and The window 21
And the position of the window 21 is updated. Even if the face of the person is not initially located in the center of the screen and the position of the window 21 does not match the face of the person, once a part of the face having the motion vector enters the inside of the window 21, the window is displayed. Due to the action of the motion vector 21 of the window 21, the window 21 gradually approaches the face of the person and finally coincides with the face.

By doing so, the selection criterion for weighting the moving image information is clarified as the window 21, and the window 21 follows the movement of the face, so that the moving image information can be reliably weighted.

Here, the processing in block units required by the ITU standard will be defined. The numbers below indicate the number of pixels in the vertical and horizontal directions. One frame of a moving image is composed of a small unit called a block composed of 8 × 8 pixels, and further, an area composed of a luminance signal Y of 16 × 16 pixels and two color difference signals Cr and Cb of 8 × 8 pixels is a macroblock. Call. Therefore, the macroblock is composed of four adjacent luminance signals Y and one color difference signal Cr and Cb, which is a total of six blocks.

The standard number of pixels in one frame adopted in the ITU standard is a luminance signal Y of 144 × 176 pixels.
And two color difference signals Cr and Cb of 72 × 88 pixels, a quarter common intermediate format QCIF (Quar).
ter Common Intermediate F
orat), CIF (Y: 288 × 352 pixels, Cr / Cb: 144 × 176 pixels) and 4CIF
(Y: 576 × 704 pixels, Cr / Cb: 288 × 35
There are several types, such as 2 pixels), and
All frame sizes permitted to be used by the U standard are targeted, and implementation is performed within the range of restrictions to which the standard for processing signals in macro blocks and block units is applied or applied mutatis mutandis. Improving the image quality within the range of such constraints is the gist of the new invention.

Next, FIG. 2 is an explanatory view of the face window and the peripheral movement window. This is because when the face movement is relatively small, the interlocutor's attention is focused on the hands around the face. It was an attempt to answer an interest. The specific operation involves obtaining a difference between the previous frame image and the current frame image, and performing an operation such that the difference value is equal to or greater than a predetermined threshold value. The target portion such as an arm having the second most importance next to the face is surrounded by 16 ×. The peripheral motion window 51, which is a variable region and is composed of an image block having a size of 16 pixels (hereinafter referred to as a macro block) or an image block having a size of 8 × 8 pixels (hereinafter referred to as a block), has priority. Information processing is performed.

Then, the whole image forming the peripheral motion window 51 is divided into small rectangular blocks and the sequential processing is performed, and the motion vector associated with the moving image of the block is used to move to the next frame. The position and area of the peripheral motion window of are estimated, and the peripheral motion window 51 is made to follow the estimated position and area. The peripheral motion window 51 is different from the face window 21, and is calculated on the principle that the difference between the previous frame image and the current frame image is larger than a certain threshold value, and is changed to an arbitrary shape as appropriate. .

The peripheral movement window 51 adaptively changes the threshold value so that it approaches the window 21 of the face when the face moves strongly and covers the periphery of the face when there is almost no face movement. Especially when the hand is moved, the peripheral movement window 51 can cover the movement of the hand. By doing so, not only the human face as a subject, but also the selection criterion for weighting the moving image information up to the target portion accompanied by the motion of the gesture, as the peripheral movement window 51, In addition, the peripheral motion window 51 can follow the motion of the person's hand or the like, and the moving image information can be reliably weighted to the target portion that is mainly the person's face and is accompanied by a gesture motion.

Next, FIG. 3 is an explanatory diagram of a method of reducing the information amount of the background, and the background is distinguished not only by the window 21 of the face but also by the frame of the peripheral movement window 51. As explained in Section 2. And
When the movement of the background is strong and the amount of moving image information is large, the amount of movement of the background is reduced and the background image quality is intentionally deteriorated. A time direction filter (not shown) that adds and mixes the data of the preceding image frame at a predetermined ratio is reduced, and the moving image information amount of the background is reduced.

Both the face window 21 and the peripheral movement window 51 are thick in both areas, and conversely, the background image 22 (see FIG. 1) other than that has the time required to intentionally reduce the image quality and reduce the amount of information. The directional filter is applied. Here, instead of processing the macroblock itself of the current frame, the previous image frame at the same position (hereinafter, also referred to as “previous frame” or “previous reference frame”)
From the macroblock of and the macroblock of the current frame,
A macroblock derived by weighting and averaging is used. For the average weight, the average value of the pixel values of the previous frame macroblock and the current frame at the same position can be considered. In the extreme case, if only the macroblock of the previous frame is replaced, the average screen looks like a still image. By suppressing the noise and movement of the background image in this way,
Only a small amount of information can be allocated to the background screen, and a large amount of information can be allocated to the face and peripheral areas.

In this way, the image information of a moving picture that consumes a large amount of transmission information passes through the time filter, so that the amount of information can be drastically reduced instead of degrading the image quality appropriately. When the image that has undergone this information amount drastic reduction processing is decoded and reproduced, it is sufficient to give the impression that the image quality is slightly degraded only in the scene that changes in a fast motion. Specifically, when transmitted using this videophone in a running car, the scenery flowing from the car window that appears as the background of the talker is slightly blurred. By doing so, it is possible to identify and appropriately process the case where the background movement is strong and the information amount of which is to be greatly reduced, and to appropriately thin the information amount of the unimportant background moving image to the extent that it does not make it uncomfortable.

Here, the relationship between the I frame, the P frame and the B frame will be described in advance. Since the I frame is encoded using only the image information of the current input frame, the image does not depend on the image quality of the decoded image before that. P
The frame generates a prediction image that is motion-compensated by the motion vector of each macroblock obtained by the motion predictor based on the image of the previous reference frame (the decoded I frame or P frame immediately before the current frame). , The information of the difference image between the predicted image and the image of the current input frame is compressed (discrete cosine transform, quantization, variable length coding) and transmitted to the decoder.

In the decoder, the compression code of the difference image information is decoded (discrete cosine inverse transform, inverse quantization, variable length decoding), and motion compensation is separately performed by the motion vector transmitted from the encoder. The current P frame is decoded by adding it to the predicted image (the same as that generated on the encoder side) (decoded P frame = predicted image + decoded difference image). However, noise due to compression encoding is mixed in the decoded difference image. The image quality of the decoded P frame is
Since it largely depends on the prediction accuracy of the predicted image (the similarity between the predicted image and the image of the current input frame), it is directly affected by the image quality of the previous reference frame that is the source of the predicted image.

That is, if the image quality of the previous reference frame is poor,
The image quality of the current P frame to be decoded is also deteriorated, and the image quality of the subsequent P frames are also chained. On the contrary, the improvement of the image quality of the P frame also improves the image quality of the frame that refers to it and further improves the image quality of subsequent frames in a chained manner. The B frame is based on the images of two reference frames (I frame or P frame decoded immediately before and immediately after the current frame) that are temporally preceding and subsequent to each other. And sends the compression code of the difference image information to the decoder.

In the decoder, a predicted image generated based on the motion vector and the prediction method (forward prediction, backward prediction, bidirectional prediction) information separately transmitted from the encoder by the same decoded difference information as described above. Decode the current B frame by adding (same as that generated by the encoder). B
The difference between the frame and the P frame is that the prediction image is generated based on two reference frames, the prediction accuracy is higher than that of the P frame, and the compression code amount of the difference image is small. Since the frame itself does not become a reference frame, the deterioration of the image quality of the B frame does not adversely affect the image quality of other (subsequent) frames. However, the image quality of the B frame itself is greatly influenced by the image quality of the reference frame, as in the P frame.

Next, FIG. 4 is an explanatory diagram of a method for reducing the amount of information by B frame processing, in which the names of signals and processing in each processing process are described, and the names of functional units of each signal processing are not necessarily described. Not what I did. Therefore, with respect to each processing, only the names of the functional units are designated by reference numerals as necessary. First, in the encoder shown in FIG.
The current frame image, the previous reference frame image, and the subsequent reference frame image are input to the motion prediction function unit 41. The motion prediction function unit 41 performs motion prediction, motion compensation, and prediction method determination. Then, the information of the difference image between the predicted image and the current frame image output from the motion prediction function unit 41 is input to the all-pixel value zeroization function unit 42, and all the pixel values of the difference information are forcibly set to zero. To do.

The zeroed all-pixel information output from the all-pixel value zeroing function unit 42 is the discrete cosine transform unit 4
The signal is input to the code generation unit 45 via the quantization unit 49 via the signal No. 3. The code generation unit 45 encodes the zeroed all pixel information while predicting the next motion of the moving image by the prediction method determined by the motion prediction function unit 41. The overall configuration of the encoder will be described later with reference to the block diagram shown in FIG. 11. FIG. 12 shows a block diagram of a decoder, and FIG. Since it will be described later, the flow of signals in the B frame processing will be shown here to explain the operation thereof.

Next, in the decoder shown in FIG. 4 (b), the decoding unit 46, which receives and decodes the moving image information which is code-compressed by the encoder and transmitted and sent through the transmission path 100, is decoded. The decoded signal is input from the unit 46 to the inverse quantization unit 47 and inversely quantized. Then, the inverse quantized signal output from the inverse quantization unit 47 is converted into the discrete cosine inverse transformation unit 48.
The difference image is restored to the difference image by inputting to the inverse discrete cosine transform, and the restored difference image and the predicted image predicted by the prediction method are added together to output a decoded image.

In the generally used moving image compression method, I which directly encodes the image information of the current input frame is used.
A frame, a P frame that predicts the image information of the current frame from the image information of the previous reference frame, and a P frame that encodes a difference image from the predicted image, and a prediction that is composed from the image information of two reference frames that are temporally preceding and following. There is a B frame that encodes a difference image from the image. The “reference frame” here refers to an already decoded I frame or P frame. Further, the “predicted image” mentioned here refers to an image in which the motion generated between the reference frame and the current input frame is predicted in macroblock units, and the amount of motion is compensated in the reference frame. Recently, H. 263 and H.264. 26
The 3 Plus standard also proposes a PB frame coding method that simultaneously codes a B frame and a subsequent P frame in macroblock units.

There are the following three types of prediction methods for the conventional B frame or B frame in the PB frame coding method. (1) Forward prediction used when predicting the previous reference frame (2) Reverse prediction used when predicting the subsequent reference frame (3) Bidirectional prediction used when predicting the preceding and following reference frames in both directions In order to correct the deviation between the current input frame image and the predicted image, these difference images were encoded and transmitted to the decoder. However, as shown in FIG. It shows a method of drastically reducing the information amount of the B frame by transmitting a zero-value image and substantially transmitting only the three types of B frame prediction methods as information.

As described above, since the B frame does not serve as a reference frame for encoding other frames, even if the image quality of the B frame is deteriorated to some extent, the subsequent frames are encoded. In addition to the above, the information amount of the B frame is drastically reduced, and the information amount of the P frame is increased accordingly, so that the image quality of the P frame is improved. It also improves the image quality of the entire image.

Further, by forcing the differential image of the B frame to be an image of all zero values, the discrete cosine transform section 43 and the quantizing section 49 enclosed by the broken line in FIG. It leads to reduction of wear. It is not necessary to change the configuration on the decoder side by introducing this new B frame encoding method on the decoder side, and compatibility with the conventional standard is completely maintained. By doing this,
When the motion of the moving image is noticeable and intense, the information amount of the difference image also increases, but in order to minimize the information amount of the difference image, all pixel values are forcibly set to zero, The image can be restored at the destination with the least amount of information.

A quantization method for reducing noise in the color difference signal of an image and improving the visual image quality of a decoded image will be described. The quality of the image reproduced by the decoder is generally determined by the amount of noise that is inserted in the process of compression-encoding the luminance signal and the two color difference signals, but human vision is particularly sensitive to the color difference signals. It is known that reducing noise in these color difference signals leads to improvement in visual image quality.

In the conventional quantization method, the frequency component data subjected to the discrete cosine transform is transformed according to the designated quantization level by the following calculation formula. | L | = [(| C | −Q · s · (p + f)) / (Q · s)] where | L | is the absolute value of the quantized data, and | C |
Is the absolute value of the original data, Q is the quantization level, s is the quantization level correction value, p is the inverse quantization correction value, f is the rounding correction parameter, and [] is the conversion operation by rounding down from a real number to an integer. Show.

The inverse quantization correction value p and the quantization level correction value s are specified by each moving image coding standard.
H. 263 and H.H. In the H.263 plus standard, the dequantization correction value p is a value of 0 (in the case of the DC component of the intra macroblock) or 0, 5 (in other cases) depending on the coding method of the macroblock and the type of frequency component data. , And the quantization level correction value s is set to 2. The intra macroblock referred to here is a macroblock that directly encodes the current macroblock image without using the prediction image.

On the other hand, the rounding correction parameter f is not defined by the standard and can be set freely. f = 0,
A value of 5 means quantization by rounding, and a value of f = 0 means quantization by rounding down. In the conventional quantization method adopted in TMF-11, the rounding correction parameter f is 0, 5 (in the case of an intra macroblock) or 0, 5 depending on the coding method of each macroblock.
It is set to 25 (other cases), and the common f value is applied to the luminance signal and the two color difference signals.

In the present invention, by setting different f values for the luminance signal and the color difference signal, the noise of the color difference signal is greatly reduced while applying the same quantization level to the luminance signal and the color difference signal. Has been successful. Specifically, the f value is determined as shown below. (1) In the case of an intra macroblock, f =
Set to 0, 5 (quantization by rounding). (2) In other cases, f = 0 (rounded down) is applied to the luminance signal and f = 0,5 (rounded off) is applied to the color difference signal.

The use of rounding down in the quantization of the luminance signal is a method that has never been considered in the past, because the quantization noise increases. However, the quantization by rounding down the luminance signal has the effect of dramatically increasing the number of quantized frequency components that take a value of 0, which significantly improves the compression ratio and causes a relative decrease in the quantization level. . The decrease in the quantization level offsets the increasing tendency of the quantization noise to some extent, and the final quantization noise is only 0,
It has been confirmed that the level is about 0 dB to 0.3 dB, which is almost the same as that of the conventional method. On the contrary, by applying rounding in the quantization of the color difference signal, the noise of the color difference signal is improved by about 1 to 2 dB to about 1 to 3 dB, and the noise ratio as a whole is improved by about 0 to 3 dB. further,
By significantly reducing the noise in the color difference signals, the color reproduction is more accurate and the visual image quality is also greatly improved.
Therefore, with a simpler structure than the conventional one, the noise reduction effect in the color difference signal can be efficiently exerted, and due to the nature of the human eye, the image quality can be substantially improved visually more than the numerical improvement level of the S / N ratio. To be

Next, here, the DC coefficient of the intra macroblock (the value in the upper left corner of the 8 × 8 block) is quantized, that is, the DC coefficient is divided by 8 and the fraction is rounded off. It is calculated by multiplying the quantized value obtained by 8. Therefore, [inverse quantized value] = 8 × [quantized value]. In addition, the current macroblock (1
The luminance signal and the color difference signal are quantized by rounding only in the case of an intra macroblock that directly encodes a 6 × 16 or 8 × 8) image. Then, in the case other than the intra macroblock, the luminance signal is quantized by rounding down and the color difference signal is quantized by rounding off.

By doing so, the noise of the color difference signal can be reduced while applying the same quantization level to the luminance signal and the color difference signal. In terms of measurement, if the noise of the same level is reduced, reducing the noise of the color difference signal rather than the luminance signal has a higher visual effect. Therefore, with a simpler structure than the conventional one, the noise reduction effect in the color difference signal can be efficiently exhibited,
Due to the nature of the human eye, it contributes to the substantial improvement of the image quality visually than the numerical improvement level of the S / N ratio.

Next, FIG. 5 is a list of conditions and variables relating to the frame level rate, and FIG. 6 is a processing flow chart of the frame level rate control. FIG. 5A shows conditions to be considered in calculating the target bit amount in frame level rate control, and R is the transmittable bit amount per second. The number G of screens per second of the input image is determined by the moving image input source (camera, video) on the encoder side, and generally takes a value of 25 to 30. The number of screens F per second of the output image is a value equal to or less than the number of screens G of the input image per second, and F and G are assumed to be an integer ratio.

Here, the number of screens C per output image frame is C = 2 when the next output frame is the PB frame coding system, and the next output frame is the I frame or P frame.
In the case of the frame coding method, C = 1. The maximum number of reproducible screens H per second of the decoder is F or more. Also, the guaranteed maximum frame delay time D
Is the delay time required from the image input on the encoder side to the image reproduction on the decoder side, with the input screen period (1 / G) seconds as a unit time. This delay time is
It depends on the time required to transmit the coded information of each screen,
Although it also depends on the processing delay times of the encoder and the decoder, these processing delay times are ignored here. still,
The conditions that D must meet are described in the next paragraph.

FIG. 5B shows three variables determined by the conditions of FIG. E is the minimum bit amount for using 100% of the bandwidth of the communication channel. The actual code bit amount is E
When the value falls below the range, a phenomenon called underflow occurs in which the communication buffer storing the output information becomes empty, and the effective value of the communication speed decreases, resulting in deterioration of image quality. Where L
Is the maximum bit amount that can guarantee the frame delay time D. When the actual code bit amount exceeds L, the delay of the reproduced image in the decoder exceeds D. In order to guarantee the frame delay time D while performing stable moving image communication with less underflow phenomenon, L must be a value equal to or greater than E. Therefore, the condition that D in FIG. The equation D ≧ (2 · C−1) G / F−1 is given. K is the amount of allocated bits per frame and is set to a value of E or more and L or less. Note that s here is a constant that takes an effective value between 0 and 1.

FIG. 5C shows variables depending on the code bit amount of each image frame. And FIG. 5 (a) (b)
The variables R, G, F, C, H, D, E, L, which appear in (c),
K, W, B, U and T all approximate integer values. Thus, the following processes frame-level rate control is all it is possible to perform integer arithmetic, to facilitate realization of hardware.

Then, FIG. 6 shows a processing flow chart of the frame level rate control. In FIG. 6 (S61), the communication buffer residual bit amount W is set to 0, and the first input screen is set to 1.
Encode as a frame. In FIG. 6 (S62), it is determined whether the bit amount B of the current frame is a positive value. If B is a positive value, the coding processing time of the current frame is set to C / F seconds, and the bit amount U transmitted during this period is set to R · C / F. If B is 0, it is determined that the current frame is skipped, and the encoding processing time is 1 / G which is the same as the input screen cycle.
Seconds, and the bit amount U transmitted during this period is set to R / G. In FIG. 6 (S63), the value obtained by adding the bit amount B of the current frame to the communication buffer residual bit amount W and the bit amount U transmitted from the communication buffer during the encoding processing time of the current frame are compared to obtain the W value. Update.

In FIG. 6 (S64), the target bit amount T of the next frame is calculated. At this time, the magnitude relationship between W and LE and between F and H is checked. Normally, W <LE is established, and at this time, the value obtained by subtracting the communication buffer residual bit amount W from the allocated bit amount K per frame is set to T. However, if W ≧ LE, the decoder cannot reproduce within the frame delay time guaranteed by the current frame, and a delay longer than the guaranteed delay time temporarily occurs (hereinafter, this is referred to as “excess delay state”). Called). After that, on the encoder side,
This temporary excess delay state K is set to be eliminated, greater than the screen speed F per second of 1 second per renewable maximum screen number H is the output image of the decoder ,
This excessive delay state is also eliminated on the decoder side.

However, when this H is equal to F, even if this temporary excessive delay state is eliminated on the encoder side, it cannot be eliminated on the decoder side, and the excessive delay state may continue. There is. Then, although a temporary excess delay state is hardly perceivable, when the excess delay state persists, the large delay is often a problem. Therefore, when W ≧ LE and F = H, T = 0. And Figure 6
When T is a positive value in (S65), the next (G / F-1) frame is skipped as normal processing, and C (the next frame is I or P frame, C =
In the case of 1, PB frame, the input screen of C = 2) is encoded. If the value of T is 0 or negative, the next input screen is skipped without being encoded. The bit amount B of the frame when skipped is B = 0.

As described above, the delay time required from the image input on the encoder side to the image reproduction on the decoder side can be strictly controlled, and even when a frame drop occurs on the output screen, the conventional C.G. In contrast to the frame dropping of / F screens, the present invention suppresses the frame dropping of only one screen. Therefore, the number of screens dropping the frame is reduced as a whole, and stable communication of moving images becomes possible.

Also, comparing means for comparing the bit amount B of the current frame of the encoded moving image information with the communication buffer residual bit amount W is shown in FIGS. The control means for controlling the target bit amount T of the next frame so that the residual bit amount W is not exhausted is shown in FIG. 6 (S64) (S65). By using the control result by the control means, the delay time and the frame drop occurring until the camera input image is output as a decoded image via the encoder, the transmission path 100 and the decoder are minimized. The calculation means of the target bit amount T of the next frame in the frame level rate control is established. By doing this, phenomena that could not be controlled sufficiently in the past, such as underflow of the communication buffer that reduces the effective communication speed and delay of the reproduced image,
Highly controllable and lip synchronization can be realized by extremely simple calculation.

Next, a macroblock level rate control method using the code bit amount as an index will be described with reference to FIGS. 7, 8, 9, and 10. FIG. 7 is a list of variables and constants related to the macroblock level rate,
Particularly, the average value Qa of the quantization levels of the coded macroblocks of the previous frame is the coded macroblock (not the intra macroblock, the macroblock of which the quantized frequency component and motion vector component are all zero is not coded. Value) divided by the number of macroblocks applied to

As shown in FIG. 8, the first (i
A first calculation means (S1) for calculating the initial value Q of the quantization level applied to the macroblock of (1) using the weighted average Qa of the quantization level of the coding macroblock of the previous frame. ing. In FIG. 8 (S3), the target bit amount and the actual code amount up to the current macroblock are used to calculate the fine adjustment amount of the quantization level applied to the second and subsequent (i = 2, 3 to N) macroblocks. The second calculation means using the quantization level applied to the first (i = 1) macroblock is configured as shown in FIGS. 9 and 10.

FIG. 8 is a processing flow chart of macroblock level rate control. However, this macroblock level rate control is not applied to the one-frame encoding of the first input screen, and when it is not applied, the quantization level is fixed to a specific value. In FIG. 8 (S1), the state of the previous frame is checked. When the previous frame is the first I frame or is skipped (B ′ = 0), the initial value Q ′ of the quantization level of the previous frame is used as it is as the initial value Q of the quantization level of the current frame. In other cases, a value obtained by multiplying the average value Qa of the quantization levels of the coding macroblocks of the previous frame by the ratio B ′ / T ′ of the actual coding bit amount B ′ to the target bit amount T ′ of the previous frame. Is an argument of the function CQ, and the output of the function CQ is the initial value Q of the quantization level of the current frame.

Here, FIG. 9 is a diagram for explaining the definition of the function CQ (x) in the macroblock level rate control. In the function CQ, as shown in FIG. 9, the argument x is the allowable maximum value of the quantization level. It is a "clipping" function that outputs Qmin when it is less than Qmin, and outputs the value as it is when x is within the allowable range. When the actual code bit amount is larger than the target bit amount in the previous frame, the initial value of the quantization level is set higher than Qa, and conversely, when the actual code bit amount is smaller than the target bit amount, Q
By setting it lower than a, the "control speed" of this macroblock level rate control is adaptively adjusted and its control capability is improved.

In addition to this, the predicted value J of the average bit amount per non-zero quantized frequency component, which is required in FIG. 8 (S1), is calculated. In this calculation, a value obtained by dividing the total amount of bits spent on frequency components in the previous frame by the number of non-zero quantized frequency components (in FIG. 7, “average bit amount per non-zero quantized frequency component of previous frame”). The value obtained by adding J and J ′ in the ratio of t: (1−t) is used as a new J value. t is a constant that takes a real value of 0 or more and 1 or less. In FIG. 8 (S2), if the target bit amount T of the current frame is a positive value, the current frame is actually encoded.
Go to 6).

In FIG. 8 (S3), each macroblock (i)
In the first half of the coding / compression process of moving image information in (i = 1, 2 to N), discrete cosine transform is performed on the difference image (see FIG. 4) described above, and the discrete cosine transformed frequency component is quantized. Quantization is performed using the quantization level q. Figure 8
In (S4), the quantization level q is updated to an appropriate value.
Since the details of the processing of this part are shown in FIG. 10,
It will be described later along with it. In FIG. 8 (S5), which is the latter half of the coding / compression process of the moving image information in each macroblock (i), variable length coding is performed, and the value of B is updated to include the code of macroblock (i). To do. Further, inverse quantization is performed to generate a discrete cosine restored image. Figure 8 (S
In 6), after the processing of all macroblocks is completed and the code of the current frame is completed, the values are replaced with the values of Q ′ and B′T ′ to prepare for the processing of the next frame. All the above processing is realized by integer arithmetic.

FIG. 10 shows each macroblock (i)
9 is a flow chart of the update calculation process of the quantization level q in FIG. 8, which is a detailed description of the process in FIG. In FIG. 10 (S7), it is first determined whether each macroblock (i) is coded. The condition that a macroblock is not coded is that it is not an intra macroblock and that the quantized frequency component and motion vector component are all zero. If not encoded,
The quantization level q is not updated.

In FIG. 10 (S8), macroblock (i)
When is encoded, first, four variables d, h, a, and e which are indices for update calculation of the quantization level q are calculated. d
Is a predicted value of the bit amount of the current macroblock (i),
The number z of non-quantized frequency components of the current macroblock (i)
Is multiplied by the predicted value J of the average bit amount per non-zero quantized frequency component calculated in (S1), and the expected bit amount V other than the frequency component is added. As the expected bit amount V other than the frequency component, a value that is experimentally obtained in advance is used. h is a predicted value of the bit amount consumed by the remaining (unprocessed) macroblocks, and is a remaining bit predicted value. a is the remaining bit value that can be consumed after the macroblock (i), and is the remaining bit allowable value. e is a target value of the bit amount consumed by the remaining (unprocessed) macro blocks when it is assumed that each macro block (i) generates the same code bit amount, and is a remaining bit target value. When the bit remaining amount value a is significantly larger than the bit remaining amount predicted value h , the quantization level q is increased to increase the information generation amount, and conversely, the bit remaining amount value a is significantly larger than the bit remaining amount predicted value h. If it is very small, the quantization level q is lowered to suppress the information generation amount.

In FIG. 10 (S9), parameters b1 and b2 are set.
Ask for. b1 is a bias that acts so as to prevent q from becoming larger when the current quantization level q is larger than the initial value Q of the quantization level applied to the first macroblock, and b2 is a bias that q does not exceed Q. It is a bias that acts to make it difficult to be smaller than. FIG. 10 (S1
In 0), the parameters c1 and c2 are obtained. c1 = f + g · b1 c2 = f + g · b2 The constant f used in the calculation here is a parameter for adjusting the sensitivity of the rate control, and usually a value of 1 or more is used. The constant g is a parameter for adjusting the strength of action of the biases b1 and b2, and a value of 0 or more is usually used. Therefore, the parameter (c1) and the parameter (b) obtained by multiplying the parameter (b1) by a constant (g) of 0 or more and adding a constant (f) of 1 or more
Multiply 2) by a constant (g) of 0 or more and a constant (f) of 1 or more
A process (S10) of obtaining the parameter (c2) to which is added is performed.

In FIG. 10 (S11), the actual quantization level q is updated. Here, the following conditions 1 to 4 are considered. Condition 1: q <Q and a <h When true, a value obtained by adding q1 to q is taken as the value of q ′. If false, condition 2 is evaluated. Condition 2: e> a · c1 When true, a value obtained by adding q1 to q is set as a value of q ′. If false, condition 3 is evaluated. Condition 3: If e · c2 <a is true, Condition 4 is evaluated. If false, let q be the value of q '. Condition 4: W + B <U If true, add q2 to q, otherwise leave q as it is. If false, let q be the value of q '. However, it is assumed that 0 <q1 ≦ qmax and qmin ≦ q2 <0 are satisfied. Further, W is the communication buffer residual bit amount shown in FIG. 5C, and U is the bit amount transmitted during the encoding processing time of the current frame shown in FIG. 5C.

Finally, the value of q'calculated by the evaluation of the above four conditions is clipped by the function CQ to be the updated value of q. In addition, H. 263 and H.H. 263 Plus Quantization Level Allowed Maximum Value Q
max is set to 31 and minimum value Qmin is set to 1,
The allowable maximum value qmax of the change amount of the quantization level of two consecutive macroblocks is +2, and the allowable minimum value qm of the change amount is
in is set to -2. All the above processes are realized by integer arithmetic or fixed point. By doing so, in order to average the bit lengths of the image frames with high accuracy, a very complicated calculation, which has been indispensable in the past, is replaced with a simple calculation, thereby performing the calculation process described above. Therefore, the delay time can be reduced and lip synchronization can be realized.

As one embodiment, the System Me that constitutes the high-performance code compression system for moving image information.
more Sharing Processor Ar
A system for memory and a low bit rate video code based on the architecture of a ray, that is, a systematic memory sharing type processor array system (hereinafter, also referred to as system MSPA) will be described with reference to the drawings.

FIG. 11 is an encoder, FIG. 12 is a decoder, and FIG.
FIG. 3 is a block diagram of an integrated encoder / decoder. FIG.
In the figure, 5 is a television camera which inputs a video signal of a moving image mainly composed of a human face to the data bus 2 through the camera interface 6. The data bus 2 is connected to the host computer 8 via the host interface 7,
Address Generation Unit, that is, centralized control unit (hereinafter referred to as AGU) 3 and DRAM @
Data is exchanged with a 13.5 MHz 256 × 16 bit memory (hereinafter, simply referred to as “memory”) 1. Note that, in FIG. 13, the two memories 1 are described because it is an integrated block diagram for both the encoder and the decoder, and therefore, one memory 1 is provided for each of the encoder and the decoder. This is because it is installed. Further, in AGU 3 and FIG. 13, since the test operation is performed in cooperation with the memory 1, it is easy to execute the operation test in the design and trial production stages.
The AGU 3 is connected to a ROM 9 in which a program for basic operation is stored in addition to the memory 1, and transmits a control signal via the control bus 4 to each of the hardware modules (hereinafter, “module” or “each element” or “element”). Also called,
(Illustrated by each element name) to enable the high-performance code compression system for moving images.

As the memory 1 in this embodiment,
Sufficient results have been obtained using a D (dynamic) RAM, and that fact is described in each figure. However, in the present invention, encoded moving image information can be appropriately written and immediately read out. The memory means can be implemented without being limited to the DRAM, and even such an embodiment can naturally be considered to be included in the gist of the present invention.

The respective elements share the image information of the moving image via the data bus 2, and the data transfer between the respective elements is performed only through the memory 1 connected to the control bus 4 outside the chip. Data transfer between each element and memory 1 is AGU3
And the ROM 9. This is the greatest feature of the system MSPA, and by performing all the data transfer between the respective elements via the memory in this way, when the AGU 3 makes a memory access to the memory 1 for the processing dependency between the respective elements, It can be determined only by the timetable of the time division processing.

There are elements shown in FIGS. 11 to 13 as the above-mentioned respective elements, each of which has an independent design through the data bus 2 and the control bus 4 without considering the processing dependence between the respective elements. Connected in parallel to form the whole. Therefore, different designers can proceed simultaneously for each of the above-mentioned elements, so that the time required for the design can be short even though the program configuration of the entire system is large. Further, if there is a design change in each of the above-mentioned elements, it is possible to flexibly deal with it by changing the program of the AGU 3. The motion vector search unit 10 includes a prediction determination unit (not shown), calculates an average movement amount of the image of the face window 21, and moves the face window 21 following the average movement amount. Then, the movement of the near future is predicted from the average movement amount in order to move the area of high image quality without delay in accordance with the swing of the photographed person's face, and the face window 21 is advanced.

Hereinafter, the window MSPA will be mainly described. A memory 1 that stores image frame information and a hardware module that operates independently of each other are coupled via a data bus 2, and a data flow and an operation schedule between the memory 1 and each of the hardware modules. The AGU 3 for controlling the ACU 3 constitutes the encoder and the decoder by a system architecture in which the AGU 3 is coupled to each of the hardware modules via a control bus 4.

In the videophone of this embodiment,
It has a configuration in which the encoder / decoder combined as shown in the block diagram of FIG. FIG. 11 shows the encoder and FIG. 12 shows the decoder separately from each other, and their duplicated configurations are omitted as appropriate. The encoder system shown in FIG. 11 has an external address of 18 bits and data of 16 bits.
Memory with megabit capacity and access time of 40 nanoseconds 1
Therefore, data for four frames of QCIF (176 × 144 pixels) can be stored. In FIG. 11, while the data from the television camera 5 is stored in the memory 1, it is similarly sequentially compressed for each macro block of 16 × 16 pixels.

First, the motion vector search unit 10 searches from which position in the previous frame the macroblock to be processed has moved, and outputs it as a motion vector. At this time, with respect to macroblocks that do not belong to the face window 21 or the peripheral motion window 51, the image information is deteriorated by the action of a time filter (not shown). Also, when the image information of a still image is input to the time filter or the like, the original image information is output as it is, and conversely, when the image information of a video with a lot of movement is input, the movement is eased. Information is manipulated as described above. In this way, the image information of a moving image that consumes a large amount of transmission information passes through the time filter, so that the amount of information can be sharply reduced instead of degrading the image quality appropriately. When the image that has undergone this information amount drastic reduction processing is decoded and reproduced, it is sufficient to give the impression that the image quality is slightly degraded only in the scene that changes in a fast motion. Specifically, when transmitted using this videophone in a running car, the scenery seen from the car window reflected as the background of the talker is slightly blurred.

Further, the motion compensation unit 11 uses the obtained motion vector to generate difference data between the macroblock of the current frame being processed and the region of the previous frame where it is supposed to have moved, Write to memory 1. In the discrete cosine (inverse) transform unit / (inverse) quantizer 12, the difference data read from the memory 1 is divided into 8 × 8 pixel blocks, which is a quarter of the macroblock, by a discrete cosine transform, and 8 × 8 pixels are obtained. The frequency component of 8 pixels is obtained, and the quantization operation for changing the number of representation bits is performed at high speed based on the quantization step for each macroblock, and the result is output to the memory 1.

In the variable length encoder 13, the memory 1
An appropriate code is assigned to the frequency component of the quantized and bit-reduced difference data that is read from, and stored in an internal buffer (not shown). The internal buffer outputs encoded data to the outside at a constant transmission rate. Further, the discrete cosine (inverse) transforming unit / (inverse) quantizing unit 12 is, for example, 35
One image consisting of 2 × 288 pixels is divided into pixel blocks of 8 × 8 pixels, and DCT (Discrete Cosi)
ne Transform) Discrete cosine transform (hereinafter,
Frequency components are decomposed (orthogonal transformation) by DCT transformation), high-frequency terms are rounded and information is compressed, and each coefficient after DCT transformation is divided by a certain divisor to round the remainder.
These function in the forward direction in the encoding process and in the reverse direction in the decoding process. Since the decoder of FIG. 12 does not have the forward transform, the discrete cosine inverse transform unit / inverse quantization unit 12b
Is provided.

The discrete cosine (inverse) transformer / (inverse) quantizer 12, the P macroblock reconstructor 14, the B macroblock estimator 15a, and the block distortion eliminator 16 convert the quantized frequency component from the quantized frequency component. It functions in the process of reconstructing the current macroblock and is used for compression processing of the next frame. Then, the discrete cosine inverse transform unit / inverse quantization unit 12
Then, as the inverse operation of the discrete cosine and the quantization, the quantized data of the frequency component of the difference data is input from the memory 1, the original bit is restored by the inverse quantization operation, and the original cosine inverse transform is performed. Restore to differential data,
Store the result in memory 1.

The P macroblock reconstructing unit 14 is common to FIGS. 11, 12, and 13, and here, the previous frame data obtained from the differential data and the motion vector is read from the memory 1 and added. By doing so, the macroblock data of the current frame is restored and written in the memory 1.

Further, in FIG. 13, the B macroblock estimation unit /
There is a reconstructing unit 15, which processes P frames and B frames, respectively. The encoder shown in FIG. 11 has a B macroblock estimating unit 15a, and the decoder shown in FIG. 12 has a B macroblock reconstructing unit 15b. Are installed. B in the encoder
The macroblock estimation unit 15a reads the P macroblock reconstructed from the memory 1 and the macroblock of the previous frame to which it has moved from the memory 1, and mixes both with the B macroblock data estimated forward from the previous frame. Then, the B macro block estimated is constructed, and the similarity with the B macro block actually input from the television camera 5 is compared. Which of these is the best is determined for each macroblock, and only that information is transmitted via the variable length encoder 13. In the decoder, the B macroblock reconstructing unit 15b actually reproduces B frame data.

The block distortion removing filter 16 is shown in FIG.
1, FIG. 12 and FIG. 13, the P macroblock restored from the memory 1 is read out, and the operation of the block distortion removal filter 16 removes the grid-like noise that is unsightly at the joint between macroblocks. Write to memory 1.

The AGU 3 also controls the execution of each module and the exchange of data between each module and the memory 1. The AGU 3 operates in accordance with a program stored in the ROM 9, and its program instructions include address generation of the memory 1 and access control of the memory 1.

The host interface 7 is the AGU.
In place of the ROM 9 of No. 3, an instruction from the external host computer 8 is input to the system, execution control of each module and data transfer between the memory 1 and the host computer 8 are performed. As described above, in the window MSPA, the hardware modules (each element) for executing all the respective functions at high speed are connected to the external memory 1 via the data bus 2. That is, each element receives input data from the external memory 1, processes the data, and outputs the result to the external memory 1 again. Therefore, there is no direct data transfer between the above-mentioned elements, and the memory 1
Through. Further, the AGU 3 controls the data transfer between each of the above elements and the memory 1 in accordance with a command from the ROM 9.
The two memories 1 are allocated to the encoder and the decoder, respectively, but the number of memory 1 need not be two as long as a total of two response processing capacities are provided.

This is the greatest feature of the window MSPA. By thus performing all data transfer between the respective elements via the memory in this way, the processing dependency between the respective elements is AGU3 of the access to the memory 1. It can be decided only by the timetable The processing dependency between the above elements is AGU3.
Since each element is designed independently, it is connected in parallel via the data bus 2 and the control bus 4 to form the whole. By doing so, the independence of the design of each of the plurality of each element can be obtained, the constraint condition on the design can be remarkably reduced, and a plurality of designers can share the respective designs at the same time. As a result, despite the large scale of the entire system, the time required for its design is short.

The processing dependency of each element is AGU3.
Can be flexibly designed by the program. In the actual design, it is necessary to finish the processing of each P frame and B frame macroblock in 15,625 clocks. However, the motion vector search process requires a long processing time although the access time of the memory 1 for data input / output is short, and the motion vector search process as a whole is 13,0.
Time of 00 clocks is required. Since the other processes are each configured by the hardware mechanism, all the remaining 1
Processing within 5,625 clocks is possible. The motion vector search processing and other processing are pipeline parallel processing for each macroblock. That is, the compression processing of 100 macroblocks included in one frame is sequentially executed, but after the motion vector search processing is executed within 15,625 clocks, the motion vector search processing of the next 15,625 clocks is started. The point is.

In this way, the motion vector search process and other processes are simultaneously executed in most cycles. These pipeline processes can be flexibly designed by the architecture of the present invention by the program of AGU3.

Further, since all the input / output data of each element are stored in the memory 1, the test from the outside can be simplified. Actually, by issuing a command from the AGU 3 from the host computer 8 via the host interface 7, the data in the memory 1 is read into the host computer 8 after the data setting in the memory 1 is executed by the host computer 8 and the processing of each element is executed. It is possible to inspect. Further, by providing an inspection program that allows the host computer 8 to read the internal data of each of the above-mentioned elements, the inspection is made easy.

The above-mentioned respective elements have no dependency on their respective operations, and the clocks are supplied only to the elements that operate, and it is not necessary to operate the other elements. Therefore, the power consumption of the entire system is reduced. It can be reduced.

Next, a description unique to the decoder will be given with reference to FIG. The frequency domain differential data decoded by the variable length decoder 17 is actually restored by the B macroblock reconstructing unit 15b and the block distortion removal filter 16 of the encoder to restore B frame data. Then, similarly to the encoder, each of the above-described independently operating elements receives data from the memory 1, processes the data, and then returns the data to the memory 1 for storage. AGU3 controls these controls. The final reproduced image is displayed through the LCD interface 18.
It is displayed on the LCD 19 connected to the outside.

Next, FIG. 14 shows a centralized control unit (AGU3).
Is a block diagram of the execution / test mode changeover switch 30 is determined by the command (PC command) from the host computer 8 via the host interface 7 to determine either the execution mode or the test mode. By the instruction decoding execution control device 31,
Execution instruction, memory read / write repeat instruction control unit 32 and repeat instruction start address register 3
By the operation of 3, the address-controlled data exchange with the memory 1 is performed. By these, the address of the memory 1 is generated, the memory access control signal is generated, the calculation of each element is started, and the stop signal is generated. The repeat instruction start address register 33 and the register file 34 are mainly composed of general-purpose registers.
Group, one set of eight 2-bit registers, and one set of eight 1-bit registers (details not shown).

FIG. 15 is a block diagram of the memory area of the external memory 1. The image is divided into a plurality of blocks corresponding to the instructions of the ROM 9 storing the instruction program of the execution control of each hardware module. However, the address configuration is adapted to the block system for processing the information of the block coordinate unit. As shown in FIG. 15, the AGU3 is used so that the row address and the column address are designated by the memory address.
It is controlled by the memory read / write repetitive instruction control unit 32.

An external memory 1 which is the memory area,
An instruction that enables address generation for memory access in the coordinate unit, block unit, and pixel unit of the macroblock is stored in the ROM 9. This ROM 9 is shown in FIG.
It may be arranged outside the AGU 3 as shown in FIG. The actual hardware layout is usually different from those shown in these block diagrams.

The memory space of the memory 1 is the address 1
As shown in FIG. 15, 8 bits represent the upper 1 bit of the frame, 4 bits representing the X coordinate and Y coordinate of the macroblock position, a 9-bit row address consisting of 4 bits and the lower bit of the frame, and the X coordinate of the block position. 2 representing the Y coordinate
Bit and 1 bit, 2 representing the X and Y coordinates of the pixel position
It is delimited by a 10-bit column address consisting of 3 bits. However, in addition to the four areas represented by (0,0) (0,1) (1,0) (1,1) for the information of the luminance signal Y, the block position of the color difference signals Cr, Cb A block for information is assigned to the area (0, 2) (1, 2). Also, since the data is 16 bits, X
One address is assigned to 8-bit data of two pixels adjacent in the coordinate direction. Therefore, X of the pixel position
Although the coordinates are for 8 pixels, only 2 bits are assigned.

The instruction is represented by a length of 27 bits. (1) Memory access start address instruction (2) Memory read loop instruction (3) Memory write loop instruction (4) AGU register control instruction (5) Subroutine Instructions and conditional branch instructions (6) Includes special instructions issued by the host computer. The memory access start address command of (1) is issued before executing the loop command for memory reading and writing, and sets the start address for executing the loop command. As a result, it is possible to specify an absolute address, a relative position from the macro block currently being processed, and a shift for the motion vector from the macro block currently being processed. The memory read / write loop instructions of (2) and (3) are repeated at a plurality of macroblock levels of a rectangular area, at a plurality of block levels of a rectangular area, and at a plurality of pixel levels of a rectangular area. Because it has a loop mechanism for
Normally, a loop control that is complicatedly described by a repeat statement can be easily performed by using the memory read / write loop instructions of (2) and (3). For example, reading and writing certain frame data, reading and writing certain macroblock data, and reading and writing pixels in a certain rectangular area can be executed by one memory read / write loop instruction. (4)
The AGU register control instruction is an instruction for setting or clearing data of an auxiliary register used for controlling the execution order of modules (elements) by the AGU 3.
The subroutine instruction and conditional branch instruction in (5) are AGU
3 is an instruction for program control. The special instruction issued by the host computer 8 in (6) is an instruction valid only in the test mode in which the instruction from the host computer 8 is accepted instead of the execution mode operating based on the program ROM 9. It also includes step execution instructions for executing the specified number of steps. In the test mode, all the commands (1) to (3) can be issued from the host computer 8.

Therefore, in the test mode, the image data is written from the host computer 8 to the memory 1, the element converted into the execution mode is operated, the mode is returned to the test mode again, and the operation result written in the memory 1 is stored in the host. It can be read by the computer 8. In this way
It becomes possible to verify the operation at each element level. The special instruction issued by the host computer 8 includes an instruction for reading the internal state of each element, and the operation verification of each element at the circuit level is enabled by the same method.

Now, the "motion vector search module (element)" will be described. In the motion vector search, for each macroblock of the current frame to be processed, the most similar 16 × 16 pixel area is searched around the position of the macroblock in the previous frame. The actual search is performed at a resolution of half the pixel using interpolation processing in a 48 × 48 pixel range that moves up to 16 pixels in the vertical and horizontal directions from the current position. Any 16x1 within this range
From the difference of each pixel between the 6-pixel region and the pixel of the macroblock of the current frame, the sum SAD of their absolute values
(Sum of Absolute Difference
e) (hereinafter referred to as SAD). This operation is performed for all possibilities, and the smallest one is found, and that area is set as the position in the previous frame to which the processing macroblock has moved. Also, the position of the processing macroblock and the motion vector are obtained from the position. This operation is 4
A 16 × 16 window area is set in the 8 × 48 search area, and the window processing for obtaining the SAD is performed on the entire search area, which is one form of window processing unique to image processing.

Conventionally, there is a window MSPA as a construction method suitable for this window processing. In the present invention, search data and reference data are sequentially input from a memory without lowering the high parallel efficiency which is a characteristic of the window MSPA. , Performs window parallel processing. In this way, the problem of speeding up the processing in the conventional hardware implementation of the moving image compression processing is solved.

Next, FIG. 16 is a motion vector search circuit diagram, in which an external memory 1 for storing image data and a data format conversion for successively inputting the macroblock data from the memory 1 for each macroblock are performed. Buffer 3 for
9 is connected to the data bus 2 and exchanges data with each other. The 3-port data output from the buffer 39 via the internal bus 40 is the processor elements 101, 102 to 13 connected in the 32 parallel array form.
2 is supplied to the window MSPA.

A motion vector search circuit for searching for a motion vector indicating from which position in the previous frame the macroblock of the current frame has moved by the adder 44 for performing ultra-high speed parallel calculation of the data of the processor elements 101 to 132. Are configured. That is, as shown in FIG. 16, the motion vector search circuit includes processor elements 101, 102 to 132 via a memory 1 connected to a data bus 2, a buffer 39 for data format conversion, and an internal bus 40.
And an adder 44. For example, two pixel values of search data and reference data are input to the processor elements 101 to 132, subtraction and absolute value operation are performed,
The operation of adding to the SAD up to a certain point is performed, and the window MSPA composed of the 32 processor elements 101 to 132 operates in parallel to perform 1089 times of window processing. 16 × 16 = in one window processing
Since 256 SAD processes are required, 1089 × 256 = 278,784 SAD operations are required for the entire search.

In practice, 32 processor elements (also referred to as processor arrays) perform SAD operations in parallel in one clock, so 278,784 times / 32 = 8,712 clocks are required at the fastest. Although it is not possible for 32 processor arrays to operate at all times, this system makes it possible to perform motion vector searches in a time of 13,000 clocks.

By doing so, the means for sequentially inputting the search data and the reference data from the memory 1 and executing the "window parallel processing" without deteriorating the high parallel efficiency was established. What is "window parallel processing"?
This is a process that is widely used in image processing, and means that the narrow window 21 is moved up and down, left and right all over a wide screen area to perform image recognition and information processing, but focusing on a specific pixel, Since it is included in various positions in the window 21, if the predetermined calculation process is executed once for each position to which the pixel belongs, the predetermined calculation process is repeated every time the position changes. There was a waste.

Therefore, instead of moving the window 21 up, down, left and right to repeat the predetermined calculation process many times, in order to avoid the waste of the above-mentioned duplication, the plural processes are simultaneously executed in parallel to prevent the duplicate calculation. An array processing method that improves parallel efficiency has been established by the window MSPA. The term "parallel efficiency" as used herein means ideally a case where the processing time is shortened to 1 / n times as much as the case of processing one by one when simultaneously processing in parallel by n processors.
The efficiency is defined as 0%, which is not the case. Further, the array processing method for increasing the "parallel efficiency" by avoiding the duplicate calculation has already been published, and the description thereof will be omitted.

Next, in the motion vector search circuit diagram shown in FIG. 16, the "ready-made window MSPA" used in the "window parallel processing" is "motion vector search".
It is a novel invention that has never been seen in the prior art. This “motion vector search” is performed only for the luminance signal Y in units of 16 × 16 pixel macroblocks, and searches from which position in a previous frame a certain macroblock has moved.

Next, FIG. 17 is a block diagram of a discrete cosine transformer and a quantizer. The data bus 2 has a DRA.
In addition to M1, the input terminal of the inverse quantizer 53 is connected to one of the three input terminals of the three-way selection switch 54, and the output terminal of the two-way selection switch 59 is connected. The output terminal of the inverse quantizer 53 and the output terminal of the transposition operation buffer 57 are connected to the remaining two input terminals of the changeover switch 54. The output end of the changeover switch 54 is connected to the input end of the data format converter 55. The output terminal of the data format converter 55 is connected to the input terminal of the discrete cosine transform / inverse converter 56. Discrete cosine transform /
At the output end of the inverse transformer 56, the input end of the transposition operation buffer 57, the input end of the quantizer 58, and the changeover switch 5
One of the two input terminals 9 is connected. The output terminal of the quantizer 58 is connected to the other input terminal of the changeover switch 59. Here, the forward and reverse operations of the discrete cosine transformer and the quantizer will be described. The block data of 8 × 8 = 64 pixels stored in the memory 1 is sequentially input to the inverse quantizer 53 and the discrete cosine transform / inverse transformer 56 via the data bus 2. Inverse quantizer 5
3 and quantizer 58 are highly pipelined modules that can process one data in one clock cycle, and are designed so as not to impair high speed operation.

Further, the data format converter 55 is a converter for the discrete cosine transform / inverse converter 56 from the bit parallel data format to the bit serial data format. The discrete cosine transform / inverse transform 56 decomposes the ordinary two-dimensional processing into two one-dimensional processing, and the transposition operation buffer 57 performs the transposition operation of 8 × 8 data required at this time. Therefore, details of the data format converter 55 are shown in FIG. 18, and the operation will be described. First, 16-bit bit parallel data is input from the input line 61 every clock, and is stored in 16 registers in total in the registers 62 and 63. When eight data are collected in the register 62 or the register 63, the data are simultaneously output from these registers in the bit serial form through the connection lines 64 and 65, and the input changeover switch 66 is used to register the register 62 or the register 6 with each other.
3 output data is selected.

The data format converter 55 will be further described below by focusing on the fact that the data format converter 55 is provided with two sets of registers 62 and 63. By using the two sets of registers 62 and 63, two processes of receiving the input data to the data format converter 55 and transferring the data group from the lower digit to the upper digit accompanied by the shift operation are simultaneously executed in parallel, and We are trying to speed up the process. In FIG. 18, the operation will be described by dividing the operation timing into 8 clocks. 1) The first 8
In the clock, 16-bit data is transferred to the serial register 62.
Data 0 to 7 are input in this order. 2) In the next 8 clocks, not the data input to the register 62, but the 8 pieces of data obtained by collecting only the least significant bits of the registers of the 8 sections are output from the registers, and at the same time, the respective data of the registers of each section are the lower bits. Is shifted in the direction. Eight parallel data thus input are collected and sequentially output from lower bit data to upper bit data. If data storage in the register 62 is prohibited during this period, the register 63 needs to store the next 8 clocks instead. In this way, to the data format converter 55,
The two sets of registers 62 and 63 are alternately used by the operation of the input changeover switch 66 to continuously receive the input data from the memory 1 and to speed up the data processing by the pipeline parallel, that is, without lowering the data transfer rate. , Two sets of data format conversion means for converting sequential data into parallel data.

The data transfer rate referred to here is the speed of how many bits of data can be transferred per second. In addition, in the case of sequential data,
Bits are transferred bit by bit. For example, when a data transfer rate of 1 million bits / second is required, it is sufficient to use 10 lines having a data transfer rate of 100,000 bits / second in parallel.

Further, as shown in the block diagram of the discrete cosine transform / inverse converter in FIG. 19, eight bit permutation data are input to the input processing unit 71, passed through eight 16-bit cumulative adders 72, and finally processed. Input to the output processing unit 73.

Further, as shown in detail in the block diagram of the input processing unit in the discrete cosine transform / inverse transformer in FIG.
One bit serial input data is input to the bit serial adder 81 and the bit serial subtractor 82, and these outputs are input to the eight bit dispersion calculation ROM 83. In addition, 8
One input data is directly another 8 bit RO for variance calculation
Input to M84. These two sets of ROM outputs are selected by a changeover switch 86 controlled by a changeover signal 85. As the switching signal 85, “0” is input when the discrete cosine transform is executed, and “1” is input when the discrete cosine inverse transform is executed.

Further, as shown in detail in the block diagram of the output processing unit in the discrete cosine transform / inverse transformer in FIG.
One bit parallel input data is input to the bit parallel adder 91 and the bit parallel subtractor 92. These two sets of RO
The M output is selected by the changeover switch 94 controlled by the changeover signal 93. As the switching signal 93, “0” is input when the discrete cosine transform is executed, and “1” is input when the discrete cosine inverse transform is executed. The output of the changeover switch 94 is input to the eight registers 95, and is sequentially output from these registers through the output line 96.

Now, the following signal processing will be described before transmitting the video signal mainly composed of the face of the talker photographed by the TV camera attached to the videophone to the receiver through the telephone line. Incidentally, since the bidirectional communication is performed, the information transmission for the round trip of the transmission and the reception is performed at the same time on the same route, but the description of the bidirectional communication is omitted here. Traditional ITU
International standard H.264. It is based on a signal processing circuit for low bit rate moving image transmission conforming to H.263 standard. As a function to be added to the base, a mechanism for extracting a face portion from a video signal to configure a face window 21 and moving the face window 21 according to the movement of the face, and a movement of a background portion other than the face There is a time filter function that suppresses noise and a new mechanism for controlling the transmission rate with a small transmission delay. By these, appropriate compression signal processing is performed on the image information of the moving image.

As a function to be added to the base, first, a video signal inputted by a television camera or the like is subjected to recognition processing,
There is a calculation program for calculating an average value of motion vectors generated by the pixels forming the window 21 of the face, which is a movable specific area on the screen of the moving image, along with the movement of the face. When the listener listens to the visual field with his or her naked eyes, the listener's face follows the movement of the speaker's face, so that the speaker's face is always positioned at the center of the listener's visual field. This action has the highest resolution near the center of the visual field of the human eye, so position it in the center of the visual field to clearly see the object of interest, and collect the maximum value without leaking valuable and useful information. It is an instinctual, unconscious or conscious behavior that feels necessary. In this way, the action of attempting to position the speaker's face at the center (not at the center of the screen) of the window 21 of the face where the listener's eyes have excellent resolution is equivalent to the electronic device having the first intelligence. An arithmetic program for calculating the average value of the motion vectors of the face window 21 including the face of the person in order to replace it with a mechanical device, and the face window 2 corresponding to the average value.
The window position control program for moving 1 works.

As the second intelligence, the face window 21 is used.
In the case where the background image 22 other than the above has a large movement and there is a lot of moving image information, there is an encoding algorithm which is a calculation program that considerably reduces the movement amount of the background image 22 and deteriorates the image quality. This arithmetic program suppresses the encoded moving image information amount by replacing the image information of the subsequent frame with a value obtained by adding up the information indicating each pixel at the same position in the preceding and succeeding frames and dividing by 2 is doing. By suppressing the amount of information, the face enhancement type H.264 has a control mechanism of an output transmission rate that minimizes an image transmission delay in a transmission path having a limited transmission capacity. The H.263 plus encoding algorithm and the function of the program that can change the execution order determination and data transfer timing make the face look clear and natural, and the background image 22 deteriorates in quality. Is not extreme and can be slightly blurred.

The face window 21 supplements the outline of the face of the talker substantially at the center, and the background image 2 other than the face window 21 is supplemented.
The amount of information of 2 is made coarse, the amount of information in the window 21 of the face is kept to a minimum, and the total sum of the image information amounts of these video signals is minimized. This makes the amount of information applicable to the restriction conditions of the telephone line. Further, the window 21 continues to be updated to the latest position by the face window position control program which moves the window 21 of the face in accordance with the movement of the face which oscillates as the talker speaks. Therefore, only the face is always an image with clear and natural movement that is close to the face of a person watching on a home black and white television, and unnecessary information in the background can be compressed to the maximum, so transmission efficiency is high. Can be maintained.

Next, since the amount of information that can be transmitted by the videophone is fixed, it is necessary to thoroughly compress moving image information that does not affect the viewer. And, even if the information that cannot be partially ignored has a small effect on the whole, it is possible to increase other important information by intentionally deleting it, and consequently improve the overall performance.

Further, the zeroization of the difference all pixel information in the B frame information and the quantization by rounding down the data of the luminance signal other than the intra macro block can also significantly reduce the amount of moving image information. Although these operations themselves cause the deterioration of the image, the information amount of other data can be increased by the amount of the reduced information amount, and as a result, the image quality is improved.

Next, regarding the lip synchronization method for matching the movement of the lips with the sound, a newly developed "rate control method" is used to form one frame, that is, a moving image component image (25 or 30 images per second on a television). The code amount per hit is leveled, and the predetermined code amount, for example, 112 bytes, and the amount of information that can be transmitted per unit time, for example, a transmission rate of 27 Kbs.
If the communication delay time that can be predicted from the above is already inserted into the “rate control mechanism” and it is predicted that the communication delay time of 10 frames always occurs, for example, the future is predicted from the change amount from the past to the present. And process the information. Since the future is predicted within a range in which there is no practical problem even if some disappointment occurs, the processed information according to the embodiment of the present invention is applied to a voice received without delay. We have succeeded in keeping the image delay within 3 frames.

Even in a videophone which incorporates a lot of these intelligences, if the hardware for realizing the intelligences does not operate at high speed, the only method is to reduce the number of frames per second. In such a TV image with dropped frames, it is no longer a natural dialogue. The newly developed window MSPA, its constituent elements such as the AGU centralized controller, the motion vector search circuit, the discrete cosine transform / quantization circuit, and the dequantization circuit / discrete cosine inverse transform have the target hardware in a circuit scale. It shows a circuit that achieves its purpose with a small, low-cost circuit.

In short, in order to make the videophone look clear and natural, (a) weighting on the importance of each object, emphasizing only the part to be seen, the face. (B) Reduction of useless information amount to make the entire screen clear. (C) Minimize the delay time that occurs in image processing in order to create a natural atmosphere by synchronizing the screen and voice. (D) A low-cost and high-speed hardware configuration for creating a screen that operates at a natural speed. It is an image information transmission system consisting of.

The videophone is only one embodiment of the present invention, and the applicable range of the efficient code compression system for moving image information is endless. Then, the image quality of the part that is considered to be of little interest to the user is degraded, and instead, the image quality of the part where the interest is concentrated is maintained, and the image quality of the wasteful part that flows too quickly is minimized. Even if compressed to the amount of information, the atmosphere of the image will not be spoiled for the original purpose of image transmission,
The realization of natural movement can be regarded as included in the requirements of the present invention.

[0132]

According to the present invention, it is easy to be interested.
Only the conspicuous face part as a specific area (window)
A code compression system that gives priority information processing
Support conventional telephone lines with limited transmission capacity .
With minimal amount of image information transmission compressed as much as possible, for natural conversation
A close facial expression, especially lip movement that matches the talking, that is, lip synchronization
High-performance code compression system for moving image information
To provide a videophone, which is extremely simple and
High performance at low cost.

Since the present invention is a high-performance code compression system for moving picture information constructed as described above, it can be used in an established telephone line before an optical fiber to exert its maximum effect. In other words, it has been constructed on the premise of efficiently transmitting only the voice with a limited frequency band, and it remains practical even within the limits of the information transmission capacity of the conventional telephone network including wireless. In addition to lip-synchronizing the transmission of the transmitted television image with the voice transmitted along with it, the delay of the image transmission is almost eliminated from the absolute speed of the electromagnetic wave that arrives at the light speed, unless it is an astronomical distance. , It was possible to have a nearly natural TV conversation.

The present invention, which is effective in the conventional telephone line network, is also effective in the future optical fiber network. That is, it is possible to increase the number of subscribers of the communication by the videophone using the present invention in the transmission path of high speed and large capacity. Therefore, it brings the convenience to the general public.

In addition, the system of the present invention can be applied to, for example, 128M.
If it is applied to a recording / playback device combined with a bit DRAM, a moving image containing sound can be compressed to a data amount of 34 Kbits per second and can be recorded and played for one hour, so that a tape drive device of a conventional video tape recorder is not required, The recording / reproducing device can be manufactured at a low price. For the same reason as above,
The ROM image reproducing device can be easily spread. The application range of these is from children's toys to audiovisual materials, daily necessities and public works, and their convenience is immeasurable.

[Brief description of drawings]

FIG. 1 is an explanatory diagram of a face window.

FIG. 2 is an explanatory diagram of a face window and a peripheral movement window.

FIG. 3 is an explanatory diagram of a method for reducing the amount of background information.

FIG. 4 is an explanatory diagram of a method of reducing the amount of information by B frame processing.

FIG. 5 is a list of conditions and variables relating to a frame level rate.

FIG. 6 is a processing flowchart of frame level rate control.

FIG. 7 is a list of variables and constants related to macroblock level rates.

FIG. 8 is a processing flowchart of macroblock level rate control.

FIG. 9 is a diagram illustrating the definition of a function CQ (x) in macroblock level rate control.

FIG. 10 is a flow chart of update calculation processing of a quantization level q in a macroblock (i).

FIG. 11 is a block diagram of an encoder.

FIG. 12 is a block diagram of a decoder.

FIG. 13 is a block diagram of an integrated encoder / decoder.

FIG. 14 is a block diagram of a centralized control unit (AGU).

FIG. 15 is a configuration diagram of a memory area of an external memory.

FIG. 16 is a motion vector search circuit diagram.

FIG. 17 is a block diagram of a discrete cosine transformer and a quantizer.

FIG. 18 is a block diagram of a data format converter.

FIG. 19 is a block diagram of a discrete cosine transform / inverse transformer.

FIG. 20 is a block diagram of an input processing unit in a discrete cosine transform / inverse transformer.

FIG. 21 is a block diagram of an output processing unit in a discrete cosine transform / inverse transformer.

[Explanation of symbols]

1 memory 2 data bus 3 AGU 4 control bus 8 Host computer 9 Program ROM 10 Motion vector search unit 21 Face window 22 background image 39,57 buffers 41 Motion Prediction Function Unit 42 All pixel value zeroization function unit 45 Code Generator 46 Decoding section 47 inverse quantizer 48 Discrete Cosine Inversion Unit 44 Adder 49 Quantizer 51 Peripheral motion window 55 Data format converter 56 Discrete Cosine Transform / Inverse Transformer 101, 102, ..., 132 Processor element

─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Kazuto Ito 255 Shimookubo, Urawa-shi, Saitama Department of Electrical and Electronic System Engineering, Saitama University (72) Inventor Tomohiko Otsuka 1220-2, Kusada-cho, Hachioji-shi, Tokyo Tokyo Institute of Technology, Department of Electrical Engineering (72) Inventor Trio Adino 2-12-1, Ookayama, Meguro-ku, Tokyo Tokyo University of Technology Graduate School of Science and Engineering Department of Electrical and Electronic Engineering (72) Inventor Chawalit Honsawaik 2-12-1 Ookayama, Meguro-ku, Tokyo Within the Department of Electrical and Electronic Engineering, Graduate School of Science and Engineering, Tokyo Institute of Technology (56) Reference JP-A-6-168330 (JP, A) JP-A-10-275237 (JP , A) JP 10-63855 (JP, A) JP 10-51755 (JP, A) JP 9-185708 (JP, A) JP 2000-295600 (JP, A) D ngju Li (and three others), Mu ltimedia LSI Desig n Based on Window- MSPA Architecture, ISPACS'99, the United States, IEEE, 1999 December 8, p. 187-190 (58) Fields investigated (Int.Cl. ⁷ , DB name) H04N ^7/ 24-7/68 H03M 7/30

Claims (9)

(57) [Claims] 1. A stores image data Okumemo Li (1)
And a window (21) that is movable on a screen of the moving image being recognized, that is, a window (21) on which priority information processing is performed, and the entire image forming the window (21) is divided into rectangular small blocks. Then, the sequential processing means for sequentially inputting the search data and the reference data for each of the small blocks from the memory (1) to execute the window parallel processing, and the motion vector associated with the motion of the moving image of the small blocks, The motion vector search unit (10) to be searched and the motion vector searched by the motion vector search unit (10) are used to display the face window (21) of the next frame.
Estimate the position of the
1) a window position control program that follows
A high-performance code compression system for moving image information, comprising a motion prediction function unit that inputs a current frame image, a previous reference frame image, and a subsequent reference frame image to perform motion prediction, motion compensation, and prediction method determination, An all-pixel value zeroization function unit that inputs difference information between a prediction image output from the motion prediction function unit and the current frame image and forcibly sets all pixel values of the difference information to zero, and all pixel values thereof Code generation for inputting the zeroed all-pixel information output from the zeroization function unit and encoding the zeroed all-pixel information while predicting the next motion of the moving image by the prediction method determined by the motion prediction function unit And a B frame
It includes an encoder for processing and input a decoder for receiving and decoding the encoder codes the compressed video information which is forwarded via the transmission path is transmitted by the decoded signals output from the decoding section And a dequantization unit for dequantizing the signal, a dequantization signal output from the dequantization unit, and a discrete cosine inverse transformation unit that decimates the discrete cosine and transforms it to a difference image, and its decompression. Bei addition unit, the outputting the decoded picture adding the predicted image predicted by the prediction method and the difference image
In addition , the present invention includes a decoder for B-frame processing, and quantizes the luminance signal and the color difference signal by rounding off only in the case of an intra macroblock which directly encodes the current macroblock image without using the prediction image. color difference signals quantized is quantized by rounding the devaluation luminance signal when a non includes noise reduction means for reducing the noise of the color difference signal while applying the same quantization level into a luminance signal and a color difference signal, said noise The reduction means multiplies the sum of the inverse quantization correction value (p) and the rounding correction parameter (f) by the quantization level (Q) and the quantization level correction value (s) to obtain the discrete cosine transformed frequency component. The absolute value of the original data consisting of (| C |)
The absolute value of the quantized data obtained by rounding down the real value obtained by dividing the value obtained by subtracting from the value obtained by multiplying the quantization level (Q) and the quantization level correction value (s) (| L | ) and comprises quantization means for, and, as said quantization means, a quantization by rounding off a luminance signal and a color difference signal only in the case of an intra macroblock directly coding the current macroblock image without the predictive picture the correction parameter to the first quantization means for setting the (f) to 0,5, to quantize the devaluation the luminance signal in the case of other than the intra-macro block, the correction parameter (f) was set to 0, the color difference signals to quantized by rounding, characterized by comprising a second quantizing means for setting said correction parameter (f) in 0,5 kinematic
High-performance code compression system for image information. 2. Comparing means for comparing a bit amount of encoded moving image information with a communication buffer residual bit amount, and target bits of a frame so that the communication buffer residual bit amount is not exhausted by the comparison result of the comparing means. It occurs until the camera input image is output as a decoded image via the encoder, the transmission path and the decoder by using the control means for controlling the amount and the control result by the control means. The high-quality moving image information according to claim 1 , further comprising: a calculation unit for calculating a target bit amount of a frame in frame level rate control, which minimizes a delay time and frame drop .
Noh code compression system. 3. A first calculation means for calculating a quantization level applied to a first macroblock of a frame by using a weighted average of the quantization levels of respective macroblocks of a previous frame, and second and subsequent macros. a fine adjustment amount of the quantization level applied to the block, the target bit amount, a second calculating means for calculating using the actual code amount and the quantization level applied to the first macroblock to the current macroblock high-performance code compression system of video information according to claim 2, characterized in that it comprises a. 4. A memory (1) for storing image frame information.
And a hardware module, which operates independently of each other, via a data bus (2), and the memory (1)
Centralized control device (3) for controlling the flow of data and the operation schedule between the hardware module and each of the hardware modules
Is provided with an encoder and / or a decoder configured by a system architecture coupled to each of the hardware modules via a control bus (4).
High performance of the moving image information according to any one of claims 1 to 3.
Code compression system. 5. A memory (1) comprising an area of a memory (1) having an address configuration adapted to a block system for dividing an image into a plurality of blocks and processing information in coordinate units of the blocks.
If, coordinate units of the address generation for accessing the memory (1) the macroblock, block, ROM that stores instructions及beauty instruction program execution control of each hardware module that enables a pixel unit ( 9)
A high-performance code compression system for moving image information according to claim 3 or 4 , further comprising a centralized control device (3) provided with. 6. The array processing means for simultaneously processed in parallel by multiple processors, focusing on a particular pixel, by moving vertically and horizontally all over the narrow window (21), various positions in the window (21) A predetermined calculation process is executed once for each changing position of the specific pixel to which the pixel included in, and the predetermined calculation process is performed by moving the window (21) up, down, left and right every time the position changes. as an alternative means of predetermined calculation process to start over even degree, the image data stored in your Kume memory a (1), the memory (1) or llama black data format for conversion to sequentially input the data blocks for each macro block the buffer (39), a plurality of processes <br/> Tsu support elements connected in parallel an array supplied with data of a plurality ports outputted from the buffer (39) (101-1 Memory sharing type of array consisting of 2)
Adder for searching for a motion vector indicating from which position in the previous frame the macroblock of the current frame has moved, by means of a configuration and an operation means for performing ultra-high speed parallel operation of the data of the processor elements (101 to 132) a motion vector search circuit consisting of (44), characterized by comprising a billing
A high-performance code compression system for moving image information according to any one of Items 1 to 5 . 7. The input data of the memory (1) is continuously input through two sets of registers (62), (63) which are alternately switched by an input selector switch (66), and is vertically 8
By horizontal 8 becomes data luma black blocks on account 64 pixels successively inputted for each macro block without lowering the data transfer rate, sequential converts data into a parallel data two sets of data format conversion A processor element (1) for performing a two-dimensional discrete cosine transform of the parallel data through a means (55) and a data format conversion means (55) thereof.
01-132) and the two-dimensional discrete cosine transform output from the processor element (101-132), and quantizes the output.
Moving image according to any one of claims 1 to 6, characterized in that it comprises a quantization module for data output for sequentially storing data in Li (1) (12) (58), the High-performance code compression system for information. 8. stores image data Okumemo Li (1)
And a window (21) that is movable on a screen of the moving image being recognized, that is, a window (21) on which priority information processing is performed, and the entire image forming the window (21) is divided into rectangular small blocks. Then, the sequential processing means for sequentially inputting the search data and the reference data for each of the small blocks from the memory (1) to execute the window parallel processing, and the motion vector associated with the motion of the moving image of the small blocks, The motion vector search unit (10) to be searched and the motion vector searched by the motion vector search unit (10) are used to display the face window (21) of the next frame.
Estimate the position of the
1) a window position control program that follows
A high-performance code compression system of video information with the camera input image mark-decoder, the delay time frame which occurs until it is outputted as a decoded image through a transmission path and decoder As a frame level rate control program that minimizes dropout, a process of setting the communication buffer residual bit amount (W) to 0 and encoding the first input screen as one frame (S61)
Then, it is determined whether the bit amount (B) of the current frame is a positive value, and if the bit amount (B) is a positive value, the number of screens per output image frame (C) is the number of screens per second of the output image ( (C / F) seconds divided by F) is used as the encoding processing time of the current frame, and this encoding processing time (C / F) seconds is multiplied by the amount of bits (R) that can be transmitted per second and transmitted during this period. The bit amount (U) to be generated is set to a predetermined value (R · C / F), and if the bit amount (B) is 0, it is determined that the current frame is skipped, and the encoding processing time (C / F) Second is input image 1
The same (1 / G) seconds as the input screen cycle, which is the reciprocal of the number of screens (G) per second, and the bit amount (U) transmitted during this input screen cycle (1 / G) seconds are (R / G ) Is set to (S62), and communication is performed between a value obtained by adding the bit amount (B) of the current frame to the communication buffer residual bit amount (W) and the encoding processing time (C / F) seconds of the current frame. The bit amount (U) transmitted from the buffer is compared, and the communication buffer residual bit amount (W) is compared.
A process of updating the value (S63), a value (D + 1) obtained by adding 1 to the guaranteed maximum frame delay time (D), and a bit amount (R) that can be transmitted per second and an input screen cycle (1 / G) Value multiplied by seconds (R / G) ・ (D +
From 1), the number of screens per output image frame (C)
A value (R / F) · (C-1) obtained by multiplying the value (C-1) obtained by subtracting 1 from the above by the amount of bits (R) that can be transmitted and dividing by the number of screens per second (F) of the output image. ) Is set to obtain a variable (L), and the predetermined value (R.
C / F) comparison and / or decoding of the magnitude relationship between a value obtained by subtracting the minimum bit amount (E) for using 100% of the communication channel bandwidth and the communication buffer residual bit amount (W) Comparison between the maximum number of reproducible screens per second (H) and the number of output screens per second (F), and at the same time the target bit amount (T) of the next frame is calculated. When the result of the processing (S64) and the comparison processing (S64) is smaller in the communication buffer residual bit amount (W) or the number of screens per second (F) of the output image (W <LE), A value obtained by subtracting the communication buffer residual bit amount (W) from the allocated bit amount (K) per frame is set as the target bit amount (T) of the next frame. When the target bit amount (T) is a positive value, , As normal processing, the next (G / F-1) frames are Skipped, and then the 1 or encoding two input screen, the target bit amount (T) value is zero or negative if the current in the case of skipping skips without encoding the one next input screen Processing for setting the bit amount (B) of the frame to zero (S6
5) and a comparing step of comparing the bit amount (B) of the current frame with the communication buffer residual bit amount (W) (S62) (S63).
And a step (S64) (S65) of controlling the target bit amount (T) of the next frame so that the communication buffer residual bit amount (W) is not exhausted according to the comparison result of the comparison steps (S62) (S63). A high-performance code compression system for moving picture information, characterized by comprising: 9. Vertical 8 after the frame level rate control
Quantum in each macroblock consisting of 64 pixels in the horizontal direction
A macro block level rate control mechanism that uses the code bit amount as an index to optimally adjust the coding level.
As a gram, before the initial value of the quantization level applied to the first macroblock state examining the current frame of frame (Q), the average value of the quantization level of the coded macroblocks of the previous frame (Qa) using If the target bit amount (T) of the current frame is a positive value, the current frame is actually encoded; otherwise, all macroblocks are processed. (S2), which is the first half of the coding / compression process of the moving image information in each of the macroblocks (i) among all the macroblocks (i) ,
The image of the immediately preceding decoded I frame or P frame
And the motion vector of each macroblock obtained by the motion estimator.
The motion-compensated prediction image is generated by
A process of performing a discrete cosine transform on the difference image between the measured image and the image of the current input frame, and quantizing the frequency component subjected to the discrete cosine transform using the quantization level (q) (S
3) and the process of updating the quantization level (q) to an appropriate value (S4,
S7 to 11) and the latter half of the coding compression processing of the moving image information in each macroblock (i), variable length coding is performed, and the value of B is updated to include the code of macroblock (i). Processing to generate a discrete cosine restored image by inverse quantization (S
5), after the processing of all the macroblocks is completed and the coding of the current frame is completed, the quantization level initial value (Q ′) of the previous frame, the bit amount (B ′) of the previous frame, and the previous frame The step of preparing for the processing of the next frame (S6) by replacing with the value of the target bit amount (T ′) and the condition that the macroblock is not coded are that the quantization frequency component and motion vector component are all zero, not the intra macroblock. And that the quantization level (q) is not updated when not encoded,
There are four variables that are indexes for determining whether or not each macroblock (i) is coded (S7) and for updating the quantization level (q) when the macroblock (i) is coded. The predicted value (d) of the bit amount of the macroblock (i),
Remaining bit prediction value (h), the process of calculating the remaining bits tolerance (a) and the remaining bit target value (e) and (S8), the current of the quantization level (q) is the first macroblock A parameter (b1) which is a bias that acts so as to make it difficult for the quantization level (q) to become larger when it is larger than the initial value (Q) of the applied quantization level.
And a process (S9) for obtaining a parameter (b2) that is a bias that acts so that the quantization level (q) is less likely to become smaller than the initial value (Q), and a constant of 0 or more for the parameter (b1). Parameter (c1) obtained by multiplying (g) by a constant (f) of 1 or more, and parameter (c2) obtained by multiplying parameter (b2) by a constant (g) of 0 or more and adding a constant (f) of 1 or more. To obtain (S
10) and the update value (q1) in the positive direction is the maximum quantization level (qma
x), which is a positive value and does not reach the updated value (q
Under the condition that 2) is a negative value that does not reach the minimum quantization level (qmin), the current quantization level (q) becomes smaller than the initial value (Q) and the remaining bit allowable value (a) Is smaller than the remaining bit prediction value (h)
If the condition is true, the value obtained by adding the update value (q1) to the quantization level (q) is used as the value of the variable (q '). If the condition is false, the second condition is evaluated to determine the remaining amount. Bit allowance (a)
And the parameter (c1) is smaller than the remaining bit target value (e), the second condition is true, the value obtained by adding q1 to the quantization level (q) is a variable (q ′). )
The value, if false evaluates the third condition, the remaining bit target value (e) and parameter product of the third made smaller than the remaining bit tolerance (a) of (c2) conditions If true, the fourth condition is evaluated, and if false, the quantization level (q) is used as the value of the variable (q '), and the communication buffer residual bit amount (W) and the current frame bit amount (B). Sum of input images
If the fourth condition that is smaller than the bit amount (U) transmitted during the surface period (1 / G) seconds is true, the quantization level (q) is added to the update value (q2) In the case where the quantization level (q) is left as it is, and when it is false, the actual quantization level (q) is updated using the quantization level (q) as the value of the variable (q ′) (S11). , And a value clipped by a function (CQ) that stores the value of the variable (q ′) calculated by the evaluation of the first to fourth conditions within an allowable limit is used as an updated value of the quantization level (q). The high-performance code compression system for moving image information according to any one of claims 2 to 8 , characterized in that
Mu .