JP2005295215A - Moving image encoding device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve the image quality of an attentional region required in a monitoring camera or the like by suppressing the encoding amount of a static region low and encoding a moving region in high definition. <P>SOLUTION: Whether each region is a moving region or a static region is determined on the basis of the magnitude of a motion vector "MV" and a difference value "DIFF" obtained on the basis of an input frame and a reference frame. A moving part such as a figure part is determined as a moving region to skip smoothing processing, performing encoding processing such as DCT (discrete cosine transform) processing and quantization processing. A nonmoving part such as a background part is determined as a static region to perform smooth filtering processing, subsequently performing DCT encoding processing such as DCT processing and quantization processing. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a moving image coding apparatus that increases the transmission efficiency by coding a moving image with a smaller code amount, and in particular, reduces the code amount by filtering processing for a still region, and reduces the code amount of a motion region accordingly. The present invention relates to a moving image encoding apparatus that increases the image quality by increasing the number.

  As a moving image encoding apparatus that encodes a moving image, an apparatus shown in FIG. 7 is conventionally known.

  In the moving image encoding apparatus 101 shown in this figure, an input image signal is stored in a frame memory 102, and a signal within a frame or between frames is encoded.

  In intra-frame coding (hereinafter referred to as `` intra coding ''), only the image signal in the frame is independently encoded, and in inter-frame coding (hereinafter referred to as `` inter-coding ''), immediately before, A prediction signal using the immediately following frame as a reference frame is generated, and a prediction error is encoded via the subtraction unit 103.

  In intra coding, an image signal is converted into a DCT code by a DCT unit (discrete cosine transform unit) 104, passes through a quantization unit 105, is encoded by a variable length coding unit (VLC) 106, and is stored in a buffer unit 107. Later it is stored or transmitted. The code quantized by the quantization unit 105 is reconstructed by the inverse quantization unit 108 and the IDCT unit (inverse discrete cosine transform unit) 109 and stored in the frame memory unit 111 via the addition unit 110.

  In the inter coding, a motion detection unit (ME) 112 detects a motion vector of an image signal stored in the frame memory unit 102 while referring to the image signal stored in the frame memory unit 111, thereby predicting a signal. And the prediction error is supplied to the subtraction unit 103. Then, it is converted into a quantized value by the DCT unit 104 and the quantizing unit 105, encoded by the variable length coding unit (VLC) 106 together with the motion vector, stored in the buffer unit 107, stored, stored, or transmitted. . Also, the quantized code is supplied to the adding unit 110 via the inverse quantization unit 108 and the IDCT unit 109, reconfigured by the motion compensation unit (MC) 113, and stored in the frame memory unit 111.

  In accordance with the procedure described above, in inter coding, the amount of code is reduced using the correlation between adjacent frames. At that time, in order to limit the code amount to the specified amount according to the bandwidth limit at the time of code transmission and the capacity limit at the time of storage, usually depending on the complexity of each macroblock and the size of the prediction error with adjacent frames The code amount control is performed by adjusting the quantization parameter so that the code is encoded with the code amount within the limit. For this reason, when there is little correlation between adjacent frames, in order to suppress the amount of generated codes, generally quantization is coarsely performed, and image quality deteriorates.

In general, the visual sensitivity of the stationary region is higher than that of the moving region. Using this property, for example, in Patent Document 1 (Japanese Patent Laid-Open No. 62-164392) and Patent Document 2 (Japanese Patent Laid-Open No. 63-121374 (prefilter control based on the magnitude of a motion vector)), The high frequency component is removed according to the magnitude of the motion vector. That is, by blurring a moving region with low visual sensitivity, the correlation between adjacent frames is increased and the amount of codes is reduced.
JP 62-164392 A JP-A-63-121374

  By the way, when transmitting video information obtained by a surveillance camera to a recording device provided in a surveillance room, error accumulation due to DCT calculation mismatch between codecs and early normal image output at the start of encoded data reception are realized. Therefore, it is necessary to periodically insert intra-coded images in order to minimize the adverse effects of missing images due to communication errors on the transmission path.

  However, since such an intra-encoded image is encoded only with an image signal within the frame, the amount of code increases. Therefore, when the bandwidth of the communication line is limited, the code of the moving image is increased accordingly. There is a problem in that the amount must be suppressed and the overall image quality deteriorates.

  Furthermore, when intra coding is performed, there are many images with fine patterns, such as merchandise display shelves in the store monitoring system and trees outdoors in the civil engineering monitoring system, and the code amount burden is large even in the static region. . In particular, when encoding is performed at a low bit rate, a large amount of code is required for a fine picture portion. As a result, if intra coding is periodically inserted, the amount of code assigned to inter coding is reduced, and the image quality is significantly reduced.

  However, in surveillance cameras, images of moving areas are generally more important than still areas, and there is a strong demand to transmit or record higher-definition images for moving areas. Improvement is desired. In other words, even if the definition of the still region is slightly reduced, it is required to view the region with movement in higher definition instead. For example, in intra coding, a fine pattern image that is a code amount burden may be lowered in the case of a still region.

  Therefore, instead of handling images equally in the still and moving regions, the code amount of the still region is reduced so that a larger amount of code can be allocated to the moving region, or more intra-coded images are inserted. There was a need for a technique to make it possible.

  In view of the above circumstances, the present invention distinguishes an image for each region according to the degree of motion, performs a filtering process on a static region, and removes high frequency components, thereby reducing the amount of codes and increasing the amount of motion region. It is an object of the present invention to provide a moving image encoding apparatus that can improve the image quality of a moving area by assigning the amount of codes.

  In order to achieve the above object, according to a first aspect of the present invention, in a moving image coding apparatus that divides each frame image into blocks and sequentially codes each frame image, the present invention corresponds to the previous frame image encoded. A still frame that divides the current frame image into a motion region and a still region based on a motion vector and a difference value obtained by performing a matching process on the reference frame image and the current frame image to be encoded next. The region / motion region determination unit and the region determined to be a motion region by the still region / motion region determination unit are skipped, and the region that is determined to be a still region is filtered. A smoothing filter unit and an encoding unit for encoding the output of the filter unit with a preset algorithm are provided.

  Further, in claim 2, in the video encoding device according to claim 1, a plurality of motion vector threshold values and a plurality of difference threshold values are set in the still region / motion region determination unit, In accordance with the threshold value, the static region is determined to have a plurality of levels, and the filter unit includes a plurality of filters, and for each region determined as the static region, a plurality of filters are prepared for each region. The filter processing is performed by selecting a filter according to the determined level.

  According to the moving image coding apparatus of the present invention, the image is distinguished for each region according to the degree of motion, and the high-frequency component is removed by performing filtering on the still region, thereby reducing the code amount and moving image. The image quality of the moving area can be improved by assigning a larger amount of code to the area.

  In addition, the bandwidth of the still area is further limited to reduce the amount of generated code burdened by low bit rate encoding, improve image quality degradation caused by the insertion of intra encoding, and periodically perform intra encoding. Error accumulation due to DCT calculation mismatch between codecs can be eliminated, and early normal image output at the start of encoded data reception and early restoration from missing images due to communication errors during code data transmission can do.

<< First Embodiment >>
FIG. 1 is a block diagram showing a first embodiment of a moving picture coding apparatus according to the present invention.

  In the moving image encoding device 1a shown in this figure, when performing inter encoding, the motion detection unit (ME) 3 stores the frame image stored in the frame memory unit 4 while referring to the frame image stored in the frame memory unit 2. The motion vector of the frame image input this time is detected and a prediction signal is generated. Then, the motion compensation unit 5 generates a prediction error from the prediction signal, supplies this to the subtraction unit 8 together with the motion vector, and the DCT unit 9 and the quantization unit 10 convert the prediction error into a quantized value. Next, the quantized value is encoded together with the motion vector by the variable length coding unit (VLC) 11, stored in the buffer unit 12, and then stored or transmitted. In addition, the quantized code is supplied to the adder 15 via the inverse quantizer 13 and the IDCT unit 14, and the frame image is reconstructed in consideration of the prediction error from the motion compensator (MC) 5. , It is stored in the frame memory unit 4.

  In addition, when performing intra coding, the difference unit 16a refers to the frame image stored in the frame memory unit 4 while referring to the difference value of the frame image input this time and stored in the frame memory unit 2. This is detected and supplied to the still area / motion area determination unit 6a to determine whether each area is a motion area or a still area. Then, based on the determination result, the smoothing filtering process is performed on the still region portion by the filter unit 7a, and then the subtraction unit 8, the DCT unit 9, the quantization unit 10, and the variable length coding unit (VLC) 11 are used. After being encoded and stored in the buffer unit 12, it is output to the outside.

  As described above, in the conventional coding example, the motion vector is obtained by the motion detection unit only in the case of inter coding. In the first embodiment, the reference image that is the previous frame is similarly used in intra coding. Perform block matching. However, in intra coding, the motion vector detection of the motion detection unit 3 is not performed, and each block of the frame image stored in the frame memory unit 2 by the difference unit 16a and the previous frame stored in the frame memory unit 4 A difference value from the same position block of the image is obtained.

  Next, the still area / motion area determination process and the smooth filtering process used in the first embodiment will be described in detail.

  First, for the motion vector, a difference value from the previous frame is obtained for each block by a general method, and the motion vector “MV (x, y)” is determined from the position where the difference value is minimized. At this time, the minimum difference value is set to “DIFF”. In the inter coding, the motion vector “MV” and the difference value “DIFF” are input to the still region / motion region determination unit 6a. In intra coding, the difference value “DIFF” is input to the still region / motion region determination unit 6a to determine whether the region is a still region or a motion region.

As a determination method performed by the still region / motion region determination unit 6a, as shown in the flowchart of FIG. 2, when performing inter coding, the motion vector “MV” corresponding to the current input image and the difference value “DIFF” "(Step S1, S2), when the magnitude of the motion vector" MV "exceeds a preset motion vector threshold" MV _THD "(step S3), or the difference value" DlFF " Exceeds the preset difference threshold value “DIFF _THD ” (step S4), it is determined as a motion region (step S5).

When the magnitude of the motion vector “MV” is equal to or less than the motion vector threshold “MV _THD ” and the difference value “DlFF” is equal to or less than the difference threshold “DIFF _THD ” (steps S3 and S4), the motion vector “MV” is stationary. A region is determined (step S7).

In addition, when intra coding is performed, the capturing of the motion vector “MV” corresponding to the current input image is skipped, and only the difference value “DIFF” is captured (steps S1 and S2), and the difference value “DlFF” is captured. "" Exceeds the difference threshold "DIFF _THD " (step S4), it is determined as a motion region (step S5), and the filtering process by the filter unit 7a is skipped (step S6). Further, when the difference value “DIFF” is equal to or less than the difference threshold value “DIFF _THD ” (step S4), it is determined as a still area (step S7).

  Next, based on the determination result of the still region / motion region performed by the still region / motion region determination unit 6a, the filter unit 7a performs a smoothing filtering process using a smoothing filter on the still region of the input image (step S8). At this time, as the smoothing filter, for example, a filter that averages and replaces 2 × 2 pixels constituting the block as shown in the schematic diagram of FIG. With this smoothing filter, it is possible to reduce the code amount by quasi-halving the resolution of the image and reducing the high-frequency component.

  Further, as a smoothing filter that can be used in the filter unit 7a, other filters, for example, those using a simple average value as shown in the schematic diagram of FIG. 3B, or the schematic diagram of FIG. An average value by weighted addition as shown may be used. Further, a well-known edge-preserving smoothing filter or the like may be used in order to avoid the static region from being uniformly blurred and the edge portion from being deteriorated.

  What kind of filter is adopted is selected depending on the hardware and software to be implemented. In general, when the size of the filter is increased, the amount of processing increases. Therefore, when mounting on hardware with low performance, an increase in the amount of processing is prevented by using a small filter. Further, when mounting on hardware equipped with a special engine for the spatial filter, a high-performance filter can be selected and configured.

  As a result, when an input frame and a reference frame are input as an input frame and a reference frame, for example, as shown in the schematic diagrams of FIGS. 4A and 4B, the magnitude of the motion vector “MV” Based on the difference value “DIFF”, as shown in the schematic diagram of FIG. 4C, the determination of the motion area / still area is performed for each area (in this example, a 16 × 16 pixel macroblock area). A moving part, such as a part, is determined as a motion area, the smoothing process is skipped, and other background parts are determined as a still area, and a smoothing filter process is performed.

  As described above, in the first embodiment, each region is a motion region or a static region based on the magnitude of the motion vector “MV” and the difference value “DIFF” obtained based on the input frame and the reference frame. The moving part, such as a person part, is determined to be a motion region, the smoothing process is skipped, the encoding process such as the DCT process or the quantization process is performed, and the background part is not moving. Since the part is determined to be a static region, smoothing filter processing is performed, and then encoding processing such as DCT processing and quantization processing is performed. It can be encoded, and the image quality of the attention area required in a surveillance camera or the like can be improved.

<< Second Embodiment >>
FIG. 5 is a block diagram showing a second embodiment of the moving picture coding apparatus according to the present invention. In this figure, parts corresponding to those in FIG. 1 are denoted by the same reference numerals.

  In the moving picture encoding apparatus 1b shown in this figure, the input original picture is stored in the frame memory unit 2. As for the inter-coding, the original image stored in the frame memory unit 2 and the reference stored in the frame memory unit 4 by the motion detection unit (ME) 3 as in the first embodiment shown in FIG. The frame image is compared to determine a motion vector, and a prediction signal is generated. Then, the motion compensation unit 5 generates a prediction error from the prediction signal, which is supplied to the subtraction unit 8 together with the motion vector, and the prediction error is converted into a quantized value by the DCT unit 9 and the quantization unit 10. Next, the quantized value is encoded by the variable length coding unit (VLC) 11 together with the motion vector, accumulated in the buffer unit 12, and then stored, stored, or transmitted. Further, the quantized code is supplied to the adding unit 15 via the inverse quantization unit 13 and the IDCT unit 14, and the frame image is reconstructed in consideration of the prediction error from the motion compensation unit (MC) 5. It is stored in the frame memory unit 4.

  For intra coding, the original image stored in the frame memory unit 2 is delayed by one frame and transferred to the frame memory unit 21. Then, the difference unit 16b compares the previous frame image stored in the frame memory unit 21 with the current input image stored in the frame memory unit 2 to obtain a difference value. It is input to the motion region determination unit 6b, and it is determined for each region whether it is a motion region, a first still region, or a second still region. Then, based on the determination result, after the first smoothing filtering process and the second smoothing filtering process are performed on the first still area part and the second still area part by the filter unit 7b, the subtraction unit 8 The data is encoded through the DCT unit 9, the quantization unit 10, and the variable length coding unit (VLC) 11, stored in the buffer unit 12, and then output to the outside.

  That is, the moving image encoding device 1a shown in FIG. 1 uses the reconstructed reference frame image stored in the frame memory unit 4 in the same manner as the inter encoding when obtaining the difference value. Therefore, the image quality of the reference frame image deteriorates due to the accumulation of quantization error, and as a result, the difference value error increases. In view of this, in the moving image encoding apparatus 1b, the difference value is obtained by using the previous original image stored in the frame memory unit 21, and the accuracy of the difference value detection is increased.

  In this moving image encoding device 1b, the frame memory unit 21 for storing the previous input image is required. Therefore, when the memory capacity is limited, the memory amount is reduced by configuring as shown in FIG. Although it is necessary, when a large-capacity memory can be used, it can be configured as in the present embodiment to improve accuracy.

In the moving image encoding device 1b, a plurality of motion vector threshold values “MV _THD ” and difference value threshold values “DIFF _THD ” are set in the still region / motion region determination unit 6b, and the motion region is determined. When determining the still region, the still region is divided into several stages according to the motion vector “MV” and the difference value “DIFF”, and the smoothing filter process is performed by the filter unit 7b.

For example, as shown in the flowchart of FIG. 6, the first motion vector threshold “MV _THD1 ”, the first difference threshold “DlFF _THD1 ”, the second motion vector threshold “MV _THD2 ”, the second difference a threshold value "DIFF _THD2" provided conditions so as to satisfy the "MV _THD1> MV _THD2", the value of the first motion vector threshold "MV _THD1", the second motion vector threshold "MV _THD2" And the value of the first difference threshold “DlFF _THD1 ” and the value of the second difference threshold “DIFF _THD2 ” are adjusted so that the condition “DIFF _THD1 > DFF _THD2 ” is satisfied. To do.

When inter coding is performed, the motion vector “MV” corresponding to the current input image and the difference value “DIFF” are captured (steps S21 and S22), and the magnitude of the motion vector “MV” is the first. When the motion vector threshold “MV _THD1 ” is exceeded (step S23), or when the difference value “D _lFF ” exceeds the first difference threshold “DIFF _THD1 ” (step S24), Determination is made (step S25), and the smoothing filtering process is skipped by the filter unit 7b (step S26).

When the magnitude of the motion vector “MV” is equal to or _smaller than the first motion vector threshold “MV _THD1 ” and the difference value “DlFF” is equal to or _smaller than the first difference threshold “DIFF _THD1 ” (step S23, S24, S27), when the magnitude of the motion vector “MV” exceeds the second motion vector threshold “MV _THD2 ” (step S28), or the difference value “DIFF” is the second difference threshold When “DIFF _THD2 ” is exceeded (step S29), it is determined as the first still region (step S30), and otherwise (step S29), it is determined as the second still region (step S32).

In addition, when intra coding is performed, the capturing of the motion vector “MV” corresponding to the current input image is skipped, and only the difference value “DIFF” is captured (steps S21 and S22). When “” exceeds the first difference threshold value “DIFF _THD1 ” (step S24), it is determined as a motion region (step S25).

When the difference value “DIFF” is equal to or _smaller than the first difference threshold value “DIFF _THD1 ” (step S24), the difference value “DIFF” exceeds the second difference threshold value “DIFF _THD2 ”. (Step S29), it determines with a 1st still area (step S30), and when other than that (step S29), it determines with a 2nd still area (step S32).

  Next, based on the determination result of the still area / motion area determination unit 6b, the first still area is smoothed using the first smoothing filter (step S31), and the second still area is determined. Uses the second smoothing filter 2 having a higher degree of smoothing than the first smoothing filter to smooth the image (step S33).

  For example, the area determined to be the first still area is smoothed using the “2 × 2” average value filter, and the average value of “3 × 3” is determined for the area determined to be the second still area. When smoothing is performed using a filter and the high frequency component is reduced by the smoothing process, the code amount of the first still region is made smaller than the code amount of the motion region, and the code amount of the first still region is 2 Decrease the code amount of the still area.

  As described above, in the second embodiment, when the original image stored in the frame memory unit 2 is delayed by one frame, transferred to the frame memory unit 21, and is subjected to intra coding, the difference unit 16b Since the previous frame image stored in the memory unit 21 and the current input image stored in the frame memory unit 2 are compared to obtain a difference value, the difference value is stored in the frame memory unit 4. Thus, it is possible to prevent deterioration of the image quality of the reference frame image due to accumulation of quantization error, which occurs when the reconstructed reference frame image is used.

In the second embodiment, when the still region / motion region determination unit 6b determines the motion region and the still region, the motion vector threshold value “MV _THD ” and the difference value threshold value “DIFF _THD ” Are set, and according to the motion vector “MV” and the difference value “DIFF”, the still area is divided into several stages and the smoothing filter processing is performed. By further restricting the band, it is possible to reduce the amount of generated code that becomes a burden in low bit rate encoding, and to improve image quality degradation caused by insertion of intra encoding. Thereby, intra coding can be periodically inserted, and error accumulation due to DCT calculation mismatch between codecs can be eliminated.

  In addition, by cyclic insertion of intra coding, it is possible to realize early normal image output at the start of encoded data reception and early restoration from missing images due to communication errors during code data transmission.

  In the second embodiment, an example in which an original image is used in intra coding and a plurality of threshold values is used has been described. However, the present invention provides a decoded image by intra coding as in the first embodiment. A configuration in which a plurality of threshold values are used may be used. Further, when an original image is used in intra coding as in the second embodiment, a combination using a single threshold as in the first embodiment may be used.

It is a block diagram which shows 1st Embodiment of the moving image encoder by this invention. 3 is a flowchart illustrating a processing example of a still area / motion area determination unit and a filter unit illustrated in FIG. 1. It is a schematic diagram which shows an example of the smoothing filter used with the filter part shown in FIG. It is a schematic diagram which shows the example of the determination process of the still area / motion area determination part shown in FIG. It is a block diagram which shows 2nd Embodiment of the moving image encoder of this invention. 6 is a flowchart illustrating an example of processing of a still area / motion area determination unit and a filter unit illustrated in FIG. 5. It is a block diagram which shows an example of the moving image encoding apparatus known conventionally.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1a, 1b: Video coding apparatus 2: Frame memory part 3: Motion detection part 4: Frame memory part 5: Motion compensation part 6a, 6b: Still area / motion area determination part 7a, 7b: Filter part 8: Subtraction part 9: DCT part (encoding part)
10: Quantization unit (encoding unit)
11: Variable length encoding unit (encoding unit)
12: Buffer unit 13: Inverse quantization unit 14: IDCT unit 15: Adder unit 16a, 16b: Difference unit 21: Frame memory unit

Claims (2)

In a moving image coding apparatus that divides each frame image into blocks and sequentially encodes them,
The current frame image based on the motion vector and difference value obtained by performing the matching process on the reference frame image corresponding to the previous encoded frame image and the current frame image to be encoded next. A still region / motion region determination unit that divides the region into a motion region and a still region;
A filter unit that skips the filtering process for the area determined to be a moving area by the stationary area / motion area determining unit, and performs a smoothing process by performing a filtering process on the area determined to be a stationary area;
An encoding unit for encoding the output of the filter unit with a preset algorithm;
A moving picture encoding apparatus comprising: The moving image encoding device according to claim 1,
A plurality of motion vector threshold values and a plurality of difference threshold values are set in the still region / motion region determination unit, and the still region is determined at a plurality of levels according to the threshold values, and the filter unit Comprises a plurality of filters, for a region determined to be a static region, a filter process is performed by selecting a filter according to the determined level for each region from a plurality of prepared filters,
A moving picture coding apparatus characterized by the above.

Images