US20180020233A1 - Moving image encoding device - Google Patents
Moving image encoding device Download PDFInfo
- Publication number
- US20180020233A1 US20180020233A1 US15/548,673 US201615548673A US2018020233A1 US 20180020233 A1 US20180020233 A1 US 20180020233A1 US 201615548673 A US201615548673 A US 201615548673A US 2018020233 A1 US2018020233 A1 US 2018020233A1
- Authority
- US
- United States
- Prior art keywords
- intra
- prediction
- offset value
- prediction mode
- input image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
- H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/107—Selection of coding mode or of prediction mode between spatial and temporal predictive coding, e.g. picture refresh
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/11—Selection of coding mode or of prediction mode among a plurality of spatial predictive coding modes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
- H04N19/126—Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
- H04N19/14—Coding unit complexity, e.g. amount of activity or edge presence estimation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/625—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using discrete cosine transform [DCT]
Definitions
- the present invention relates to a moving image encoding device that encodes a moving image.
- encoding is to determine the intra-prediction mode using a local decoded image, and to generate a prediction image using the local decoded image.
- this impairs the advantageous effect of improvement in speed in a case where the speed of the moving image encoding device is intended be improved through parallel processing. Accordingly, intra-prediction mode determination often uses an input image.
- PTL 1 an intra-prediction is performed using an input image.
- the mode is forcibly changed to a DC mode.
- the DC mode creates a prediction image using the average of adjacent pixel data items; although image degradation can be suppressed, the advantageous effect of improving the image quality is insufficient.
- the technology of PTL 1 cannot be adopted.
- PTL 2 describes a method of reducing degradation in image quality during fading. This method switches weighted prediction on and off through control on an area-by-area basis. In the windowboxed area, this prediction is turned off. Unfortunately, the technology of PTL 2 is inapplicable to normal video without fading.
- the present invention has been made to solve the problems described above, and has an object to obtain a moving image encoding device that can improve the image quality of video including a motionless area with flat pixel values, such as in windowboxing.
- a moving image encoding device includes: a computation unit subtracting a prediction image from an input image; an orthogonal transformation unit applying orthogonal transformation to an output of the computation unit; a quantization unit quantizing an output of the orthogonal transformation unit; an encoding unit encoding an output of the quantization unit; and a prediction mode determination unit determining a prediction mode from the input image, wherein the prediction mode is different according to types of an I-picture, a P-picture and a B-picture.
- the present invention makes it possible to improve the image quality of video including a motionless area with flat pixel values, such as in windowboxing.
- FIG. 1 is a diagram showing a moving image encoding device according to an embodiment of present invention.
- FIG. 2 is a diagram showing a processing flow in the prediction mode determination unit.
- FIG. 3 is a diagram showing an offset value calculation flow.
- FIG. 4 is a diagram showing an intra-prediction mode offset value calculation flow.
- FIG. 5 is a diagram showing an intra-offset value calculation flow.
- FIG. 6 is a diagram showing an intra-prediction mode determination flow.
- FIG. 7 is a diagram showing the intra/inter determination flow.
- FIG. 8 is a diagram showing an example of video (input image).
- FIG. 9 is a diagram showing the frequency distribution of differences from pixel values after encoding with the pixel value of the input image being “a” with respect to a flat part encircled by a rectangle in FIG. 8 .
- FIG. 1 is a diagram showing a moving image encoding device according to an embodiment of present invention.
- a computation unit 1 subtracts a prediction image from an input image.
- An orthogonal transformation unit 2 applies orthogonal transformation, such as discrete cosine transform or Karhunen-Loève transform, to an output of the computation unit 1 . Note that any method of orthogonal transformation may be adopted.
- a quantization unit 3 quantizes the output of the orthogonal transformation unit 2 . Any quantization method may be adopted.
- An encoding unit 4 encodes the output of the quantization unit 3 and generates a stream output.
- An inverse quantization unit 5 inverse-quantizes the output of the quantization unit 3 .
- any inverse quantization method may be adopted only if the method supports the quantization process in the quantization unit 3 .
- An inverse orthogonal transformation unit 6 applies inverse orthogonal transformation to the output of the inverse quantization unit 5 .
- any inverse orthogonal transformation method may be adopted only if the method supports the orthogonal transformation process in the orthogonal transformation unit 2 .
- a computation unit 7 adds the prediction image to the output of the inverse quantization unit 5 , and obtains a local reconstructed image. The reconstructed image is subjected to a filter process through a deblocking filter 8 and a SAO (Sample Adaptive Offset) 9 , and is subsequently stored in a frame memory 10 .
- SAO Sample Adaptive Offset
- a prediction mode determination unit 11 determines the prediction mode from the input image.
- the prediction mode is different according to types of an I-picture, a P-picture and a B-picture. More specifically, the I-picture is a picture to be subjected only to intra-prediction. Accordingly, only the intra-prediction is performed to determine the prediction mode. In a case where the P-picture or B-picture, intra-prediction and inter-prediction are performed, and the intra-prediction mode and the inter-prediction mode are determined. Next, intra/inter determination that selects any one of the determined intra-prediction mode and inter-prediction mode to thus determine the prediction mode.
- the intra-prediction mode performs determination through intra-prediction using the pixel values of the image in a surrounding area positioned around a coding target block.
- the inter-prediction mode performs determination through inter-prediction using the input image and a reference image supplied from the frame memory 10 .
- a prediction image generation unit 12 generates a prediction image using the mode determined by the prediction mode determination unit 11 and the reference image supplied from the frame memory 10 .
- the amount of characteristic that can be calculated from information on the input image is used.
- the amount of characteristic may be information on a pixel value, such as the luminance or chrominance, and can be obtained in the unit of CU (Coding Unit) or picture.
- a flat area determination parameter is used to determine a motionless area that has flat pixel values and characterizes windowboxing.
- Examples of flat area determination parameters include the activity, and the sum of absolute differences between adjacent pixels.
- the activity is a parameter that represents the degree of distribution of the pixel values of the input image, and is the amount of characteristic that is important in measurement of the complexity of video.
- the sum of absolute differences between adjacent pixels is a parameter that represents the relationship with adjacent pixels, and is the amount of characteristic for measuring the similarities and differences of the adjacent pixels.
- the activity can be obtained by Math. 1.
- n is the block size
- X is the pixel value
- X ( ⁇ ) is the average value of the pixel values X in the block size. Note that any block size for obtaining the activity may be adopted. The sum of absolute differences between adjacent pixels can be obtained by Math. 2.
- n is the block size
- X and Y represent the pixel values of adjacent pixels. Note that any block size for obtaining the sum of absolute differences between adjacent pixels may be adopted.
- the offset value is obtained on the basis of the flat area determination parameter, and the intra-prediction is executed on the basis of the evaluation value to which the offset value has been added.
- the offset value is obtained on the basis of the flat area determination parameter, any one of the intra-prediction mode and the inter-prediction mode is selected on the basis of the evaluation value to which the offset value has been added, and the selected mode is applied to each block of the input image.
- Such addition of the offset value obtained from the flat area determination parameter to the evaluation value to be used for mode determination can improve the image quality of video that includes a motionless area having flat pixel values, such as of windowboxing.
- FIG. 2 is a diagram showing a processing flow in the prediction mode determination unit.
- the mode offset value and the intra-offset value are set to zero.
- it is determined whether the coding target picture is the I-picture or not (step S 1 ).
- the intra-prediction is performed that determines one mode among 35 intra-prediction modes using the intra-prediction mode evaluation value (step S 2 ).
- the intra-prediction mode evaluation value is a value obtained by adding the intra-prediction mode offset value to an existing evaluation value.
- the intra-prediction mode offset value is a value determined by the flat area determination parameter.
- the inter-prediction is performed that detects a motion vector from the reference image and determines the optimal motion vector (step S 3 ).
- the intra-prediction is performed (step S 4 ).
- the one optimal inter-mode and the one optimal intra-mode are determined.
- the intra/inter determination is performed that determines the best mode from the determined optimal inter-mode using the inter-evaluation value and optimal intra-mode using the intra-evaluation value (step S 5 ).
- improvement in intra-prediction (steps S 2 and S 4 ) and improvement in intra/inter determination (step S 5 ) can be independently configured. More specifically, the case where only the intra-prediction is intended to be enabled can be achieved by not invoking the offset value calculation block in step S 5 ; the case where only the intra/inter determination is intended to be enabled can be achieved by not invoking the offset value calculation blocks in steps S 2 and S 4 .
- the windowboxed area is an area determined to be flat on the basis of the flat area determination parameter. Consequently, it is required that the intra-prediction direction from the block determined to be nonflat should not be selected.
- the determination of the intra-prediction mode and the intra/inter determination select the mode with the lowest evaluation value. Consequently, increase in the evaluation value can make the mode resistant to being selected.
- the evaluation value is calculated on a mode-by-mode basis, and the offset value is added to the evaluation value to correct the evaluation value.
- the offset value is determined from the magnitude relationship between the flat area determination parameter of the adjacent block to be used for prediction and the set threshold. For example, the offset value is stepwisely changed by setting multiple thresholds and by configuring setting so that the larger the flat area determination parameter of the adjacent block is, the larger the offset is set.
- FIG. 3 is a diagram showing an offset value calculation flow.
- step S 11 it is determined whether the application area is limited or not.
- the evaluation value to which the offset value has been added is used for every block (step S 12 ).
- step S 12 the intra-prediction mode offset value used during intra-prediction is calculated using the flow of FIG. 4
- the intra-offset value used during intra/inter determination is calculated using the flow of FIG. 5 .
- the flat area determination parameter of the coding target block is obtained (step S 13 ). It is determined whether the flat area determination parameter of the coding target block is at least the designated threshold or not (step S 14 ). When the flat area determination parameter is lower than the threshold, the evaluation value to which the offset value has been added is used (step S 15 ). In the offset value calculation (step S 15 ), the intra-prediction mode offset value used during intra-prediction is calculated using the flow of FIG. 4 , and the intra-offset value used during intra/inter determination is calculated using the flow of FIG. 5 .
- the pixel values largely vary and are not constant, such as of a black border.
- the flat area determination parameter is at least the threshold, zero is substituted into the offset value, and this function is not applied (step S 16 ).
- this function is not applied (step S 16 ).
- zero is substituted into the offset value according to the flat area determination parameter to limit the application range, thereby preventing the area with nonflat pixel values from being affected.
- FIG. 4 is a diagram showing an intra-prediction mode offset value calculation flow.
- FIG. 5 is a diagram showing an intra-offset value calculation flow.
- the multiple thresholds 1 to n are set, and the offset values different according to the thresholds are set.
- the intra-prediction mode offset value calculation the adjacent block used for intra-prediction is changed. Consequently, the offset value is set with respect to each intra-prediction mode.
- the intra-prediction modes allow prediction in 35 different directions. Prediction based on the block similar to the coding target block reduces the error, thereby improving the encoding efficiency. Specifically, in a motionless area with flat pixel values, such as of a black border, degradation in image quality is subjectively prominent even if the difference of pixel values is small. Thus, the calculated offset value is added to the existing evaluation value, thereby facilitating selection of prediction from the similar block.
- FIG. 6 is a diagram showing an intra-prediction mode determination flow.
- the maximum value is set into the best intra-evaluation value (step S 21 ).
- the loop of the intra-prediction modes (0 to 34) is started (step S 22 ).
- the existing evaluation value is calculated (step S 23 ).
- the existing evaluation value is calculated by the following Math. 3.
- D indicates the error between the input image and the prediction image
- ⁇ indicates a weighting coefficient
- H indicates header cost required during prediction. It is represented that the lower the evaluation value is, the smaller the amount of data is or the smaller the error from the input image and the better the encoding efficiency the mode has.
- the existing evaluation value represented in Math. 2 is used as it is without reappraisal of this value, and the offset value is added, thereby allowing this embodiment to be easily executed.
- the mode offset value is added to the existing evaluation value to obtain the evaluation value (mode) (step S 24 ).
- Such use of the offset value calculated for every prediction mode allows the appropriate prediction mode to be selected. For example, in a case where the coding target block is in contact with the boundary of the black border at a lower part of the screen in windowboxing video having upper and lower black borders, the larger the flat area determination parameter of the upper adjacent block is, the larger the offset value of the prediction mode in the upward direction is, thereby making the intra-prediction in the upward direction resistant to being selected.
- the prediction is performed on the basis of the black border part. Consequently, the prediction residual becomes small, thereby improving the intra-block image quality.
- the offset is added to the evaluation value so that the precedence can be given to the inter-prediction with respect to the intra-prediction in motionless areas with flat pixel values, such as of black borders.
- the degradation of the reference image used for inter-prediction is low, and the prediction error tends to be low.
- the motion vector is small. Accordingly, the amount of encoding is small. Consequently, it is appropriate to give precedence to the inter-prediction.
- FIG. 7 is a diagram showing the intra/inter determination flow.
- the existing intra-evaluation value and the existing inter-evaluation value are calculated (steps S 31 and S 32 ).
- the intra-evaluation value is obtained by adding the intra-offset value to the existing intra-evaluation value (step S 33 ).
- a process of adding the offset value to the existing intra-evaluation value used for intra/inter determination is added.
- the intra/inter determination what has the lower evaluation value is selected.
- the intra-offset value is set to a positive value, the intra-evaluation value becomes large, which makes the intra resistant to being selected.
- the intra-offset value is set to a negative value, the intra tends to be selected.
- setting the intra-offset value to a positive value makes the intra with a large prediction error more resistant to being selected. Selection with the precedence being given to the inter-prediction can improve the image quality.
- the intra-block is selected in the intra/inter determination, improvement in image quality has already been achieved in the intra-prediction.
- FIG. 8 is a diagram showing an example of video (input image).
- FIG. 9 is a diagram showing the frequency distribution of differences from pixel values after encoding with the pixel value of the input image being “a” with respect to a flat part encircled by a rectangle in FIG. 8 .
- the distribution after application has more pixel values close to “a”.
- the image quality where the pixel values after encoding are closer to the pixel values of the input image can be considered as image quality with low degradation. Consequently, application of this embodiment confirms that the image has pixel values close to those of input image and has lower degradation.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Discrete Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Description
- The present invention relates to a moving image encoding device that encodes a moving image.
- Intrinsically, encoding is to determine the intra-prediction mode using a local decoded image, and to generate a prediction image using the local decoded image. However, this impairs the advantageous effect of improvement in speed in a case where the speed of the moving image encoding device is intended be improved through parallel processing. Accordingly, intra-prediction mode determination often uses an input image.
- Unfortunately, since final predicted image generation necessarily uses the local decoded image, a possible difference between the input image and the local decoded image causes a problem in that generation of the prediction image from the local decoded image cannot allow the optimal intra-prediction mode to be selected. In particular, in a case where there is a difference between the input image and the local decoded image in an area with flat pixel values, such as a black border of a windowboxed image, an appropriate prediction error cannot be calculated, an intra-prediction mode having a large error is selected, and image degradation becomes visually prominent. Images similar to windowboxed images include a title image which has flat pixel values over the entire image and in which characters are partially written.
- [PTL 1] JP 2010-283739 A
- [PTL 2] JP 2012-244353 A
- In
PTL 1, an intra-prediction is performed using an input image. In a case where the difference between the input image and the local decoded image is large, the mode is forcibly changed to a DC mode. Unfortunately, the DC mode creates a prediction image using the average of adjacent pixel data items; although image degradation can be suppressed, the advantageous effect of improving the image quality is insufficient. In case of incapability of calculating the difference between the input image and the local decoded image, the technology ofPTL 1 cannot be adopted. -
PTL 2 describes a method of reducing degradation in image quality during fading. This method switches weighted prediction on and off through control on an area-by-area basis. In the windowboxed area, this prediction is turned off. Unfortunately, the technology ofPTL 2 is inapplicable to normal video without fading. - The present invention has been made to solve the problems described above, and has an object to obtain a moving image encoding device that can improve the image quality of video including a motionless area with flat pixel values, such as in windowboxing.
- A moving image encoding device according to the present invention includes: a computation unit subtracting a prediction image from an input image; an orthogonal transformation unit applying orthogonal transformation to an output of the computation unit; a quantization unit quantizing an output of the orthogonal transformation unit; an encoding unit encoding an output of the quantization unit; and a prediction mode determination unit determining a prediction mode from the input image, wherein the prediction mode is different according to types of an I-picture, a P-picture and a B-picture.
- The present invention makes it possible to improve the image quality of video including a motionless area with flat pixel values, such as in windowboxing.
-
FIG. 1 is a diagram showing a moving image encoding device according to an embodiment of present invention. -
FIG. 2 is a diagram showing a processing flow in the prediction mode determination unit. -
FIG. 3 is a diagram showing an offset value calculation flow. -
FIG. 4 is a diagram showing an intra-prediction mode offset value calculation flow. -
FIG. 5 is a diagram showing an intra-offset value calculation flow. -
FIG. 6 is a diagram showing an intra-prediction mode determination flow. -
FIG. 7 is a diagram showing the intra/inter determination flow. -
FIG. 8 is a diagram showing an example of video (input image). -
FIG. 9 is a diagram showing the frequency distribution of differences from pixel values after encoding with the pixel value of the input image being “a” with respect to a flat part encircled by a rectangle inFIG. 8 . -
FIG. 1 is a diagram showing a moving image encoding device according to an embodiment of present invention. Acomputation unit 1 subtracts a prediction image from an input image. Anorthogonal transformation unit 2 applies orthogonal transformation, such as discrete cosine transform or Karhunen-Loève transform, to an output of thecomputation unit 1. Note that any method of orthogonal transformation may be adopted. Aquantization unit 3 quantizes the output of theorthogonal transformation unit 2. Any quantization method may be adopted. Anencoding unit 4 encodes the output of thequantization unit 3 and generates a stream output. - An
inverse quantization unit 5 inverse-quantizes the output of thequantization unit 3. Note that any inverse quantization method may be adopted only if the method supports the quantization process in thequantization unit 3. An inverseorthogonal transformation unit 6 applies inverse orthogonal transformation to the output of theinverse quantization unit 5. Note that any inverse orthogonal transformation method may be adopted only if the method supports the orthogonal transformation process in theorthogonal transformation unit 2. Acomputation unit 7 adds the prediction image to the output of theinverse quantization unit 5, and obtains a local reconstructed image. The reconstructed image is subjected to a filter process through adeblocking filter 8 and a SAO (Sample Adaptive Offset) 9, and is subsequently stored in aframe memory 10. - A prediction
mode determination unit 11 determines the prediction mode from the input image. The prediction mode is different according to types of an I-picture, a P-picture and a B-picture. More specifically, the I-picture is a picture to be subjected only to intra-prediction. Accordingly, only the intra-prediction is performed to determine the prediction mode. In a case where the P-picture or B-picture, intra-prediction and inter-prediction are performed, and the intra-prediction mode and the inter-prediction mode are determined. Next, intra/inter determination that selects any one of the determined intra-prediction mode and inter-prediction mode to thus determine the prediction mode. - The intra-prediction mode performs determination through intra-prediction using the pixel values of the image in a surrounding area positioned around a coding target block. The inter-prediction mode performs determination through inter-prediction using the input image and a reference image supplied from the
frame memory 10. - A prediction
image generation unit 12 generates a prediction image using the mode determined by the predictionmode determination unit 11 and the reference image supplied from theframe memory 10. - To obtain the characteristics of an image, such as of windowboxing, the amount of characteristic that can be calculated from information on the input image is used. The amount of characteristic may be information on a pixel value, such as the luminance or chrominance, and can be obtained in the unit of CU (Coding Unit) or picture. In this embodiment, a flat area determination parameter is used to determine a motionless area that has flat pixel values and characterizes windowboxing. Examples of flat area determination parameters include the activity, and the sum of absolute differences between adjacent pixels. The activity is a parameter that represents the degree of distribution of the pixel values of the input image, and is the amount of characteristic that is important in measurement of the complexity of video. The sum of absolute differences between adjacent pixels is a parameter that represents the relationship with adjacent pixels, and is the amount of characteristic for measuring the similarities and differences of the adjacent pixels.
- The activity can be obtained by Math. 1.
-
- Here, n is the block size, X is the pixel value, X (−) is the average value of the pixel values X in the block size. Note that any block size for obtaining the activity may be adopted. The sum of absolute differences between adjacent pixels can be obtained by Math. 2.
-
- Here, n is the block size, and X and Y represent the pixel values of adjacent pixels. Note that any block size for obtaining the sum of absolute differences between adjacent pixels may be adopted.
- For the I-picture, the offset value is obtained on the basis of the flat area determination parameter, and the intra-prediction is executed on the basis of the evaluation value to which the offset value has been added. For the P-picture and B-picture, the offset value is obtained on the basis of the flat area determination parameter, any one of the intra-prediction mode and the inter-prediction mode is selected on the basis of the evaluation value to which the offset value has been added, and the selected mode is applied to each block of the input image. Such addition of the offset value obtained from the flat area determination parameter to the evaluation value to be used for mode determination can improve the image quality of video that includes a motionless area having flat pixel values, such as of windowboxing.
-
FIG. 2 is a diagram showing a processing flow in the prediction mode determination unit. First, the mode offset value and the intra-offset value are set to zero. Next, it is determined whether the coding target picture is the I-picture or not (step S1). When the coding target picture is the I-picture, the intra-prediction is performed that determines one mode among 35 intra-prediction modes using the intra-prediction mode evaluation value (step S2). The intra-prediction mode evaluation value is a value obtained by adding the intra-prediction mode offset value to an existing evaluation value. The intra-prediction mode offset value is a value determined by the flat area determination parameter. - When the picture is other than the I-picture (P-picture or B-picture), the inter-prediction is performed that detects a motion vector from the reference image and determines the optimal motion vector (step S3). As with the case of I-picture, the intra-prediction is performed (step S4). Thus, the one optimal inter-mode and the one optimal intra-mode are determined.
- Next, the intra/inter determination is performed that determines the best mode from the determined optimal inter-mode using the inter-evaluation value and optimal intra-mode using the intra-evaluation value (step S5). Note that in the present invention, improvement in intra-prediction (steps S2 and S4) and improvement in intra/inter determination (step S5) can be independently configured. More specifically, the case where only the intra-prediction is intended to be enabled can be achieved by not invoking the offset value calculation block in step S5; the case where only the intra/inter determination is intended to be enabled can be achieved by not invoking the offset value calculation blocks in steps S2 and S4.
- The windowboxed area is an area determined to be flat on the basis of the flat area determination parameter. Consequently, it is required that the intra-prediction direction from the block determined to be nonflat should not be selected. The determination of the intra-prediction mode and the intra/inter determination select the mode with the lowest evaluation value. Consequently, increase in the evaluation value can make the mode resistant to being selected. Thus, the evaluation value is calculated on a mode-by-mode basis, and the offset value is added to the evaluation value to correct the evaluation value. The offset value is determined from the magnitude relationship between the flat area determination parameter of the adjacent block to be used for prediction and the set threshold. For example, the offset value is stepwisely changed by setting multiple thresholds and by configuring setting so that the larger the flat area determination parameter of the adjacent block is, the larger the offset is set.
-
FIG. 3 is a diagram showing an offset value calculation flow. First, it is determined whether the application area is limited or not (step S11). In a case where the application area is not specifically limited, the evaluation value to which the offset value has been added is used for every block (step S12). In the offset value calculation (step S12), the intra-prediction mode offset value used during intra-prediction is calculated using the flow ofFIG. 4 , and the intra-offset value used during intra/inter determination is calculated using the flow ofFIG. 5 . - In a case where the application area is limited, the flat area determination parameter of the coding target block is obtained (step S13). It is determined whether the flat area determination parameter of the coding target block is at least the designated threshold or not (step S14). When the flat area determination parameter is lower than the threshold, the evaluation value to which the offset value has been added is used (step S15). In the offset value calculation (step S15), the intra-prediction mode offset value used during intra-prediction is calculated using the flow of
FIG. 4 , and the intra-offset value used during intra/inter determination is calculated using the flow ofFIG. 5 . - In the block where the flat area determination parameter is at least the threshold, the pixel values largely vary and are not constant, such as of a black border. Thus, when the flat area determination parameter is at least the threshold, zero is substituted into the offset value, and this function is not applied (step S16). Typically, since areas with nonflat pixel values are the majority, zero is substituted into the offset value according to the flat area determination parameter to limit the application range, thereby preventing the area with nonflat pixel values from being affected.
-
FIG. 4 is a diagram showing an intra-prediction mode offset value calculation flow.FIG. 5 is a diagram showing an intra-offset value calculation flow. InFIGS. 4 and 5 , themultiple thresholds 1 to n are set, and the offset values different according to the thresholds are set. In the intra-prediction mode offset value calculation, the adjacent block used for intra-prediction is changed. Consequently, the offset value is set with respect to each intra-prediction mode. - The intra-prediction modes allow prediction in 35 different directions. Prediction based on the block similar to the coding target block reduces the error, thereby improving the encoding efficiency. Specifically, in a motionless area with flat pixel values, such as of a black border, degradation in image quality is subjectively prominent even if the difference of pixel values is small. Thus, the calculated offset value is added to the existing evaluation value, thereby facilitating selection of prediction from the similar block.
-
FIG. 6 is a diagram showing an intra-prediction mode determination flow. First, the maximum value is set into the best intra-evaluation value (step S21). Next, the loop of the intra-prediction modes (0 to 34) is started (step S22). Next, the existing evaluation value is calculated (step S23). The existing evaluation value is calculated by the following Math. 3. -
Existing evaluation value=D+λ*H Math. 3 - Here, D indicates the error between the input image and the prediction image, λ indicates a weighting coefficient, and H indicates header cost required during prediction. It is represented that the lower the evaluation value is, the smaller the amount of data is or the smaller the error from the input image and the better the encoding efficiency the mode has. The existing evaluation value represented in Math. 2 is used as it is without reappraisal of this value, and the offset value is added, thereby allowing this embodiment to be easily executed.
- Next, the mode offset value is added to the existing evaluation value to obtain the evaluation value (mode) (step S24). Next, it is determined whether the evaluation value (mode) is lower than the best intra-evaluation value or not (step S25). In the higher case, the best intra mode is not updated, and the next intra-prediction mode is evaluated. On the contrary, in the lower case, the best intra evaluation value is updated to the evaluation value (mode) (step S26), and the next intra-prediction mode is evaluated. After evaluation of all the intra-prediction modes is completed, the loop of the intra-prediction modes (0 to 34) is finished (step S27).
- Such use of the offset value calculated for every prediction mode allows the appropriate prediction mode to be selected. For example, in a case where the coding target block is in contact with the boundary of the black border at a lower part of the screen in windowboxing video having upper and lower black borders, the larger the flat area determination parameter of the upper adjacent block is, the larger the offset value of the prediction mode in the upward direction is, thereby making the intra-prediction in the upward direction resistant to being selected. Thus, in a case where the black border part is encoded, the prediction is performed on the basis of the black border part. Consequently, the prediction residual becomes small, thereby improving the intra-block image quality.
- For the P-picture and B-picture, the offset is added to the evaluation value so that the precedence can be given to the inter-prediction with respect to the intra-prediction in motionless areas with flat pixel values, such as of black borders. Typically, in the flat areas, the degradation of the reference image used for inter-prediction is low, and the prediction error tends to be low. Furthermore, in the inter-prediction in the areas with flat pixel values, the motion vector is small. Accordingly, the amount of encoding is small. Consequently, it is appropriate to give precedence to the inter-prediction.
-
FIG. 7 is a diagram showing the intra/inter determination flow. First, the existing intra-evaluation value and the existing inter-evaluation value are calculated (steps S31 and S32). Next, the intra-evaluation value is obtained by adding the intra-offset value to the existing intra-evaluation value (step S33). Next, it is determined whether the intra-evaluation value is lower than the existing inter-evaluation value or not (step S34). In a case where the intra-evaluation value is lower, the best mode is determined to be the intra-mode (step S35). In the other case, the best mode is determined to be the inter-mode (step S36). Thus, in this embodiment, a process of adding the offset value to the existing intra-evaluation value used for intra/inter determination is added. - In the intra/inter determination, what has the lower evaluation value is selected. When the intra-offset value is set to a positive value, the intra-evaluation value becomes large, which makes the intra resistant to being selected. On the contrary, when the intra-offset value is set to a negative value, the intra tends to be selected. When the block with degraded image quality is the intra, setting the intra-offset value to a positive value makes the intra with a large prediction error more resistant to being selected. Selection with the precedence being given to the inter-prediction can improve the image quality. When the intra-block is selected in the intra/inter determination, improvement in image quality has already been achieved in the intra-prediction.
-
FIG. 8 is a diagram showing an example of video (input image).FIG. 9 is a diagram showing the frequency distribution of differences from pixel values after encoding with the pixel value of the input image being “a” with respect to a flat part encircled by a rectangle inFIG. 8 . In comparison between the frequency distributions before and after application of this embodiment, the distribution after application has more pixel values close to “a”. Typically, the image quality where the pixel values after encoding are closer to the pixel values of the input image can be considered as image quality with low degradation. Consequently, application of this embodiment confirms that the image has pixel values close to those of input image and has lower degradation. - 1 computation unit, 2 orthogonal transformation unit, 3 quantization unit, 4 encoding unit, 11 prediction mode determination unit
Claims (5)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2015-062905 | 2015-03-25 | ||
JP2015062905A JP6052319B2 (en) | 2015-03-25 | 2015-03-25 | Video encoding device |
PCT/JP2016/056641 WO2016152446A1 (en) | 2015-03-25 | 2016-03-03 | Moving image encoding device |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2016/056641 A-371-Of-International WO2016152446A1 (en) | 2015-03-25 | 2016-03-03 | Moving image encoding device |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/452,025 Division US11432005B2 (en) | 2015-03-25 | 2019-06-25 | Moving image encoding device |
Publications (1)
Publication Number | Publication Date |
---|---|
US20180020233A1 true US20180020233A1 (en) | 2018-01-18 |
Family
ID=56977877
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/548,673 Abandoned US20180020233A1 (en) | 2015-03-25 | 2016-03-03 | Moving image encoding device |
US16/452,025 Active 2036-07-06 US11432005B2 (en) | 2015-03-25 | 2019-06-25 | Moving image encoding device |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/452,025 Active 2036-07-06 US11432005B2 (en) | 2015-03-25 | 2019-06-25 | Moving image encoding device |
Country Status (5)
Country | Link |
---|---|
US (2) | US20180020233A1 (en) |
EP (2) | EP3869799B1 (en) |
JP (1) | JP6052319B2 (en) |
BR (1) | BR112017017282A2 (en) |
WO (1) | WO2016152446A1 (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20170180730A1 (en) * | 2014-09-24 | 2017-06-22 | Hitachi Information & Telecommunication Engineering, Ltd. | Moving image coding device, moving image decoding device, moving image coding method, and moving image decoding method |
US20190045182A1 (en) * | 2017-12-20 | 2019-02-07 | Intel Corporation | Decoupled prediction and coding structure for video encoding |
US20190356912A1 (en) * | 2018-05-18 | 2019-11-21 | Fujitsu Limited | Information processing apparatus, information processing method and computer-readable recording medium having stored program therein |
US11432005B2 (en) * | 2015-03-25 | 2022-08-30 | Ntt Electronics Corporation | Moving image encoding device |
US20240107170A1 (en) * | 2016-10-04 | 2024-03-28 | B1 Institute Of Image Technology, Inc. | Image data encoding/decoding method and apparatus |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090011007A1 (en) * | 2006-02-03 | 2009-01-08 | Evonik Roehm Gmbh | Pharmaceutical Compositions Containing Mixtures of Polymers and Active Agents Poorly Soluble in Water |
US20090290643A1 (en) * | 2006-07-12 | 2009-11-26 | Jeong Hyu Yang | Method and apparatus for processing a signal |
US20100266042A1 (en) * | 2007-03-02 | 2010-10-21 | Han Suh Koo | Method and an apparatus for decoding/encoding a video signal |
JP2013138502A (en) * | 2013-03-04 | 2013-07-11 | Canon Inc | Coding apparatus, control method of the same, and computer program |
US20130229485A1 (en) * | 2011-08-30 | 2013-09-05 | Nokia Corporation | Apparatus, a Method and a Computer Program for Video Coding and Decoding |
US20150312545A1 (en) * | 2012-07-04 | 2015-10-29 | Lidong Xu | Inter-view filter parameters re-use for three dimensional video coding |
US20160105682A1 (en) * | 2014-10-09 | 2016-04-14 | Qualcomm Incorporated | Intra block copy prediction restrictions for parallel processing |
Family Cites Families (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR100487989B1 (en) * | 1997-10-23 | 2005-05-09 | 미쓰비시덴키 가부시키가이샤 | Image decoder and image encoder |
GB2351916B (en) * | 2000-03-23 | 2001-05-30 | Christopher Howard Field | Putters |
EP1569460B1 (en) * | 2002-11-25 | 2013-05-15 | Panasonic Corporation | Motion compensation method, image encoding method, and image decoding method |
TWI245539B (en) * | 2004-09-22 | 2005-12-11 | Sony Taiwan Ltd | Spatial domain pre-processing to reduce the intra-coding computation complexity |
JP4877449B2 (en) * | 2004-11-04 | 2012-02-15 | カシオ計算機株式会社 | Moving picture coding apparatus and moving picture coding processing program |
FR2880745A1 (en) * | 2005-01-07 | 2006-07-14 | France Telecom | VIDEO ENCODING METHOD AND DEVICE |
JP2007228519A (en) * | 2006-02-27 | 2007-09-06 | Renesas Technology Corp | Image encoding device and image encoding method |
US8036270B2 (en) * | 2006-07-27 | 2011-10-11 | Sharp Laboratories Of America, Inc. | Intra-frame flicker reduction in video coding |
JP2007251996A (en) * | 2007-05-24 | 2007-09-27 | Toshiba Corp | Moving picture coding method, and apparatus adopting same |
JP2009111691A (en) * | 2007-10-30 | 2009-05-21 | Hitachi Ltd | Image-encoding device and encoding method, and image-decoding device and decoding method |
EP2081386A1 (en) * | 2008-01-18 | 2009-07-22 | Panasonic Corporation | High precision edge prediction for intracoding |
JP2009182623A (en) * | 2008-01-30 | 2009-08-13 | Panasonic Corp | Image encoding method |
JP5183573B2 (en) | 2009-06-08 | 2013-04-17 | 三菱電機株式会社 | Video encoding device |
JP5215951B2 (en) * | 2009-07-01 | 2013-06-19 | キヤノン株式会社 | Encoding apparatus, control method therefor, and computer program |
US8711930B2 (en) * | 2009-07-09 | 2014-04-29 | Qualcomm Incorporated | Non-zero rounding and prediction mode selection techniques in video encoding |
JP2012244353A (en) | 2011-05-18 | 2012-12-10 | Sony Corp | Image processing device and method |
GB2491391B (en) * | 2011-06-02 | 2014-09-03 | Canon Kk | Encoding mode values representing prediction modes |
JP2014082639A (en) * | 2012-10-16 | 2014-05-08 | Canon Inc | Image encoder and method of the same |
JP6052319B2 (en) * | 2015-03-25 | 2016-12-27 | Nttエレクトロニクス株式会社 | Video encoding device |
-
2015
- 2015-03-25 JP JP2015062905A patent/JP6052319B2/en active Active
-
2016
- 2016-03-03 US US15/548,673 patent/US20180020233A1/en not_active Abandoned
- 2016-03-03 BR BR112017017282-8A patent/BR112017017282A2/en active Search and Examination
- 2016-03-03 WO PCT/JP2016/056641 patent/WO2016152446A1/en active Application Filing
- 2016-03-03 EP EP21167177.1A patent/EP3869799B1/en active Active
- 2016-03-03 EP EP16768349.9A patent/EP3276959B1/en active Active
-
2019
- 2019-06-25 US US16/452,025 patent/US11432005B2/en active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090011007A1 (en) * | 2006-02-03 | 2009-01-08 | Evonik Roehm Gmbh | Pharmaceutical Compositions Containing Mixtures of Polymers and Active Agents Poorly Soluble in Water |
US20090290643A1 (en) * | 2006-07-12 | 2009-11-26 | Jeong Hyu Yang | Method and apparatus for processing a signal |
US20100266042A1 (en) * | 2007-03-02 | 2010-10-21 | Han Suh Koo | Method and an apparatus for decoding/encoding a video signal |
US20130229485A1 (en) * | 2011-08-30 | 2013-09-05 | Nokia Corporation | Apparatus, a Method and a Computer Program for Video Coding and Decoding |
US20150312545A1 (en) * | 2012-07-04 | 2015-10-29 | Lidong Xu | Inter-view filter parameters re-use for three dimensional video coding |
JP2013138502A (en) * | 2013-03-04 | 2013-07-11 | Canon Inc | Coding apparatus, control method of the same, and computer program |
US20160105682A1 (en) * | 2014-10-09 | 2016-04-14 | Qualcomm Incorporated | Intra block copy prediction restrictions for parallel processing |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20170180730A1 (en) * | 2014-09-24 | 2017-06-22 | Hitachi Information & Telecommunication Engineering, Ltd. | Moving image coding device, moving image decoding device, moving image coding method, and moving image decoding method |
US10116936B2 (en) * | 2014-09-24 | 2018-10-30 | Hitachi Information & Telecommunication Engineering, Ltd. | Moving image coding device, moving image decoding device, moving image coding method, and moving image decoding method |
US11432005B2 (en) * | 2015-03-25 | 2022-08-30 | Ntt Electronics Corporation | Moving image encoding device |
US20240107170A1 (en) * | 2016-10-04 | 2024-03-28 | B1 Institute Of Image Technology, Inc. | Image data encoding/decoding method and apparatus |
US20190045182A1 (en) * | 2017-12-20 | 2019-02-07 | Intel Corporation | Decoupled prediction and coding structure for video encoding |
US10687054B2 (en) * | 2017-12-20 | 2020-06-16 | Intel Corporation | Decoupled prediction and coding structure for video encoding |
US20190356912A1 (en) * | 2018-05-18 | 2019-11-21 | Fujitsu Limited | Information processing apparatus, information processing method and computer-readable recording medium having stored program therein |
Also Published As
Publication number | Publication date |
---|---|
US11432005B2 (en) | 2022-08-30 |
JP2016184801A (en) | 2016-10-20 |
EP3276959B1 (en) | 2021-05-19 |
WO2016152446A1 (en) | 2016-09-29 |
JP6052319B2 (en) | 2016-12-27 |
BR112017017282A2 (en) | 2018-04-17 |
EP3276959A4 (en) | 2018-10-17 |
EP3869799B1 (en) | 2022-09-14 |
EP3276959A1 (en) | 2018-01-31 |
US20190320201A1 (en) | 2019-10-17 |
EP3869799A1 (en) | 2021-08-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11432005B2 (en) | Moving image encoding device | |
US10091500B2 (en) | Video coding method using at least evaluated visual quality and related video coding apparatus | |
JP5793511B2 (en) | Deblocking filtering control | |
JP4246723B2 (en) | Intraframe predictive coding control method, intraframe predictive coding control apparatus, intraframe predictive coding control program, and computer-readable recording medium storing the program | |
JP5276170B2 (en) | Method and apparatus for detecting banding artifacts | |
CN110036637B (en) | Method and device for denoising and vocalizing reconstructed image | |
JP5553979B2 (en) | Selection of coding tools for video coding based on human visual tolerance | |
US10869042B2 (en) | Template based adaptive weighted bi-prediction for video coding | |
US20130058420A1 (en) | Decoding apparatus, decoding method, and computer-readable storage medium | |
JP5491506B2 (en) | Method and apparatus for detecting dark noise artifacts | |
JP2006067302A (en) | Apparatus, method, and program for image encoding | |
US10419775B2 (en) | Moving image encoding apparatus, image capturing apparatus, moving image encoding method, and non-transitory computer readable storage medium | |
JP4829951B2 (en) | Intraframe predictive coding control method, intraframe predictive coding control apparatus, intraframe predictive coding control program, and computer-readable recording medium storing the program | |
JP6012307B2 (en) | Moving picture coding apparatus, control method therefor, and program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: NTT ELECTRONICS CORPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SHIMAZAKI, AKIKO;NAKAMURA, KEN;ONISHI, TAKAYUKI;AND OTHERS;REEL/FRAME:043203/0149 Effective date: 20170628 Owner name: NIPPON TELEGRAPH AND TELEPHONE CORPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SHIMAZAKI, AKIKO;NAKAMURA, KEN;ONISHI, TAKAYUKI;AND OTHERS;REEL/FRAME:043203/0149 Effective date: 20170628 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |