RU2737038C2 - Image processing device and method - Google Patents

Abstract

FIELD: encoding and decoding of images.

SUBSTANCE: image processing device comprises a decoding unit configured to generate a decoded image by decoding an encoded stream; and a filter unit configured to apply unlocking filtering to neighboring pixels adjacent to the boundary between blocks in the decoded image formed by the decoding unit, using a filter parameter set in accordance with a range, including a value greater than 9 and a corresponding quantisation parameter greater than or equal to 45 in the ratio between the quantisation parameter and the filter parameter, set when selecting an unlocking filter force or limiting filtered pixels after an unlocking filter. Present invention can be used, for example, in image processing devices.

EFFECT: technical result consists in block noise suppression.

24 cl, 66 dwg

Description

Technical field to which the invention relates

The present disclosure relates to an image processing apparatus and method, and in particular to an image processing apparatus and method capable of suppressing block noise.

State of the art

In recent years, devices that digitally process image information have become widespread, which in this case, for transmitting and storing information with high efficiency, perform compressed image coding, adapting a coding method in which compression is performed by an orthogonal transform such as discrete cosine transform. and motion compensation using redundancy unique to the image information. Examples of the encoding method are MPEG (Moving Picture Experts Group) or H.264 and MPEG-4 Part 10 (Advanced Video Coding; hereinafter referred to as H.264 / AVC), etc.

In addition, at present, in order to further improve the coding efficiency compared to H.264 / AVC, the JCTVC (Joint Collaboration Team-Video Coding) group, which is the joint ITU-T (International Telecommunication Union - Telecommunications sector) and ISO / IEC (International Organization for Standardization / International Electrotechnical Commission), a standardization of a coding method called HEVC (High Efficiency Video Coding) was developed. For the HEVC standard, the Committee's draft, which is the first draft specification, was released in February 2012 (for example, see Non-Patent Document 1).

In the HEVC project, at the moment, the deblocking filter, the adaptive loop filter and the adaptive displacement filter are designed to be built into the loop and there are two deblocking filter parameters, which are parameter β and parameter Tc. Parameter β has 52 values from 0 to 51 and is fixed as 64 if more than 51 is entered. Parameter Tc has 54 values and is fixed as 13 if more than 53 is entered.

The range of the quantization parameter QP that can be used in HEVC is 0 to 51, but the offset range of the deblocking filter parameter is –26 to 26. Therefore, in theory, the input range in the LUT (Look Up Table) for the deblocking filter parameter is ranges from 0 to 77.

List of literature

Non-patent documents

Non-Patent Document 1: Benjamin Bross, Woo-Jin Han, Jens-Rainer Ohm, Gary J. Sullivan, Thomas Wiegand, "High efficiency video coding (HEVC) text specification draft 6", JCTVC-H1003 ver20, 2012.2.17

Disclosure of invention

Problems Solved by the Invention

However, since the quantization parameter QP, which is the actual input of the deblocking filter parameter in the LUT, is 0-51, the maximum offset value is not used.

In addition, as described above, even if 64, which is the maximum value of the β parameter, and the value 13, which is the maximum value of the Tc parameter in HEVC, are used, there have been cases in which block noise that is visible remains in some sequences.

The present disclosure has been made in view of such a situation and is suitable for block noise suppression.

Problem solution

An image processing apparatus according to the first embodiment of the present disclosure comprises: a decoding unit forming an image by decoding a coded stream; and a filter unit performing filtering with a filter using a parameter obtained by expanding the range of a filter parameter to control the filter so as to increase the filtering strength to form a block border of an image by the decoding unit.

The filter parameter range is expanded to increase the filtering strength of the filter in a state consistent with the existing range.

The filter parameter range is adjusted to be continuous over the existing area and the extended area.

The filter parameter range is expanded by adding a new filter parameter.

A filter parameter is a threshold value used in determining whether or not to filter.

The filter is an deblocking filter and the filter parameter is a parameter used when determining whether or not a deblocking filter is required or when determining the choice of filter strength.

The filter parameter range expands in a linear fashion.

The angle of inclination of the linear shape can be adjusted.

The filter parameter range expands in a curved shape.

The filter is an unblocking filter and the filter parameter is a parameter used in determining the choice of filtering strength of the unblocking filter, or used in constraint when filtering by the filter.

The filter block may perform filtering with a filter using a parameter extended by updating the table using the slope of a computational expression that as a target expresses a filter parameter with a block edge of the image generated by the decoding block.

The filter block may perform filtering with a filter using a parameter extended by adding a filter parameter offset to the filter parameter, with the block edge of the image generated by the decoding block as the target.

The filter parameter may contain two types of parameters that use the common offset, and optionally, a parameter calculator may be provided that calculates the two types of parameters using the common offset.

The filter parameter may contain two types of parameters, and further may be provided a first parameter calculating unit calculating the first filter parameter using an offset of the first parameter, and a second parameter calculating unit calculating a second filter parameter using an offset of the first parameter, the difference between the first and second filter parameters, and offset of the second parameter.

The filter is a deblocking filter.

The filter is an adaptive bias filter.

The decoding unit can decode a coded stream in a processing unit having a hierarchical structure.

An image processing method of an image processing apparatus of the first embodiment of the present disclosure comprises a step of: generating an image by decoding a coded stream; and performing filtering with a filter using the parameter obtained by expanding the range of the filter parameter to control the filter so as to increase the filtering strength, with the block edge of the image generated by the decoding unit as a target.

An image processing apparatus according to a second embodiment of the present disclosure comprises: a filter unit performing filtering with a filter using a parameter obtained by expanding the filter parameter range to control the filter so as to increase the filtering strength, with the block edge of the locally decoded image as a target; an encoding unit generating a codestream by encoding an image using an image filtered by a filter of the filter unit; and a transmitting unit transmitting a coded stream generated by the coding unit.

An image processing method of an image processing apparatus of the second embodiment of the present disclosure comprises the steps of: performing filtering with a filter using a parameter obtained by expanding the range of a filter parameter to control the filter so as to increase the filtering strength, with a block boundary of a locally decoded image as a target ; form a coded stream by encoding the image and using for this the image filtered by the filter; and transmitting the generated codestream.

In the first embodiment of the present disclosure, the codestream is decoded to form an image. Then, using the block boundary as the target, the filter is filtered using the parameter obtained by expanding the range of the filter parameter to control the filter to increase the strength of the filter.

In addition, in the second embodiment of the present disclosure, with a block boundary of a locally decoded image as a target, filtering is performed with a filter using a parameter obtained by expanding the range of a filter parameter to control the filter to increase the filtering strength. Then, a codestream is generated by encoding a picture using a filtered image, and the generated codestream is transmitted.

In addition, the image processing apparatus described above may be an independent apparatus, or may be an internal unit that configures an image encoding apparatus or an image decoding apparatus.

Results of the invention

According to the first embodiment of the present disclosure, it is possible to decode an image. In particular, it is possible to suppress block noise.

According to a second embodiment of the present disclosure, it is possible to encode an image. In particular, it is possible to suppress block noise.

Brief Description of Drawings

FIG. 1 is a block diagram of an exemplary typical structure of an image encoding apparatus.

FIG. 2 is a flow diagram of an exemplary encoding process flow.

FIG. 3 is a block diagram of an exemplary typical structure of an image decoding apparatus.

FIG. 4 is a flowchart of an exemplary decoding process flow.

FIG. 5 is a value and graph of a deblocking filter parameter α in an AVC method.

FIG. 6 is a value and graph of a deblocking filter parameter β in an AVC method.

FIG. 7 shows the values of the parameters β and Tc corresponding to the quantization parameter Q in the HEVC method.

FIG. 8 shows the relationship between the quantization parameter QP and the parameter β in the first extension method of the present method.

FIG. 9 - values and graph of the quantization parameter QP and parameter β in the case of expansion as a linear function for k = 2.

FIG. 10 - values and graph of the quantization parameter QP and parameter β in the case of expansion as a linear function for k = 4.

FIG. 11 shows the values and graph of the quantization parameter QP and the parameter β in the case of expansion as a linear function for k = 8.

FIG. 12 is a value and graph of the quantization parameter QP and the parameter β in the case of expansion as an exponential function.

FIG. 13 is a value and graph of a quantization parameter QP and a parameter Tc in the first spreading method of the present technology.

FIG. 14 is a block diagram of a configuration example of a deblocking filter to which the present technology is applied.

FIG. 15 is a block diagram of a configuration example of a filter execution unit performing luminance edge filtering.

FIG. 16 is a block diagram of a configuration example of the β generating unit.

FIG. 17 is a block diagram of a configuration example of an extended β generating unit.

FIG. 18 is a block diagram of another configuration example of the extended β generating unit.

FIG. 19 is a block diagram of a configuration example of a Tc generating unit.

FIG. 20 is a block diagram of a configuration example of an extended Tc generating unit.

FIG. 21 is a block diagram of another configuration example of an extended Tc generating unit.

FIG. 22 is a block diagram of a configuration example of a filter execution unit performing color contrast border filtering.

FIG. 23 is a flowchart of the deblocking filtering process.

FIG. 24 is a flowchart for performing luma border filtering operations.

FIG. 25 is a flowchart of the β generation process.

FIG. 26 is a flowchart illustrating a limitation.

FIG. 27 is a flowchart of a Tc generation process.

FIG. 28 is a flowchart illustrating another limiting example.

FIG. 29 is a flowchart illustrating color contrast edge filtering.

FIG. 30 is a value and graph of the quantization parameter QP and the parameter β (new β) in the second expansion method of the present technology.

FIG. 31 illustrates a relationship between a QP quantization parameter and a tc parameter and an adjustment amount for an approximate expression.

Fig. 32 illustrates an example of a parameter β in a third expansion method according to the present technology.

FIG. 33 illustrates an example of a Tc parameter in a third spreading method according to the present technology.

FIG. 34 is a value and graph of the quantization parameter QP and the parameter β in the case of expansion within the quantization parameter QP (0-51).

FIG. 35 is a block diagram of a configuration example of the β generating unit.

FIG. 36 is a flowchart of the β generation process.

FIG. 37 is a block diagram of a configuration example of a deblocking filter to which the present technology is applied.

FIG. 38 is a block diagram of a configuration example of a filter execution unit performing luminance edge filtering.

FIG. 39 is a block diagram of a configuration example of the β generating unit.

FIG. 40 is a block diagram of a configuration example of a Tc generating unit.

FIG. 41 is a flowchart of the deblocking filtering process.

FIG. 42 is a flowchart showing a filter installation process.

FIG. 43 is a flowchart of the β generation process.

FIG. 44 is a flowchart of a Tc generation process.

FIG. 45 - transfer of displacement.

FIG. 46 is an example syntax.

FIG. 47 is a block diagram of a configuration example of a deblocking filter to which the present technology is applied.

FIG. 48 is a flowchart showing a filter configuration setting process.

FIG. 49 is a block diagram of a configuration example of a deblocking filter to which the present technology is applied.

FIG. 50 is a flowchart showing a filter configuration setting process.

FIG. 51 - transfer of displacement.

FIG. 52 is an example of syntax.

FIG. 53 is an example of a method for encoding multi-projection images.

FIG. 54 is a block diagram of a configuration example of a deblocking filter to which the present technology is applied.

FIG. 55 is an example of a basic configuration of a multi-projection image decoding apparatus to which the present technology is applied.

FIG. 56 is an example of a method for encoding hierarchical images.

FIG. 57 is an example of a basic configuration of a hierarchical picture decoding apparatus to which the present technology is applied.

FIG. 58 is a general configuration example of a hierarchical picture decoding apparatus to which the present technology is applied.

Fig. 59 is a block diagram of an example of a typical computer structure.

FIG. 60 is a block diagram of one example of the basic structure of a television device.

FIG. 61 is a block diagram of one example of the basic structure of a mobile telephone device.

FIG. 62 is a block diagram of one example of a basic structure of a recording / reproducing apparatus.

FIG. 63 is a block diagram of a configuration example of an image acquiring apparatus.

FIG. 64 is a block diagram of an example of using scalable coding.

FIG. 65 is a block diagram of another example of using scalable coding.

FIG. 66 is a block diagram of yet another example of using scalable coding.

Method for carrying out the invention

Methods for carrying out the present disclosure (hereinafter referred to as embodiments) will be described below. The description will be done in the following order.

1. General information about the device and operating procedure

2. Explanation of the traditional way

3. First embodiment (first extension method)

4. Second embodiment (second extension method)

5. Third embodiment (third extension method)

6. Fourth embodiment (fourth extension method)

7. Fifth Embodiment (First Bias Transfer Method)

8. Sixth Embodiment (Second Bias Transfer Method)

9. Seventh Embodiment (Multi-Projection Image Coding / Multi-Projection Decoding Apparatus)

10. Eighth embodiment (hierarchical picture encoding / hierarchical picture decoding apparatus)

11. Ninth embodiment (computer)

12. Applications

13. Applications of scalable coding

1. General information about the device and operating procedure

Example of the structure of an image encoder

FIG. 1 shows the configuration of an embodiment of an image decoding apparatus as an image processing apparatus to which the present disclosure is applied.

The image encoding apparatus 11 shown in FIG. 1 encodes image data using a prediction process. Here, as the coding method, for example, HEVC (High Efficiency Video Coding) method or the like is used.

Additionally, in the HEVC method, a coding unit (CU) is defined. CU is also referred to in relation to a coding tree block (CTB), and is a partial area of the picture of a picture block, which plays the same role as a macroblock in the H.264 / AVC method. Although the latter is fixed at 16x16 pixels, the former is not fixed and is described in the image compression information in each of the sequences.

For example, in the sequence parameter set (SPS) contained in the encoded data to be outputted, the maximum size (LCU (Largest Coding Unit)) and the minimum size (SCU (Smallest Coding Unit)) of the block are defined. coding CU.

Within each LCU, division into smaller CUs can be performed by setting split_flag = 1 in a range no less than the SCU size. The 2Nx2N CU is divided into NxN CUs, which are one level lower when the split_flag is "1".

In addition, the CU is divided into prediction units (PU), which is an area (partial image area of a picture block) to be subjected to an intra- or inter-prediction process, and is divided into transform units (TU), which is an area (a partial image area of a picture block) that is must undergo an orthogonal transformation process. Currently, according to HEVC, 16x16 and 32x32 orthogonal transforms can be used in addition to 4x4 and 8x8 transforms.

The image encoding apparatus 11 shown in FIG. 1, contains an analog-to-digital (A / D) converter 21, a frame rearrangement buffer 22, an arithmetic unit 23, an orthogonal converter 24, a quantizer 25, a lossless encoder 26, and a storage buffer 27. The encoding device 11 comprises an inverse quantizer 28, an inverse orthogonal a converter 29, an arithmetic operation unit 30, a deblocking filter 31a, a frame memory 32, a switch 33, an intra-frame predictor 34, a motion analysis / compensator 35, a prediction image switch 36, and a speed controller 37.

In addition, the image coding apparatus 11 comprises an adaptive offset filter 41 and an adaptive loop filter 42 between the deblocking filter 31 and the frame memory 32.

The A / D converter 21 performs A / D conversion on the input image data, outputs the image data to the frame rearrangement buffer 22, and stores the image data therein.

The frame rearranging buffer 22 rearranges the picture frames stored in the display order into the frame order for encoding according to the GOP (Group of Picture) structure. The frame rearranging buffer 22 supplies the image in which the frames have been rearranged to the arithmetic unit 23. In addition, the frame rearrangement buffer 22 also supplies an image in which the frames have been rearranged to the intra-frame predictor 34 of the motion analysis / compensator 35.

The arithmetic unit 23 subtracts the prediction image supplied from the intra-frame predictor 34 or the motion analysis / compensator 35 through the prediction image switch 36 from the image read from the image rearrangement buffer 22 and outputs the resulting difference information to the orthogonal transformer 24.

For example, in the case of a picture on which encoding is performed, the arithmetic operation unit 23 subtracts the prediction picture supplied from the intra-predictor 34 from the picture read from the frame rearrangement buffer 22. In addition, for example, in the case of an image on which encoding is performed, the arithmetic operation unit 23 subtracts a prediction image supplied from the motion analysis / compensator 35 from the image read from the frame rearrangement buffer 22.

The orthogonal transformer 24 performs an orthogonal transform, such as a discrete cosine transform or a Karunen-Loeve transform, on the difference information supplied from the arithmetic operations unit 23, and supplies the transform coefficient to the quantizer 25.

Quantizer 25 quantizes a transform coefficient supplied from an orthogonal transformer 24. Quantizer 25 provides a quantized transform coefficient to encoder 26 without loss.

Lossless encoder 26 performs lossless coding such as variable length coding and arithmetic coding for the quantized transform coefficient.

Lossless encoder 26 obtains a parameter such as information indicating the intra-transform mode from the intra-predictor 34 and receives from the motion analysis / compensator device 35 a parameter such as information indicating the inter-prediction mode or motion vector information. Lossless encoder 26 encodes the quantized transform coefficient and also encodes each received parameter (syntax element) and sets the result as part of the header information of the encoded data (multiplexes the result). Lossless encoder 26 supplies encoded data obtained by encoding to storage buffer 27 and stores encoded data therein.

For example, in the lossless encoder 26, a lossless coding process such as variable length coding or arithmetic coding is performed. As the variable length coding, CAVLC (Context Adapted Variable Length Coding) and the like can be mentioned. As the arithmetic coding, CABAC (Context Adapted Binary Arithmetic Coding) and the like can be mentioned.

The storage buffer 27 temporarily stores the encoded stream (data) supplied from the lossless encoder 26, and outputs the stored encoded data to, for example, a subsequent recording device (not shown) or a transmission path as an encoded picture at a predetermined time. That is, the storage buffer 27 is also a transmission unit for transmitting the codestream.

In addition, the transform coefficient quantized by the quantizer 25 is also supplied to the inverse quantizer 28. The inverse quantizer 28 inversely quantizes the quantized transform coefficient using a method corresponding to quantization by the quantizer 25. The inverse quantizer 28 supplies the obtained transform coefficient to the inverse orthogonal transformer 29.

The inverse orthogonal transformer 29 performs an inverse orthogonal transformation on the supplied transform coefficient in a manner corresponding to the orthogonal transformation process by the orthogonal transformer 24. The output obtained by the inverse orthogonal transform (reconstructed difference information) is supplied to the arithmetic unit 30.

The arithmetic operation unit 30 adds the prediction image supplied from the intra-frame predictor 34 or from the motion analysis / compensator 35 via the prediction image switch 36 to the result of the inverse orthogonal transform, that is, to the reconstructed difference information supplied from the inverse orthogonal transformer 29 to obtain locally decoded picture (decoded picture).

For example, when the difference information corresponds to a picture for which intra-frame coding is performed, the arithmetic operation unit 30 adds the prediction picture supplied from the intra-predictor 34 to the difference information. In addition, for example, when the difference information corresponds to an image for which inter-coding is performed, the arithmetic operation unit 30 adds a prediction image supplied from the motion analysis / compensator 35 to the difference information.

The decoded image resulting from the addition is supplied to the deblocking filter 31a and frame memory 32.

The deblocking filter 31a suppresses block distortion of the decoded image by appropriately performing a deblocking filtering process. The deblocking filter 31 has parameters β and Tc calculated based on the quantization parameter QP. Parameters β and Tc are threshold values (parameters) used in the determination with respect to the deblocking filter. β and Tc, which are parameters that the deblocking filter 31a has, are derived from β and Tc, which are determined in the HEVC method.

In addition, the deblocking filter 31a can be turned off by a user command, and the on / off information as to whether or not to perform deblocking filtering is input to the deblocking filter 31a by the user via an operation unit (not shown), etc. In addition, the value on the default for each offset of the above-described parameters β and Tc is 0. However, when the default value for each offset of the above-described β and Tc is a value other than 0, the user inputs a value into the deblocking filter 31a via an operation unit (not shown) and the like. Information on the enable / disable (also referred to as the DisableDeblockingFilter flag information) of the deblocking filter and each offset of the β and Tc parameters are encoded as a deblocking filter parameter by the lossless encoder 26 and transmitted to the picture decoding apparatus 51 shown in FIG. 3, which should be described below.

The deblocking filter 31a performs deblocking filtering on the image obtained from the arithmetic unit 30 using the on / off and offset information described above and β and Tc, which are parameters derived from β and Tc determined in the HEVC method. The deblocking filter 31a supplies the filtering result to the adaptive bias filter 41. In addition, a detailed configuration of the deblocking filter 31a will be described later with reference to FIG. fourteen.

The adaptive offset filter 41 performs a sample adaptive offset (SAO: Sample adaptive offset) filtering process for substantially suppressed ringing in the image after being filtered by the deblocking filter 31a.

There are nine types of offset filters in total, including two types of band offset, six types of edge offset, and no offset. An adaptive offset filter 41 filters the image after filtering with a deblocking filter 31a using an offset value for each quad tree structure, in which the type of offset filter is determined for each subdivided area and in each subdivided area. An adaptive offset filter 41 supplies the filtered image to an adaptive loop filter 42.

In addition, in the image coding apparatus 11, an offset value for each quad tree structure and divided area is calculated by the adaptive offset filter 41 and used. The computed offset value for each quadtree structure and subdivision is encoded as an adaptive offset parameter by a lossless encoder 26 and transmitted to an image coding device 51 shown in FIG. 3, which will be described later.

The adaptive loop filter 42 performs an adaptive loop filtering (ALF: AdaptiveLoop Filter) process on the image after being filtered by the adaptive bias filter 41 in the processor unit using the filter coefficient. The adaptive loop filter 42 uses a two-dimensional Wiener filter as a filter. Of course, it is also possible to use filters other than the Wiener filter. The adaptive loop filter 42 supplies the result of the filtering to frame memory 32.

In addition, although in the example of FIG. 1, this is not shown, in the image coding device 11, the filter coefficient is calculated by the adaptive loop filter 42, so that the residual error between each processing unit and the original image from the frame rearrangement buffer 22 is minimized and used. The calculated filter coefficient is encoded as an adaptive loop filter parameter by the lossless encoder 26 and transmitted to the image encoding device 51 shown in FIG. 3, which will be described later.

The frame memory 32 outputs the accumulated reference image to the intra-frame predictor 34 or to the motion analysis / compensator 35 through the switch 33 at a predetermined time.

For example, in the case of a picture on which intra-frame coding is performed, the frame memory 32 supplies a reference image to the intra-predictor 34 via the switch 33. In addition, when the inter-frame coding is performed, the frame memory 32 supplies the reference image to the motion analysis / compensator 35 via the switch 33. ...

When the reference picture supplied from the frame memory 32 is a picture on which intra-coding is performed, the switch 33 supplies the reference picture to the intra-predictor 34. In addition, when the reference picture supplied from the frame memory 32 is a picture on which inter-coding is performed , the switch 33 supplies the reference image to the motion analysis / compensator 35.

The intra-predictor 34 performs intra-prediction (intra-prediction) to generate a prediction image using the pixel values within the screen. The intra-predictor 34 performs intra-prediction in a plurality of modes (intra-prediction modes).

The intra-predictor 34 generates prediction images in all intra-prediction modes, evaluates each prediction image, and selects the optimal mode. When the intra-prediction optimal mode is selected, the intra-predictor 34 supplies the prediction image generated in the optimal mode to the arithmetic operation unit 23 or to the arithmetic operation unit 30 via the prediction image switch 36.

In addition, as described above, the intra predictor 34 appropriately supplies a parameter such as intra prediction mode information indicating the adapted intra prediction mode to the lossless encoder 26.

The motion analysis / compensator device 35 performs motion analysis on an image on which inter-coding is performed using an input image supplied from the frame rearrangement buffer 22 and a reference image supplied from the frame memory 32 through a switch 33. In addition, the analysis / the motion compensator performs a motion compensation process in accordance with a motion vector determined by motion analysis, thereby generating a prediction image (interframe image information).

The motion analysis / compensator 35 generates predicted images by performing an inter prediction process in all possible inter prediction modes. The motion analysis / compensator 35 supplies the generated prediction images to the arithmetic operation unit 23 or to the arithmetic operation unit 30 via the prediction image switch 36.

In addition, the analysis / compensator 35 provides a parameter such as inter-prediction mode information indicating an adapted inter-prediction mode or motion vector information indicating a computed motion vector to the lossless encoder 26.

The prediction image switch 36 supplies the output of the intra-predictor 34 to the arithmetic unit 23 or to the arithmetic unit 30 in the case of images on which intra-frame coding is performed, and supplies the output of the analysis / compensator 35 to the arithmetic unit 23 or to the arithmetic unit 30 operations in the case of a picture on which inter-coding is performed.

The rate controller 37 controls the speed of the quantization operation of the quantizer 25 based on the compressed image accumulated in the accumulation buffer 27 so that no overflow or negative overflow occurs.

Operation of the image coding device

A flow of an encoding process performed by the above-mentioned image encoding apparatus 11 will be described with reference to FIG. 2.

In step S11, the A / D converter 21 performs A / D conversion on the input image. In step S12, the frame rearranging buffer 22 stores the A / D conversion image and rearranges the corresponding pictures in the display order into the encoding order.

When the picture to be processed and supplied from the frame rearrangement buffer 22 is a block picture to be intra-processed, the decoded picture to be referenced is read from the frame memory 32 and supplied to the intra predictor 34 via switch 33.

In step S13, based on these pictures, the intra-predictor 34 intra-predict the pixels of the block to be processed in all possible intra-prediction modes. In addition, a pixel that is not filtered by the deblocking filter 31 is used as the decoded pixel to be referenced.

With this processing, intra prediction is performed in all possible intra prediction modes, and cost function values are calculated for all possible inter prediction modes. Then, based on the calculated values of the cost function, an intra-prediction optimal mode and a prediction image generated by intra-prediction in the intra-prediction optimal mode are selected, and the cost function value is supplied to the prediction image switch 36.

When the image to be processed and which is supplied from the frame rearrangement buffer 22 is an image to be interframe processed, the image to be referenced being read from the frame memory 32 and supplied to the analysis / compensator 35, the processing is performed with analysis / motion compensation based on these images.

With this processing, motion analysis processing is performed in all possible inter-prediction modes, and cost function values are calculated for all possible inter-prediction modes. Based on the calculated values of the cost function, the optimal inter-prediction mode is determined. Then, the prediction image generated in the inter prediction optimal mode and its cost function value are supplied to the prediction mode switch 36.

In step S15, the prediction image switch 36 determines whether the optimal intra prediction mode or the optimal inter prediction mode is selected as the optimal prediction mode based on the respective values of the cost function output from the intra predictor 34 and the motion analysis / compensator 35. Then, the prediction image switch 36 selects the prediction images in the determined optimal prediction mode and supplies them to the arithmetic units 23 and 30. The prediction image is used in the operations in steps S16 and S21, which will be described later.

In addition, the prediction image selection information is supplied to the intra predictor 34 or to the motion analysis / compensator 35. When a prediction picture is selected in the intra-prediction optimal mode, the intra-predictor 34 provides information indicating the intra-prediction optimal mode (ie, a parameter related to intra prediction) to the lossless encoder 26.

When the prediction image is selected in the inter prediction optimal mode, the motion analysis / compensation apparatus 35 outputs information indicating the inter prediction optimal mode and information related to the inter prediction optimal mode (i.e., a parameter related to motion analysis) to the encoder 26 without losses. As the information corresponding to the optimal inter-prediction mode, motion vector information, reference frame information, and the like can be mentioned.

In step S16, the arithmetic operation unit 23 calculates the difference between the image rearranged in step S12 and the prediction image selected in step S15. The prediction image is supplied from the motion analysis / compensator 35 to the arithmetic operation unit 23 through the prediction image switch 36 in the case of inter-prediction and from the intra-predictor 34 to the arithmetic operation unit 23 through the prediction image switch 36 in the case of intra-prediction.

The amount of data for the difference data is reduced compared to that of the original image. Accordingly, it is possible to compress the amount of data as compared with the case in which the picture is encoded as it is.

In step S17, the orthogonal converter 24 performs orthogonal transformation of the difference information obtained from the arithmetic operation unit 23. Specifically, an orthogonal transform such as a discrete cosine transform or a Karunen-Loeve transform is performed, and the transform coefficient is further supplied.

In step S18, the quantizer 25 quantizes the transform coefficient. In quantization, the rate is controlled as explained for the process in step S28, which will be described later.

The difference information quantized as described above is locally decoded as follows. That is, in step S19, the inverse quantizer 28 performs inverse quantization of the transform coefficient quantized by the quantizer 25 with characteristics corresponding to those of the quantizer 25. In step S20, the inverse orthogonal transformer 29 performs an inverse orthogonal transform on the transform coefficient that has been inversely quantized by the inverse quantizer 28 with characteristics corresponding to the characteristics of the orthogonal transformer 24.

In step S21, the arithmetic operation unit 30 generates a locally decoded image (an image corresponding to the input to the arithmetic operation unit 23) by adding a prediction image input through the prediction image switch 36 to the locally decoded difference information.

In step S22, the deblocking filter 31a performs deblocking filtering on the image outputted from the arithmetic operation unit 30. Next, referring to FIG. 23, the details of the deblocking filtering process will be described. In this case, the parameters β and Tc taken for β and Tc determined in the HEVC method are used as detection thresholds with respect to the deblocking filter. The filtered image from the deblocking filter 21 is output to an adaptive offset filter 41.

In addition, the ON / OFF information used in the deblocking filter 31a and each offset of the β and Tc parameters that have been inputted by the operation of an operation unit (not shown) or the like by the user are supplied to the encoder 26 without loss. as parameters of the deblocking filter.

In step S23, the adaptive offset filter 41 performs adaptive offset filtering. Through this process, image filtering is performed after filtering with the deblocking filter 31a using an offset value for each quad tree structure, in which the type of offset filter is determined for each subdivided area as well as in each subdivided area. The displaced image is fed to the adaptive contour filter 42.

In addition, a specific offset value for each quad tree structure and subdivision is provided to lossless encoder 26 as an adaptive offset parameter.

In step S24, the adaptive offset filter 42 performs adaptive contour filtering of the image after being filtered by the adaptive offset filter 41. For example, filtering is performed on the image after filtering by the adaptive displacement filter 41 in the processing unit using the filter coefficient, and the filtering result is supplied to the frame memory 32.

In step S25, the frame memory 32 stores the filtered image. In addition, images not filtered by the deblocking filter 31a, the adaptive offset filter 41, and the adaptive loop filter 42 are also supplied from the arithmetic unit 30 to the frame memory 32 and stored.

On the other hand, the transform coefficient quantized in step S18 described above is also supplied to the lossless encoder 26. In step S26, the lossless encoder 26 encodes the quantized transform coefficient outputted from the quantizer 25 and each parameter that is supplied. That is, the difference image is compressed by lossless coding such as variable length coding or arithmetic coding. Here, since the parameters are encoded, there is a deblocking filter parameter, an adaptive offset filter parameter, an adaptive loop filter parameter, a quantization parameter, motion vector information or reference frame information, prediction mode information, and the like.

In step S27, the storage buffer 27 accumulates an encoded difference image (i.e., a coded stream) as a compressed image. The compressed image accumulated in the storage buffer 27 is appropriately read out and transmitted to the decoding side along the transmission path.

In step S28, the rate controller 37 controls the speed of the quantization operation of the quantizer 25 based on the compressed image accumulated in the accumulation buffer 27 so that no overflow or negative overflow occurs.

The encoding process ends when the processing in step S28 ends.

Example of the structure of an image decoding device

FIG. 3 is a configuration of an embodiment of an image decoding apparatus in the form of an image processing apparatus to which the present disclosure is applied. The picture decoding apparatus 51 shown in FIG. 3 is an apparatus corresponding to the image encoding apparatus 11 shown in FIG. 1.

It is assumed that the encoded stream (data) that is encoded by the image encoding apparatus 11 is transmitted to the image decoding apparatus 51 corresponding to the image encoding apparatus 11 along a predetermined transmission path and decoded.

As shown in FIG. 3, the image decoding device 51 comprises a storage buffer 61, a lossless decoder 62, an inverse quantizer 63, an inverse orthogonal transformer 64, an arithmetic unit 65, a deblocking filter 31b, a frame rearrangement buffer 67, and a D / A converter 68. Image decoding device 51 also contains a frame memory 69, a switch 70, an intra-frame predictor 71, a motion analysis / compensator 72, and a switch 73.

In addition, the image decoding apparatus 51 includes an adaptive offset filter 81 and an adaptive loop filter 82 between the deblocking filter 31b and the frame rearranging buffer 67 and the frame memory 69.

The storage buffer 61 is also a receiving unit that receives the transmitted encoded data. The storage buffer 61 receives and stores the transmitted encoded data. The encoded data was encoded by the image encoding device 11. Lossless decoder 62 decodes encoded data read from storage buffer 61 at a predetermined time using a method corresponding to the method for encoding lossless encoder 26 shown in FIG. 1.

Lossless decoder 62 supplies a parameter such as information indicating the decoded intra prediction mode to the intra predictor 71, and supplies a parameter such as information indicating the inter prediction mode or motion vector information to the motion analysis / compensator 72. In addition, the lossless decoder 62 supplies the decoded deblocking filter parameter to the deblocking filter 31b, and supplies the decoded adapted offset parameter to the adaptive offset filter 81.

The inverse quantizer 63 inversely quantizes the data of the coefficient (quantization coefficient) obtained by decoding by lossless encoder 62 using a method corresponding to the quantization method of quantizer 25 shown in FIG. 1. That is, the inverse quantizer 63 inversely quantizes the quantization coefficient in the same manner as the inverse quantizer 28 shown in FIG. 1 using a quantization parameter obtained from the image encoding apparatus 11.

The inverse quantizer 63 supplies inverse quantized coefficient data, that is, the orthogonal transform coefficient, to the inverse orthogonal transformer 64. The inverse orthogonal transformer 64 performs inverse orthogonal transformation on the orthogonal transform coefficient using a method corresponding to the orthogonal transform method of the orthogonal transformer 24 shown in FIG. 1, thereby obtaining decoded residual data corresponding to residual data before orthogonal transformation in the image encoding apparatus 11.

The decoded residual data obtained by the inverse orthogonal transform is supplied to an arithmetic operation unit 65. In addition, the prediction image is supplied to the arithmetic unit 65 from the intra-frame predictor 71 or from the motion analysis / compensator 72 through the switch 73.

The arithmetic unit 65 adds the decoded residual data and the prediction image to obtain decoded image data corresponding to the image data before the prediction image is subtracted by the arithmetic unit 23 of the image encoding apparatus 11. The arithmetic operation unit 65 supplies the decoded image data to the deblocking filter 31b.

The deblocking filter 31b suppresses block distortion of the decoded image by appropriately performing the deblocking filtering process. The deblocking filter 31b is configured in substantially the same manner as the deblocking filter 31a shown in FIG. 1. That is, the deblocking filter 31b has parameters β and Tc calculated based on the quantization parameter. Parameters β and Tc are the threshold values used in the definition related to the deblocking filter. β and Tc, which are parameters that the deblocking filter 31b has, are obtained from β and Tc, which are defined in the HEVC method.

In addition, the on / off information of the deblocking filter and each parameter offset β and Tc that are encoded by the image encoding device 11 are received as the parameters of the deblocking filter by the image decoding apparatus 51, decoded by the decoder 62 without loss, and used by the deblocking filter 31b.

The deblocking filter 31b performs deblocking filtering on the image obtained from the arithmetic operation unit 30 using the on / off information and the offset described above, and β and Tc, which are parameters derived from β and Tc determined in the HEVC method. The deblocking filter 31b supplies the filtering result to the adaptive bias filter 81. In addition, a detailed configuration of the deblocking filter 31b will be described later with reference to FIG. fourteen.

The adaptive offset filter 81 performs offset filtering (SAO) mainly to suppress ringing in the image after being filtered by the deblocking filter 31b.

The adaptive offset filter 81 filters the image after filtering with the deblocking filter 31b using the offset value for each quad tree structure, in which the type of the offset filter is determined for each subdivided area and in each subdivided area. An adaptive displacement filter 81 supplies the filtered image to an adaptive loop filter 82.

In addition, the offset value for each quad tree structure and divided area is calculated by the adaptive offset filter 41 of the image coding apparatus 11 and encoded and transmitted as an adaptive offset parameter. The offset value for each quad tree structure and subdivision that has been encoded by the image coding apparatus 11 is received as an adaptive offset parameter by the image decoding apparatus 51, decoded by the lossless decoder 62, and used by the adaptive offset filter 81.

The adaptive loop filter 82 filters the image after being filtered by the adaptive displacement filter 81 in the processing unit using the filter coefficient, and the result of the filtering is fed to the frame memory 69 in the frame rearrangement buffer 67.

In addition, although in the example of FIG. 3, this is not shown, in the picture decoding apparatus 51, the filter coefficient is calculated for each LUC by the adaptive loop filter 42 of the image coding apparatus 11, and the coefficient encoded and transmitted as an adaptive loop filter parameter is decoded by the lossless decoder 62 and used.

The frame rearrangement buffer 67 rearranges images. Specifically, the frames rearranged in order of encoding by the frame rearrangement buffer 22 in FIG. 1 are rearranged to their original display order. The D / A transformer 68 D / A converts the image supplied from the frame rearrangement buffer 67 and outputs the converted image to a display (not shown) for displaying the image.

In addition, the output of the adaptive loop filter 82 is supplied to frame memory 69.

The frame memory 69, switch 70, intra predictor 71, motion analysis / compensator 72, and switch 73 correspond to frame memory 32, switch 33, intra predictor 34, motion analysis / compensator 35, and predictive image switch 36 of encoder 11, respectively.

The switch 70 reads the inter-frame processed image and the image to be referenced from the frame memory 69 and supplies these images to the motion analysis / compensator 72. In addition, the switch 70 reads the picture used for intra-prediction from the frame memory 69 and supplies the image to the intra-predictor 71.

Information indicating the intra prediction mode, which is obtained by decoding header information and the like, is respectively supplied from the lossless decoder 62 to the intra predictor 71. Based on the information, the intra predictor 71 generates a prediction image from the reference image obtained from the frame memory 69. and supplies the generated prediction image to the switch 73.

Information obtained by decoding the header information (prediction mode information, motion vector information, reference frame information, flag, various parameters, etc.) is supplied from the decoder 62 without loss to the motion analysis / compensator 72.

Based on the information supplied from the lossless decoder 62, the motion analysis / compensator device 72 generates a prediction image from the reference image obtained from the frame memory 69 and supplies the generated prediction image to the switch 73.

The switch 73 selects the prediction image generated by the analysis / compensator 72 or the intra-predictor 71 and supplies the selected prediction image to the arithmetic unit 65.

Operation of the picture decoding device

An example flow of a decoding process performed by the above-mentioned image decoding apparatus 51 will be described with reference to FIG. 4.

When the decoding process is started in step S51, the storage buffer 61 receives and accumulates the transmitted codestream (data). In step S52, the lossless decoder 62 decodes the encoded data supplied from the storage buffer 61. The I picture, P picture and B picture encoded by the lossless encoder 26 shown in FIG. 1 are decoded.

Before decoding the pictures, parameter information such as motion vector information, reference frame information, and prediction mode information (intra prediction mode or inter prediction mode) are also decoded.

When the prediction mode information is intra-prediction mode information, the prediction mode information is supplied to the intra-predictor 71. When the prediction mode information is inter-prediction mode information, prediction mode information and related motion vector information, etc., are supplied to the motion analysis / compensator 72 In addition, the deblocking filter parameter and the adaptive offset parameter are also decoded and supplied to the deblocking filter 31b and the adaptive offset filter 81, respectively.

In step S53, the intra-predictor 71 or the motion analysis / compensator 72 performs a prediction imaging process corresponding to the prediction mode information supplied from the lossless decoder 62.

That is, when information on the intra prediction mode is supplied from the lossless decoder 62, the intra predictor 71 generates an intra prediction image in the intra prediction mode. When the inter-prediction mode information is supplied from the lossless decoder 62, the motion analysis / compensator 72 performs an inter-prediction mode motion analysis / compensation process to generate an inter-prediction image.

Through this process, a prediction image (intra prediction image) generated by the intra predictor 71 or a prediction image (inter prediction image) generated by the motion analysis / compensator 72 is supplied to the switch 73.

In step S54, the switch 73 turns on the prediction image. That is, a prediction image generated by an intra-predictor 71 or a prediction image generated by an analysis / compensator 72 is supplied. Accordingly, the supplied prediction image is selected and supplied to the arithmetic operation unit 65, and added to the output of the inverse orthogonal transformer 64 in step S57, which will be described later.

In step S52, described above, the transform coefficient decoded by the lossless decoder 62 is also supplied to the inverse quantizer 63. In step S55, the inverse quantizer 63 inversely quantizes the transform coefficient decoded by the lossless decoder 62 with characteristics corresponding to those of the quantizer 25 shown in fig. 1.

In step S56, the inverse orthogonal transformer 29 performs inverse orthogonal transformation on a transform coefficient that has been inversely quantized by an inverse quantizer 28 with characteristics corresponding to those of the orthogonal transformer 24 shown in FIG. 1. As a result, the difference information corresponding to the input signal of the orthogonal converter 24 (the output signal of the arithmetic operation unit 23) shown in FIG. 1.

In step S57, the arithmetic operation unit 65 adds the prediction image selected in the process in step S54 described above, and inputs it through the switch 73 into the difference information. As a result, the original image is decoded.

In step S58, the deblocking filter 31b performs deblocking filtering on the image output from the arithmetic operation unit 65. Next, referring to FIG. 23, the details of the deblocking filtering process will be described. In this case, the parameters β and Tc taken from β and Tc determined in the HEVC method are used as detection thresholds with respect to the deblocking filter. The filtered image from the deblocking filter 31b is output to an adaptive displacement filter 81. In addition, the deblocking filtering process also uses on / off information, which is a deblocking filter parameter supplied from lossless decoder 62, and each offset of the β and Tc parameters.

In step S59, the adaptive offset filter 81 performs adaptive offset filtering. Through this process, filtering is performed on the images after filtering with the deblocking filter 31b using an offset value for each quad tree structure, in which the type of offset filter is determined for each subdivided area as well as in each subdivided area. The displaced image is fed to the adaptive contour filter 82.

In step S60, the adaptive offset filter 82 performs adaptive contour filtering of the image after being filtered by the adaptive offset filter 81. The adaptive loop filter 82 filters the input image in the processing unit using the filter coefficient calculated for each processing unit and supplies the result of the filtering to the frame rearrangement buffer 67 and frame memory 69.

In step S61, the frame memory 69 stores the filtered image.

In step S62, the frame rearranging buffer 67 rearranges the image after the adaptive loop filter 82. That is, the frames rearranged for encoding by the frame rearranging buffer 22 of the image encoding apparatus 11 are rearranged in the original display order.

In step S63, the D / A converter 68 performs D / A conversion on the image obtained from the frame rearrangement buffer 67. This image is displayed on a display (not shown) and the image is thus displayed.

The decoding process ends when the processing in step S63 ends.

2. Explanation of the traditional way

Deblocking filter parameters in AVC mode

Next, the parameters of the deblocking filter for the AVC method will be described. In the AVC method, the parameters for determining whether or not a deblocking filter is required are α and β. These values are determined according to the quantization parameter QP.

FIG. 5, position A is a graph showing a relationship between parameter α and quantization parameter QP, and position B in FIG. 5 is a table showing a relationship between a parameter α and a quantization parameter QP. Parameter α is optimized by fine adjustment based on expression (1).

Mathematical Formula 1

……(1)

……(1)

The parameter α can take values from 0 to 255 for the quantization parameter QP from 0 to 51.

FIG. 6, A is a graph showing a relationship between the β parameter and the quantization parameter QP, and B in FIG. 6 is a table showing the relationship between the β parameter and the QP quantization parameter. The β parameter is optimized by fine adjustment based on expression (2). The dashed line at position A in FIG. 6 shows expression (2), and the solid line shows the value given at position B in FIG. 6 after fine adjustment according to expression (2).

Mathematical Formula 2

β (qp) = round (0.5 * qp - 7) ... (2)

The β parameter can take values from 0 to 18 for the QP quantization parameter from 0 to 51.

Deblocking filter parameters for HEVC method

As parameters corresponding to α and β for the AVC method, the parameters β and Tc are used in the HEVC method.

Parameters β and Tc are filter parameters used in the definition with respect to the deblocking filter. Specifically, the parameter β is a threshold value used in determining whether a deblocking filter is required or not, and determining the selection of the filter strength and a parameter for adjusting the filter strength of the deblocking filter. The Tc parameter is a threshold value used in determining the filter strength selection and a parameter for adjusting the filter strength of the deblocking filter. In addition, the Tc parameter is a value that is also used for limiting during filtering. The parameters β and Tc are values set in accordance with the quantization parameter QP as shown in FIG. 7. In FIG. 7 shows the values of the parameters β and Tc corresponding to the quantization parameter QP.

Parameter β has 52 values from 0 to 51 and is fixed at 64 for entries greater than 51. Parameter Tc has 54 values and is fixed at 13 for entries greater than 53.

Here, the range of the quantization parameter QP that can be used in the HEVC method is 0 to 51, and the offset range of the deblocking filter parameter is –26 to 26. Therefore, in theory, the LUT (Look Up Table) input range for the deblocking filter ranges from 0 to 77.

However, since the quantization parameter QP, which is the actual input of the deblocking filter parameter in the LUT, is 0-51, the maximum offset value is not used.

In addition, as described above, even if 64, which is the maximum value of the β parameter, and the value 13, which is the maximum value of the Tc parameter in the HEVC method, are used, there have been cases in which block noise that is visible remains in some sequences. ...

Therefore, in the present method, in order to increase the filtering strength of the deblocking filter, the parameters β and Tc are expanded and the extended parameters β and Tc are used in the deblocking filter. That is, the deblocking filter uses the parameters obtained by expanding the range of filter parameters (parameters β and Tc) to control the filter in order to increase the filtering strength.

In addition, the expansion herein refers to expanding a parameter range for an existing parameter range with an AVC parameter range and a version 21 parameter range of the HEVC method described in Non-Patent Document 1 as the existing parameter range. In addition, in the present method, the parameter range is expanded so as to increase the filter strength relative to the noise, since there is a case where the noise remains with the existing parameters.

3. First embodiment (first extension method)

Extension of the parameter β

FIG. 8 shows the relationship between the quantization parameter QP and the β parameter in the present method. In addition, in the table shown in FIG. 8, the thick-framed area is the area expanded by the present method.

As shown within the open frame in FIG. 8, the parameter β (β (52) - β (qpmax)) corresponding to the quantization parameter QP (52 - qpmax) is extended (added). In addition, the following explanation will be given as an example for qpmax equal to 77. However, the value can be set and is not limited to 77.

The value of the β parameter to be extended can be set as a function of the quantization parameter, and there are two ways to set the β parameter. The first way to set β is a way to set the value of the parameter β as a linear function (linear form) of the quantization parameter QP. The second method of setting β is a method of setting the value of the parameter β as an exponential function (curve) of the quantization parameter QP.

With an exponential function, you can increase the change more than with a linear function.

The first way to set β: an example of a linear function

In the HEVC method, the parameter β is expressed as a linear function of the quantization parameter QP, as shown for 0 ≤ qp ≤ 51 in the following expression (3).

Therefore, as the first setting method as well as for the extended section, the parameter β is set as a linear function of the quantization parameter QP, as shown for 52 β qp ≤ pqmax in expression (3).

Mathematical Formula 3

Here k and b can be adjusted. The slope angle k is greater than 0 and b can be calculated making the boundary between the existing portion and the widened portion continuous.

In addition, also for the expanded portion, when the expansion is performed as a straight line in the HEVC method, as shown in the following expression (4), the inclination angle is k = 2 and b = -38.

Mathematical Formula 4

Position A in FIG. 9 is a graph of the relationship between the parameter β and the quantization parameter QP when the spreading is performed as a linear function for k = 2. Position B in FIG. 9 is a table of values of the parameter β and the quantization parameter QP when the spreading is performed as a linear function for k = 2. In addition, in the table shown in FIG. 9, the thick-framed portion is the portion expanded by the present method. In this case, the parameter β can take values from 0 to 116 for the quantization parameter QP from 0 to 77.

In addition, the slope of the linear function can be changed. For example, for the extended portion, the linear function when the expansion is performed making the inclination angle larger than that of the straight line in the HEVC method, that is, in the case of the inclination angle k = 4 and b = -140, is expressed by the following expression (5).

Mathematical Formula 5

Here, the reason for the slope angle k = 4 is because qp * 2 = qp << 1 (shifted 1 bit to the left) in the case of the slope angle k = 2.On the other hand, in the case of the slope angle k = 4, qp * 2 = qp << 2 (shifted 2 bits to the left).

Therefore, the calculation by means of the considered software is suitable for the moment of installation.

In addition, in the case of the inclination angle k = 4, b = -140 is set to make the boundary between the existing portion and the extended portion a continuous function.

Position A in FIG. 10 is a graph showing a relationship between a parameter β and a quantization parameter QP when spreading is performed by making the slope angle larger than that of a straight line in the HEV method, that is, the spreading is performed as a linear function for the slope angle k = 4. Position B in FIG. ... 10 is a table of values of the β parameter and the quantization parameter QP when the spreading is performed as a linear function for k = 4. In addition, in the table given at B in FIG. 10, the thick-framed area is the area expanded by the present method. In this case, the β parameter can take values from 0 to 168 for the QP quantization parameter from 0 to 77.

In addition, for the extended portion, the linear function when the extension is performed making the inclination angle larger than that of the straight line in the HEVC method, that is, in the case of the inclination angle k = 8 and b = -344, is expressed by the following expression (6).

Mathematical Formula 6

Here, the reason that the slope angle k = 8 is because qp * 8 = qp << 3 (shifted 3 bits to the left) in the case of the slope angle k = 8. Therefore, the calculation is done with the software used, which is suitable for the moment of installation. In addition, in the case of the inclination angle k = 8, b = -344 is set to make the boundary between the existing portion and the extended portion into a continuous function.

In addition, in the above explanation, examples of the inclination angle k = 2, 4 and 8 have been described as the most suitable examples. However, as long as k> 0, the present invention is not limited to the inclination angle k = 2, 4 and 8.

Position A in FIG. 11 is a graph showing the relationship between the β parameter and the quantization parameter QP when spreading is performed by making the slope angle larger than that of a straight line in the HEV method, that is, the spreading is performed as a linear function for the slope angle k = 8. Position B in FIG. ... 11 is a table of values of the parameter β and the quantization parameter QP when the spreading is performed as a linear function for k = 4. In addition, in the table shown at position B in FIG. 11, the thick-framed portion is the portion expanded by the present method. In this case, the parameter β can take values from 0 to 255 for the quantization parameter QP from 0 to 75. In addition, for example, when the quantization parameter QP is 76, 77, etc. and β is greater than 255, β = 255 is set.

The second way to set β: an example of an exponential function

Further, as a second way of setting β for the extended section, the parameter β is set as an exponential function of the quantization parameter QP, as shown for 52 ≤ qp ≤ pq _max in the following expression (7).

Mathematical Formula 7

Here, the exponential function of expression (7) is set based on the value of the parameter α in the deblocking filter of the AVC method of expression (1) described above.

For example, in the case k = 0.8, x = 6, a = -13 and b = 0 shown in the following expression (8), the coefficient (k, x, a, b) is calculated using a value that is not in HEVC, from the number of parameters α in the deblocking filter for the AVC method. Specifically, when the QP is 38, the α parameter for AVC is 63. This is close to 64, which is the β parameter value for the HEVC method when the QP is 51. Therefore, for the β QP parameter equal to 52 or higher, to apply the α parameter for of the AVC method for a QP of 39 or higher, the coefficient (k, x, a, b) is calculated such that the boundary between the existing portion and the extended portion is continuous.

Math Formula 8

Position A in FIG. 12 is a graph of the relationship between the parameter β and the quantization parameter QP when the expansion is performed as an exponential function for k = 0.8, x = 6, a = -13, and b = 0. Position B in FIG. 12 is a table of the values of the β parameter and the quantization parameter QP when the expansion is performed as an exponential function for k = 0.8, x = 6, a = -13, and b = 0. In addition, the table given at B in FIG. 12, the thick-framed portion is the portion expanded by the present method. In this case, the parameter β can take values from 0 to 255 for the quantization parameter QP from 0 to 63. Also, for example, when the quantization parameter QP is 64-77, etc. and β is greater than 255, β = 255 is set.

Since the parameter β is extended as described above, the filtering strength of the deblocking filter can be increased.

Tc installation method

The Tc parameter for the HEVC method can be applied as it is. However, as shown in the approximate expression given in the following expression (9), the Tc parameter can also be extended as a function in the HEVC method.

Math Formula 9

Tc (qp) = max (round (0.5 * qp - 13.5), 1),

qp ∈ [53, Tcqp _max ] ... (9)

In addition, in practice, like β in the AVC method, the Tc parameter is optimized by fine adjustment based on expression (9) described above.

Position A in FIG. 13 is a graph of the relationship between the Tc parameter and the QP quantization parameter when spreading is performed using the HEVC function. Position B in FIG. 13 is a table of values of the Tc parameter and the quantization parameter QP when spreading is performed using a function in the HEVC method. In addition, in the table shown at B in FIG. 13, the thick-framed portion is the portion expanded by the present method. In this case, the Tc parameter can take values from 1 to 25 for the QP quantization parameter from 0 to 77.

Since the Tc parameter is extended as described above, it is possible to further increase the filtering strength of the deblocking filter.

Deblocking filter configuration example

Next, details of the deblocking filter 31a of the image encoding apparatus shown in FIG. 1 and a deblocking filter 31b of the image decoding apparatus shown in FIG. 3. In addition, since the deblocking filter 31a of the image encoding apparatus shown in FIG. 1, and the deblocking filter 31b of the image decoding apparatus shown in FIG. 3 have basically the same configuration and operation, the following explanation will be given at the same time for both filters represented as deblocking filter 31.

Here, only the following is the difference between the deblocking filters 31a and 31b. That is, in the case of the deblocking filter 31a, on / off information as to whether or not to use the deblocking filter and each offset of the β and Tc parameters are input through an operation unit (not shown). On the other hand, in the case of the deblocking filter 31b, on / off information as to whether or not to use the deblocking filter and each parameter offset β and Tc that have been encoded by the image encoding apparatus 11 are received and input after being decoded by the lossless decoder 62.

FIG. 14 is a block diagram of a configuration example of a deblocking filter unit.

In the example shown in FIG. 14, the deblocking filter 31 is configured to insert the configuration setting unit 101 and the filter execution unit 102.

The configuration setting unit 101 receives, for example, on / off information, an offset value β and an offset value Tc, which are input from an operation unit (or from a lossless decoder 62) not shown in the diagram, and performs a configuration setting in the deblocking filter 31. B As a configuration setting, the deblocking filter is shifted, the deblocking filter is turned on / off, the boundary deviation between TU and PU, BS value deviation (Boundary Filtering Strength), etc. The configuration setting unit 101 supplies setting information to the execution unit 102 filter.

The pixel value before passing the deblocking filter from the arithmetic operations unit 30 (or the arithmetic operations unit 65) is supplied to the filter execution filter 102. The filter execution unit 102 calculates the quantization parameter QP used to generate the parameters based on the information set by the configuration setting unit 101, generates the β and Tc parameters, and determines whether filtering and filtering strength are required. Then, the filter execution unit 102 performs filtering based on the determined filtering strength or on a determination result as to whether filtering is required. The filter execution unit 102 supplies the pixel value after the deblocking filter to the next adaptive offset filter 41 (or adaptive offset filter 81).

In addition, the filter execution unit 102 performs the above-described processing on both the luminance border and the color contrast border.

An example of the configuration of the filter execution unit at the brightness border

FIG. 15 is a block diagram of a configuration example of a filter execution unit performing luminance edge filtering.

In the example shown in FIG. 15, the filter execution unit 102 is configured to contain a QP calculating unit 111, a β generating unit 112, a Tc generating unit 113, a filtering determination unit 114, and a filtering unit 115.

The pixel value before the deblocking filter from the front end is input to the QP acquiring unit 111a and filtering unit 115.

The QP calculating unit 111 is configured to contain the QP obtaining unit 111a and the QP average calculating unit 111b, and receives a pixel value before the deblocking filter from the input stage and calculates a quantization parameter QP, which is used to generate the β and Tc parameters. The QP calculating section 111 supplies the calculated quantization parameter QP to the β generating section 112 and the Tc generating section 113.

The QP acquiring unit 111a obtains the quantization parameters p_QP and q_QP for two regions (near the border) sharing the border to be processed from the input pixel value before the deblocking filter. For example, when the border extends in the vertical direction, the quantization parameter p_QP of the region p adjacent to the left side of the border is obtained, and the quantization parameter q_QP of the region q adjacent to the right side of the border is obtained. For example, when the boundary extends in the vertical direction, the quantization parameter p_QP of the region p adjacent to the left side of the boundary is obtained, and the quantization parameter q_QP of the region q adjacent to the right side of the boundary is obtained.

In addition, for example, in the image coding apparatus 11, the quantization parameter QP obtained from the quantizer 25 is stored in a memory (not shown) along with motion vector information or prediction mode information. In addition, in the image coding apparatus 51, the QP quantization parameter decoded by the lossless decoder 62 is stored in a memory (not shown) along with motion vector information or prediction mode information. The QP obtaining unit 111a obtains the quantization parameter QP from the memory storing it in this way.

The QP obtaining section 111a supplies the obtained quantization parameters p_QP and q_QP to the QP average value calculating section 111b.

The QP average calculating unit 111b calculates the average value of the quantization parameters p_QP and q_QP from the QP acquiring unit 111a and supplies the average value to the β generating unit 112 and the Tc generating unit 113 as a QP quantization parameter that is used to generate β and Tc parameters.

The β generating unit 112 generates a parameter β corresponding to the QP quantization parameter obtained from the QP average calculating unit 111b, and supplies the generated β parameter to the filter determination unit 114.

The Tc generating unit 113 generates a Tc parameter corresponding to the QP quantization parameter obtained from the QP average calculating unit 111b, and supplies the generated Tc parameter to the filtering determination unit 114.

The filter determination unit 114 determines the filtering using the β parameter obtained from the β generation unit 112, the Tc parameter obtained from the Tc generation unit 113, etc. That is, the filter determination unit 114 determines whether filtering is required using the β parameter obtained from the β generating unit 112, etc. The filtering determination unit 114 determines the filtering strength using the β parameter obtained from the β generating unit 112, the Tc parameter obtained from the Tc generating unit 113, etc. The filtering determination unit 114 supplies specific filtering information to filtering unit 115. In this case, the parameters β and Tc are also supplied to the filter unit 115. For example, the Tc parameter is used for limiting in filter block 115.

When the filter determination unit 114 determines that filtering is to be performed, the filter unit 115 performs filtering on the input pixel value before the deblocking filter with the filter strength determined by the filter determination unit 114. The filter unit 115 outputs the pixel value after the deblocking filter to a subsequent stage.

In addition, when the filter determination unit 114 determines that filtering should not be performed, the filter unit 115 outputs the input pixel value before the deblocking filter as a pixel value after the deblocking filter to the subsequent stage as it is (without performing filtering).

An example of the configuration of the formation block β

FIG. 16 is a block diagram of a configuration example of the β forming unit.

In the example shown in FIG. 16, the β generating unit 112 is configured to contain the β LUT_input calculating unit 121, the limiting unit 122, the existing β generating unit 123, and the extended β generating unit 124.

The QP quantization parameter obtained from the QP average calculator 111b is supplied to the β LUT_input calculator 121. The offset value β for the information set by the configuration setting unit 101 is supplied to the β LUT_input calculating unit 121. In addition, the value βqp _max , which is the maximum value of the quantization parameter used to generate β, for the information set by the configuration setting unit 101, is supplied to the limiting unit 122.

The β LUT_input calculating unit 121 and the limiting unit 122 calculate β LUT_input, which is a value input to the existing β generating unit 123 and the extended β generating unit 124.

That is, the β LUT_input calculator 121 calculates βtmp by adding the offset value β and the quantization parameter QP obtained from the QP average calculating unit 111b, and supplies the calculated βtmp value to the limiting unit 122.

The limiting unit 122 limits the βtmp value obtained by the β LUT_input calculator 121 to the range [0, βqp _max ] with respect to the value set by the configuration setting unit 101. The limiting unit 122 supplies the value of βtmp after limiting to the existing β generating unit 123 and to the extended β generating unit 124 as β LUT_input qp.

The existing β generating unit 123 has a Look Up Table (LUT) 131 defined in the HEVC method. When the β value of the LUT_input qp from the limiting block 122 is equal to or less than 51, the existing β generation unit 123 calculates β using the LUT 131 and supplies the calculated β value to the filter determination unit 114.

When the β LUT_input qp value obtained from the limiting block 122 is greater than 51, the extended β generating unit 124 calculates the extended β and supplies the calculated β value to the filter determination unit 114.

For example, as shown in FIG. 17, the extended β generating unit 124 is configured to contain the β (qp) calculating unit 141. The β (qp) calculator 141 generates β by dynamically using a function expression (linear function or exponential function) shown by a dotted line, where the function parameters are set in advance when qp is entered for [51, βqp _max ].

As the first β setting method, the extended β generating unit 124 generates β dynamically using a linear function in the case of 52 ≤ qp ≤ pq _max in the above expression (3). That is, in this case, β is calculated as a linear function of QP. In addition, k and b are set as parameters to the function. In this case, k and b can be transmitted to the decoding side by means of syntax.

As a second setting method for β, the extended β generating unit 124 generates β by dynamically using an exponential function in the case of 52 ≤ qp ≤ pq _max in the above expression (7). That is, in this case, the value of β is calculated as an exponential function of QP. In addition, k, b, x and a are set as parameters to the function. In this case, k, b, x and a can be transmitted to the decoding side by means of syntax.

For example, as shown in FIG. 18, the extended β generating unit 124 is configured to contain an extended Look Up Table (LUT) 153. Also, in the example shown in FIG. 18, the expanded LUT 153 is shown to contain an existing portion of the LUT 131. However, the expanded portion is only the shaded portion. The extended LUT 153 is saved after β (qp) calculator 152 provided in another computer 151 or the like takes an arbitrary qp for [0, βqp _max ] and calculates β in advance using an expression for a function (linear function or exponential function), in which the function parameters are set.

The extended β generator 124 calculates β using the extended LUT 153 when qp is input for [51, βqp _max ].

As described above, in the configuration of the β generating unit 124, it is possible to select whether to set the extended β by a determination expression (function expression) or an LUT.

Example of the configuration of the Tc formation unit

FIG. 19 is a block diagram of a configuration example of a Tc generating unit.

In the example shown in FIG. 19, the Tc generating unit 113 is configured to contain the Tc_LUT_input calculating unit 161, the limiting unit 162, the existing Tc generating unit 163, and the extended Tc generating unit 164.

The QP quantization parameter obtained from the QP average computation unit 111b is supplied to the Tc_LUT_input computation unit 161. The offset value Tc for the information set by the configuration setting unit 101 is supplied to the Tc_LUT_input calculating unit 161. In addition, the value of Tc_qp _max , which is the maximum value of the quantization parameter used for generating the Tc, for the information set by the configuration setting unit 101, is supplied to the limiting unit 162.

The Tc_LUT_input calculator 161 and the limiting block 162 calculate the Tc_LUT_input, which is a value input to the existing Tc generating section 163 and the extended Tc generating section 164.

That is, the Tc_LUT_input calculator 161 calculates Tctmp by adding the Tc offset value and the QP quantization parameter obtained from the QP average calculating unit 111b, and supplies the calculated Tctmp value to the limiting block 162.

The limiting block 162 limits the Tctmp value obtained from the Tc_LUT_input computation block 161 in the range [0, Tc_qp _max ] relative to the value set by the configuration setting block 101. The limiting unit 162 supplies the Tctmp value after limiting to the existing Tc generating unit 163 and to the extended Tc generating unit 164 as Tc_LUT_input qp.

The existing Tc generating unit 163 has a Look Up Table (LUT) 171 defined in the HEVC method. When the Tc_LUT_input qp value from the limiting block 162 is equal to or less than 53, the existing Tc generating unit 163 calculates Tc using the LUT 171, and supplies the calculated Tc value to the filtering determination unit 114.

When the Tc_LUT_input qp value obtained from the limiting block 122 is greater than 53, the extended Tc generating unit 164 calculates the extended Tc and supplies the calculated Tc value to the filtering determination unit 114.

For example, as shown in FIG. 20, the extended Tc generating unit 164 is configured to contain the Tc (qp) calculating unit 181. The Tc (qp) calculator 181 generates the Tc dynamically using the above-described expression (9), which is obtained by extending the existing QP-Tc function when qp is input for [53, Tc_qp _max ].

For example, as shown in FIG. 21, the extended Tc generating unit 164 is configured to contain an extended Look Up Table (LUT) 193. Also, in the example shown in FIG. 21, the expanded LUT 193 is shown to contain an existing portion of LUT 171. However, the expanded portion is only the shaded portion. The extended LUT 193 is stored after a Tc (qp) computation unit 192 provided in another computer 191 or the like takes an arbitrary qp value for [53, Tc_qp _max ] and calculates Tc using the extended function.

The extended Tc generating unit 164 calculates the Tc using the extended LUT 193 when qp is input for [53, Tc_qp _max ].

As described above, in the configuration of the extended Tc generating unit 164, it is possible to select whether the extended Tc is set by a determination expression (function expression) or by a LUT.

An example of the configuration of the filter execution unit at the border of color contrast

FIG. a block diagram of an example of a configuration of a filter execution unit performing filtering of a color contrast border is presented.

The filter execution unit 102 shown in FIG. 22 differs from the filter execution unit 102 shown in FIG. 15 only in that the β generating unit 112 and the filter determination unit 114 are removed. The filter execution unit 102 shown in FIG. 22 is the same as the filter execution unit 102 shown in FIG. 15, which contains a QP calculating unit 111, a Tc generating unit 113, and a filtering determination unit 115. Since the common blocks are repeated, no explanation is provided for them.

That is, in the case of the border of the color contrast, no filtration determination is performed, and the filtering unit 115 performs filtering using the Tc parameter obtained from the Tc generating unit 113. For example, the Tc parameter is used for limiting.

Deblocking filter operation procedure

Next, with reference to the flowchart shown in FIG. 23, the deblocking filtering process will be described. In addition, this deblocking filtering process is performed in step S22 in FIG. 2 and in step S58 in FIG. 4.

For example, the on / off information, the β offset value, and the Tc offset value are input through an operation unit (or lossless decoder 62) not shown in the diagram.

In step S101, the configuration setting unit 101 sets the filter bias (β bias and Tc bias). The configuration setting unit 101 supplies information about the set offsets to each of the β forming units 112 and the Tc generating unit 113 of the filter execution unit 102.

In step S102, the configuration setting unit 101 determines whether the deblocking filter can be used based on the on / off information. When it is determined in step S102 that the deblocking filter cannot be used, the deblocking filtering process ends.

When it is determined in step S102 that the deblocking filter can be used, the deblocking filtering process proceeds to step S103.

In step S103, the configuration setting unit 101 determines the boundary between the TU and the PU. In step S104, the configuration setting unit 101 determines the BS (Boundary Filtering Strength) value based on the boundary information between the TU and PU determined in step S103, prediction mode information, etc. The configuration setting unit 101 supplies information about the BS value to the QP calculator 111 of the filter execution unit 102.

In step S105, the filter execution unit 102 performs luminance edge filtering (LUMA). Although this process will be described later with reference to FIG. 24, filtering is performed at the luminance edge by the process in S105.

In step S106, the filter execution unit 102 performs color contrast edge filtering (CHROMA). Although this process will be described later with reference to FIG. 29, filtering is performed on the border of the color contrast by the process in step S106.

In step S107, the filter execution unit 102 determines whether all boundaries have been processed. When it has been determined in step S107 that not all of the boundaries have been processed, the process returns to step S105 to repeat the subsequent processes.

When it is determined in step S107 that all the boundaries have been processed, the process goes to step S108.

In step S108, the configuration setting unit 101 determines whether all CUs have been processed. When it has been determined at step S108 that not all CUs have been processed, the process returns to step S103 to repeat the subsequent processes.

When it is determined in step S108 that all CUs have been processed, the deblocking filtering process ends.

An example of filtering at the edge of brightness

Next, with reference to the flowchart shown in FIG. 24, the luma edge filtering process shown in FIG. 23.

When the Bs value is received from the configuration setting unit 101, the QP calculating unit 111 determines in step S121 whether the Bs value is greater than 0. When it is determined in step S121 that the Bs value is not more than 0, the luminance edge filtering ends. That is, in this case, the filtering at the luminance boundary is not performed.

When it is determined in step S121 that the value of Bs is not more than 0, the luminance edge filtering ends. In step S122, the QP calculator 111 receives the pixel value before the deblocking filter from the previous stage and calculates the average QP value of the two blocks (areas) sharing the border.

That is, the QP obtaining unit 111a obtains the quantization parameters p_QP and q_QP for two regions (adjacent to the border) that share the border to be processed from the input pixel value before the deblocking filter. The QP obtaining section 111a supplies the obtained quantization parameters p_QP and q_QP to the QP average value calculating section 111b.

The QP average calculating unit 111b calculates the average value of the quantization parameters p_QP and q_QP from the QP acquiring unit 111a and supplies the average value to the β generating unit 112 and the Tc generating unit 113 as a QP quantization parameter that is used to generate the β and Tc parameters.

In step S123, the β generating unit 112 generates β. Although the β forming process will be described later with reference to FIG. 25, β is generated by the process in step S123 and supplied to the filter determination unit 114.

In step S124, the Tc generating unit 113 generates a Tc. Although the Tc generation process will be described later with reference to FIG. 26, Tc is generated by the process in step S124 and supplied to the filter determination unit 114.

In step S125, the filter determination unit 114 determines the filtering. That is, the filter determination unit 114 determines whether filtering is required using the parameter β obtained from the β generation unit 112, etc. The filter determination unit 114 determines the filtering strength using the β parameter obtained from the β generation unit 112, the Tc parameter received from the Tc generating unit 113, and so on. The filtering determination unit 114 provides certain filtering information to the filtering unit 115.

In step S126, the filtering unit 115 performs filtering on the input pixel value before the deblocking filter with the filtering strength determined by the filtering determination unit 114. The filter unit 115 outputs the pixel value after the deblocking filter to a subsequent stage.

In addition, in step S125, when it is determined that filtering is not performed, the process in step S126 is skipped. In this case, the filtering unit 115 outputs the input pixel value before the deblocking filter as the pixel value after the deblocking filter to the subsequent stage as it is (without performing filtering).

An example of the β formation process

Next, with reference to the flowchart shown in FIG. 25, the β generation process in S123 shown in FIG. 24.

The QP quantization parameter obtained from the QP average calculator 111b is supplied to the β LUT_input calculator 121. The offset value β for the information set by the configuration setting unit 101 is supplied to the β LUT_input calculating unit 121.

In step S141, the β LUT_input calculator 121 calculates βtmp by adding the offset value β and the quantization parameter QP obtained from the QP average calculating unit 111b, and supplies the calculated βtmp value to the limiting unit 122.

In step S142, the limiting unit 122 limits the βtmp value obtained by the β LUT_input calculating unit 121 to the range [0, βqp _max ] with respect to the value set by the configuration setting unit 101. Although the details of the process will be described later with reference to FIG. 26, the limiting unit 122 supplies the βtmp value after limiting to the existing β generating unit 123 and to the extended β generating unit 124 as β LUT_input qp.

In step S143, the existing β generating unit 123 and the extended β generating unit 124 determine whether the qp value obtained from the limiting unit 122 is greater than 51. When it is determined in step S143 that the qp value obtained from the limiting unit 122 is equal to or less 51, the process goes to S144.

In step S144, the existing β generation unit 123 calculates β using the existing LUT 131, and supplies the calculated β to the filter determination unit 114.

When it is determined in step S143 that the qp value obtained from the restriction block 122 is greater than 51, the process proceeds to step S145.

In step S145, the extended β generating unit 124 calculates the extended β and supplies the calculated β to the filter determination unit 114.

For example, the extended β calculating unit 124 (β (qp) calculating unit 141) generates β by dynamically using a function expression (linear function or exponential function) shown by a dotted line, where the function parameters are set in advance as described above with reference to FIG. 17.

For example, the extended β generating unit 124 calculates β using the extended LUT 153, which is stored in advance after receiving an arbitrary qp for [0, βqp _max ] and calculating β using a function expression (linear function or exponential function) in which the function parameters are set, as described above with reference to FIG. 18.

Example limitation

Next, with reference to the flowchart shown in FIG. 26, the limitation in step S142 shown in FIG. 25.

In step S161, the limiting block 122 is set to β LUT_input qp = βtmp obtained from the β LUT_input calculator 121.

In step S162, the limiting unit 122 determines whether the βtmp value obtained from the β LUT_input calculator 121 is greater than βqp _max . When it is determined in step S162 that the value of βtmp is greater than βqp _max , the process proceeds to step S163.

In step S163, the limiting unit 122 sets β LUT_input qp = βqp _max and supplies β LUT_input qp to the existing β generating unit 123 and to the extended β generating unit 124.

When it is determined in step S162 that the value of βtmp is equal to or less than βqp _max , the process in step S163 is skipped and the limitation ends. That is, in this case, β LUT_input qp (βtmp) is supplied to the existing β generating unit 123 and to the extended β generating unit 124.

An example of the process of forming a TC

Next, with reference to the flowchart shown in FIG. 27, the Tc generation process in step S124 shown in FIG. 24.

The QP quantization parameter obtained from the QP average computation unit 111b is supplied to the Tc_LUT_input computation unit 161. The Tc offset value for the information set by the configuration setting unit 101 is supplied to the Tc_LUT_input calculating unit 161.

In step S181, the Tc_LUT_input calculator 161 calculates Tctmp by adding the Tc offset value and the QP quantization parameter obtained from the QP average calculating unit 111b, and supplies the calculated Tctmp value to the limiting block 162.

In step S182, the limiting block 162 limits the Tctmp value obtained from the Tc_LUT_input computation block 161 in the range [0, Tc_qp _max ] with respect to the value set by the configuration setting block 101. Although the details of this process will be described later with reference to FIG. 28, the limiting unit 162 supplies the Tctmp value after limiting to the existing Tc generating unit 163 and to the extended Tc generating unit 164 as Tc_LUT_input qp.

In step S183, the existing Tc generating unit 163 and the extended Tc generating unit 164 determine whether the qp value obtained from the limiting block 162 is greater than 53. When it is determined in step S183 that the qp value obtained from the limiting block 162 is equal to or less 53, the process goes to S184.

In step S184, the existing Tc generating unit 163 calculates the Tc using the existing LUT 171, and supplies the calculated Tc to the filtering determination unit 114.

When it is determined in step S183 that the qp value obtained from the restriction block 162 is greater than 51, the process proceeds to step S185.

In step S185, the extended Tc generating unit 164 calculates the extended Tc and supplies the calculated Tc to the filtering determination unit 114.

For example, the extended Tc calculator 164 (Tc (qp) calculator 181) dynamically generates the Tc using the above-described expression (9), which is obtained by expanding the existing QP-Tc function as described above with reference to FIG. 20.

For example, the extended Tc generating unit 164 calculates Tc using the extended LUT 193 that is stored after receiving an arbitrary qp for [53, Tcqp _max ] by calculating Tc using the extended function as described above with reference to FIG. 21.

Example limitation

Next, with reference to the flowchart shown in FIG. 28, the limitation in step S182 shown in FIG. 27.

In step S201, the limiting block 162 sets Tc_LUT_input qp = Tctmp obtained from the Tc_LUT_input calculator 161.

In step S202, the limiting block 162 determines whether the Tctmp value obtained from the Tc_LUT_input calculator 161 is greater than Tc_qp _max . When it is determined in step S202 that the value of Tctmp is greater than Tc_qp _max , the process proceeds to step S203.

In step S203, the restriction block 162 sets Tc_LUT_input qp = Tc_qp _max and supplies Tc_LUT_input qp to the existing Tc generating unit 163 and to the extended Tc generating unit 164.

When it is determined in step S202 that the value of Tctmp is equal to or less than Tc_qp _max , the process in step S203 is skipped and the limitation ends. That is, in this case, Tc_LUT_input qp (Tctmp) is supplied to the existing Tc generating unit 163 and to the extended Tc generating unit 164.

An example of filtering at the border of color contrast

Next, with reference to the flowchart shown in FIG. 29, in step S106 shown in FIG. 23, a color contrast border filtering process will be described.

When the Bs value is received from the configuration setting unit 101, the QP calculating unit 111 determines in step S221 whether the Bs value is greater than 1. When it is determined in step S221 that the Bs value is not more than 1, the color contrast border filtering ends. That is, in this case, filtering at the border of the color contrast is not performed.

When it is determined in step S221 that the value of Bs is greater than 1, the process proceeds to step S222. In step S222, the QP calculating unit 111 receives the pixel value before the deblocking filter from the previous stage and calculates the average QP value of the two blocks (areas) sharing the border.

That is, the QP obtaining unit 111a obtains the quantization parameters p_QP and q_QP for two regions (adjacent to the border) that share the border to be processed from the input pixel value before the deblocking filter. The QP obtaining section 111a supplies the obtained quantization parameters p_QP and q_QP to the QP average value calculating section 111b.

The QP average calculating unit 111b calculates the average value of the quantization parameters p_QP and q_QP from the QP acquiring unit 111a and supplies the average to the Tc generating unit 113 as a QP quantization parameter that is used to generate the Tc parameter.

In step S223, the Tc generating unit 113 generates a Tc. An explanation of the Tc generation process will not be given since the Tc generation process is essentially the same as the process described above with reference to FIG. 26, Tc is generated by the process in step S223 and supplied to the filter unit 115.

In step S224, the filtering unit 115 performs filtering on the input pixel value before the deblocking filter using the Tc parameter generated by the Tc generating unit 113, etc. The filtering unit 115 outputs the pixel value after the deblocking filter to a subsequent stage.

In addition, in the above explanation, an example of performing expansion by increasing β and Tc in order to increase the filtering strength (hereinafter also referred to as the first expansion method) has been described. However, the examples of the expansion of the β and Tc parameters are not limited to the above-described first expansion method. Next, a second expansion method will be described with changing the slope of the function without increasing the number of parameters.

4. Second embodiment (second extension method)

Example of expanding the parameter β

First, the expansion of the parameter β by the second expansion method will be described. In the HEVC method, as shown by the following expression (10), the parameter β is calculated using three expressions corresponding to the QP section

Mathematical Formula 10

In the second extension method, the tilt angle (k0, k1, k2) of these three expressions is transmitted to the decoding side by means of syntax. On the decoding side, for each slice, the table represented by the following expression (11) is updated using the slope indicated by the syntax on the encoding side.

Mathematical Formula 11

In addition, expression (11) becomes the table of expression (10) described above, in the case of k0 = 0, k1 = 1 and k2 = 2.k0 = 0, k1 = 1 and k2 = 2 are the default values and the value that should change from among these values, is transmitted to the decoding side by means of syntax.

FIG. 30 shows the relationship between the quantization parameter QP and the parameter β in the case of k0 = 0, k1 = 1 and k2 = 2 in expression (11) and the relationship between the quantization parameter QP and the parameter β (new value β) when an update such as k2 = 4.

In the graph shown at position A in FIG. 30, "before update" is shown with a dotted line, and "after update k2 = 4" is shown with a solid line. In the table shown at position B in FIG. 30, the values of the new β after updating, corresponding to 16 ≤ qp ≤ 51 after the update, are surrounded by a thick box. Among them, the values of new β corresponding to 29 ≤ qp ≤ 51, updated for k2 = 4, are shaded.

In the case of the example shown in FIG. 30, the parameter β (new value β) can take values from 0 to 110 for the quantization parameter QP from 0 to 51.

In addition, although in the example described above, β for the time qp = 0 is set to 0, β for the time qp = 0 may have other values without limiting to 0. In addition, in the example above, an example was described, in which β is calculated by means of three expressions corresponding to the QP section. However, the number of divisions of the expression for β is not limited to 3.

In addition, although in the above example, the boundary of each qp section is continuous, the boundary of each qp section may not be continuous. In addition, although in the above example, the expression for β is expressed as a linear function, the expression for β can also be expressed, for example, as an exponential function, but is not limited thereto.

As a method for transmitting the coefficient (k0, k1, k2) to the decoding side, for example, the exponential Golomb method or fixed length coding can be considered.

Example of expanding the parameter Tc

Next, the expansion of the parameter Tc using the second expansion method will be described. Also, in the case of the Tc parameter, the second expansion method is performed mainly in accordance with the same concept as for the β parameter described above.

However, in the case of the Tc parameter, unlike the case of the β parameter, in the HEVC method, the Tc parameter is not expressed in a single expression. In addition, the parameter Tc in the HEVC method when approximated is expressed by the following expression (12).

Mathematical Formula 12

…. (12)

…. (12)

That is, as shown in FIG. 31, there is a small deviation (adjustment amount (Δt)) between the ratio of the actual quantization parameter QP and the parameter Tc and expression (12) obtained by approximating the ratio.

At position A in FIG. 31, expression (12) obtained by approximating the relationship between the quantization parameter QP and the parameter Tc is defined by the dotted line “before adjustment”, and the “after adjustment” deviation between the relationship between the quantization parameter QP and the parameter Tc, expression (12) is determined by a solid line.

At position B in FIG. 31 shows the relationship between the quantization parameter QP and the parameter Tc and the adjustment amount (Δt) when the relationship is determined by expression (12).

Accordingly, in the case of the Tc parameter, as shown in the following expression (13), the approximated expression (12) is multiplied by k3 and k4, and then the amount (adjustment amount Δt) of the deviation from expression (12) is adjusted.

Math Formula 13

…. (13)

…. (13)

Then, in the second spreading method, the inclination angle (k3, k4) of these two expressions is transmitted to the decoding side by means of syntax. On the decoding side, for each slice, the table represented by the above-described expression (13) is updated using the inclination angle specified by the syntax on the encoding side. In addition, as a method for transmitting the coefficient (k3, k4) to the decoding side, for example, the exponential Golomb method or fixed length coding can be considered. However, any kind of encoding is possible.

The adjustment amount is ± k4, and the sign changes in accordance with the qp value, as shown at position B in FIG. 31. At position B in FIG. 31, when QP is 27-29 and 31, the value of the adjustment amount is positive. When is 34 and 36 to 42, the value of the adjustment amount is negative. In addition, the adjustment amount can be fixed as 1 shown at position B in FIG. 31, for example, regardless of k.

In the case of k3 = 0 and k4 = 1, expression (13) becomes an expression obtained by adding an adjustment amount (Δt) to the approximated expression table represented by expression (12) described above. K3 = 0 and k4 = 1 are default values, and a value to be changed from among these values is transmitted to the decoding side by means of syntax.

In addition, although the control amount is used in expression (13), it is also possible to use expression (14) without the control amount under the assumption that the existing Tc parameter is expressed by expression (12) described above.

Math Formula 14

….. (14)

….. (fourteen)

In addition, the multiplication by k0-k2 in the expression (11) described above and the multiplication by k3 and k4 in the expression (13) or the expression (14) described above can be preferably implemented by bit shift.

In addition, according to the second extension method described above, the meaning of the quantization parameter QP does not change. The range of the quantization parameter QP is the same as that of the HEVC method. In addition, it is possible to increase the degree of installation freedom depending on how much noise is mixed into the flow. Moreover, it can be implemented without increasing the used memory.

5. Third embodiment (third extension method)

Example of expanding the parameter β

Additionally, the expansion of the parameter β using the third expansion method will be described. The meaning of the offset (β_offset) of the β parameter in the HEVC method is the quantization parameter QP + β_offset. Accordingly, the parameter β is determined by the following expression (15)

Math Formula 15

β = β [QP + β_offset] ... (15)

That is, in the HEVC method, β_offset has been added to the QP parameter and the β value is obtained. In contrast, in the third extension method, as shown in the following expression (16), β_offset is directly added to the parameter β

Mathematical Formula 16

β = β [QP] + β_offset * α0 ... ((16)

Here, α0 is a fixed value. For example, α0 = 0, 1, 2, 3, 4, ..., and the value of α0 is not limited. It is possible to adjust β using this α0 value. In addition, α0 may not be a fixed value. For example, α0 can be set on the coding side and transmitted to the decoding side by syntax.

For example, as described with reference to FIG. 32, in the case of QP = 37, β_offset = 6 and α0 = 2, the value (= 48) for β = β (37 + 6) = β (43), which is used from expression (15) above for the HEVC method.

On the other hand, in the case of the third expansion method, the value (48) obtained by adding 12 to the value (= 36) for β = β (37) + 6 * 2 = β (37) is used from expression (16) described above.

As described above, it is also possible to directly expand the range of the β value using β_offset. This is done in the same way for the Tc parameter, as described below.

Example of expanding the parameter Tc

Next, the expansion of the parameter Tc using the third expansion method will be described. The meaning of the offset (Tc_offset) of the Tc parameter for the HEVC method is the quantization parameter QP + Tc_offset. Accordingly, the parameter Tc is determined by the following expression (17).

Mathematical Formula 17

tc = tc [QP + tc_offset] ... (17)

That is, in the HEVC method, Tc_offset has been added to the QP quantization parameter and the Tc value is obtained. In contrast, as a third extension method, as shown in the following expression (18), Tc_offset is directly added to the Tc parameter.

Mathematical Formula 18

tc = tc [QP] + tc_offset * α1 .. (18)

For example, α1 = 0, 1, 2, 3, 4, ..., and the value of α1 is not limited. Using this α1 value, Tc can be adjusted. In addition, α1 is not limited to a fixed value, and can be set on the coding side and transmitted to the decoding side by syntax.

For example, as described with reference to FIG. 33, in the case of QP = 37, Tc_offset = 6 and α1 = 1, the value (= 8) for Tc = Tc (37 + 6) = Tc (43) is used from the above expression (17) for the HEVC method.

On the other hand, in the case of the third expansion method, the value (10) obtained by adding 6 to the value (= 4) for Tc = Tc (37) + 6 * 1 = Tc (37) is used according to expression (18) described above.

As described above, it is also possible to directly expand the range of the Tc value using Tc_offset.

In addition, the multiplication by α0 in the expression (15) described above and the multiplication by α1 in the expression (18) described above can be preferably implemented by bit shift.

In addition, according to the third extension method described above, the meaning of the quantization parameter QP does not change. The range of the quantization parameter QP does not differ from the range for the HEVC method. Moreover, it can be implemented without increasing the used memory.

In addition, similar to the first expansion method, the second and third expansion methods described above are also performed by the expanded β generating unit 124 such as shown in FIG. 16 and the extended Tc generating unit 164 shown in FIG. nineteen.

In addition, the coefficients (k0-k5) in the second spreading method described above and the coefficients α0 and α1 in the third spreading method may be transmitted to the decoding side as a header such as, for example, a slice header. In addition, coefficients (k0-k5) or coefficients α0 and α1 may be transmitted to the decoding side as a NAL unit such as an adaptation parameter set (APS).

As described above, the parameters used in the determination in relation to the filter, that is, the parameters for determining the filtering strength, are expanded so as to increase the filtering strength. Therefore, since block noise can be suppressed by increasing the filter strength, it is possible to optimize the decoded image.

In addition, β and Tc were used as filter parameters to increase the filtering strength of the deblocking filter. In addition, in the range of parameters corresponding to the current conditions, there was a place where the noise disappeared. Therefore, the expansion of β and Tc was performed to further increase the filtration strength. Similar considerations can be applied to an adaptive bias filter. That is, also for the adaptive displacement filter, an additional increase in the filtering strength is possible, expanding the range of parameters of the current standard.

Here, in the first spreading method described above, an example of expanding the range of parameters in values after the quantization parameter QP (0-51), which is an input of the deblocking filter parameter in the LUT, was given. In this case, the number of tables to maintain the parameter increases dramatically or the cost of calculating new parameters increases.

In contrast, an example will be described in which the strength of the unblocking filtering can be increased much more easily.

6. Fourth embodiment (fourth extension method)

Extension of the parameter β

First, referring to FIG. 34, an example of expanding a parameter range within a quantization parameter QP (0-51) will be described.

Position A in FIG. 34 is a graph showing a relationship between a QP quantization parameter and a β parameter in the present method. Position B in FIG. 34 is a table showing the relationship between the quantization parameter QP and the β parameter in the present method. In addition, at position A in FIG. The 34 dotted line shows the prior art. In the table shown at item B in FIG. 34, the thick-framed portion is the portion expanded by the present method.

As shown within the thick frame in FIG. 34, the High section in the QP quantization parameter (0-51), that is, the β (β (45) -β (51)) parameter corresponding to the QP quantization parameter (45-51) is expanded by increasing the slope of the linear a function that directly sets the parameter to a larger tilt angle. That is, this way of expanding β is represented by the following expression (19).

Math Formula 19

In addition, although k3 = 4 is set here, k3 is not limited to 4. That is, although 4 is suitable as a value for k3 when considering a shift operation, the value of k3 may also be 3, 5, 6, etc.

The deblocking filter 31 in the fourth embodiment is basically the same as the deblocking filter 31 described with reference to FIG. 14, except for the configuration of the β generating unit 112 shown in FIG. 15. Therefore, hereinafter, an example of the configuration of the β forming unit 112 having a different configuration will be described.

An example of the configuration of the formation block β

FIG. 35 is a block diagram of a configuration example of the β forming unit.

In the example shown in FIG. 35, the β generating unit 112 is configured to contain the β LUT_input calculating unit 121 shown in FIG. 16, and block 251 forming β.

The QP quantization parameter obtained from the QP average calculator 111b is supplied to the β LUT_input calculator 121. The value (β_offdset_div2) of the β offset for the information set by the configuration setting unit 101 is supplied to the β LUT_input calculating unit 121.

The β LUT_input calculator 121 calculates βtmp, which is the β LUT_input qp value input to the β generation processing unit 251. That is, the β LUT_input calculator 121 calculates βtmp by adding the offset value β and the quantization parameter QP obtained from the QP average calculating unit 111b, and supplies the calculated βtmp value to the βtmp processing unit 251 to generate β.

The β generation processing unit 251 has a βLUT (Look Up Table) based on offline evaluation of expression (19). The β generation processing unit 251 takes the βtmp value calculated by the β LUT_input calculator 121 as β LUT_input qp, calculates β using βLUT (expression (19)), and supplies the calculated β value to the filter determination unit 114.

Moreover, the process of the deblocking filter 31 in the fourth embodiment is basically the same as the process of the deblocking filter 31 described with reference to FIG. 23 except for the β generation processing in step S123 shown in FIG. 24.

Therefore, with reference to the flowchart of FIG. 36, a β generation process (step S123 in FIG. 24) which is a different process will be described.

In step S251, the β LUT_input calculator 121 calculates βtmp = QP + β_offdset_div2 * 2, which is the β LUT_input qp value that is input to the β generation processing unit 251, and supplies the calculated βtmp value to the β generation processing unit 251.

In addition, in practice, the offset value (β_offdset_div2) for β is halved and transmitted and doubled and used when β LUT_input qp is calculated in step S251.

In step S252, the β generation processing unit 251 takes the βtmp calculated by the β LUT_input calculator 121 as β LUT_input qp, calculates β using the βLUT (expression (19)), and supplies the calculated β value to the filter determination unit 114.

As described above, the filtering strength is increased by increasing the slope of the linear function β of the High portion in the quantization parameter QP (0-51). In this case, the release force when displacement is given can be easily increased without any small effect on performance or installation under the current conditions.

In addition, although in the above explanation, the β parameter offset and the Tc parameter offset are transmitted from the coding side to the decoding side, an example of sharing the β and Tc parameter offset will now be described.

7. Fifth embodiment (first transmission method

Deblocking filter configuration example

FIG. 37 is a block diagram of an example of a configuration of a deblocking filter when the offsets of the β and Tc parameters are used together.

In the example shown in FIG. 37, the deblocking filter 31 is configured to insert a configuration setting unit 301 and a filter execution unit 302.

In contrast to the deblocking filter 31 shown in FIG. 14, the configuration setting unit 301 input the offset value (dblk_offset_div2) of the deblocking filter common in the parameters β and Tc.

That is, the configuration setting unit 301 receives the deblocking filter offset value and the DisableDeblockingFilter flag information (i.e., on / off information), for example, input from an operation unit (or lossless decoder 62) not shown in the drawing. The configuration setting unit 301 performs a configuration setting in the deblocking filter 31 based on the input information. That is, in the configuration setting unit 301, as one of the configuration settings, the deblocking filter offset value is set as the deblocking filter offset β and Tc. The configuration setting unit 301 supplies the setting information to the filter execution unit 302.

The pixel value before passing the deblocking filter from the arithmetic operations unit 30 (or the arithmetic operations unit 65) is supplied to the filter execution filter 302. The filter execution unit 302 calculates the quantization parameter QP used to generate the parameters based on information set by the configuration setting unit 301 such as the deblocking filter offset value, generates the β and Tc parameters, and determines whether filtering or filter strength is required. Then, the filter execution unit 302 performs filtering based on the determined filtering strength or on the result of a determination as to whether filtering is required. The filter execution unit 302 supplies the pixel value after the deblocking filter to the next adaptive bias filter 41 (or adaptive bias filter 81).

In addition, the filter execution unit 302 performs the above-described processing on both the luminance border and the color contrast border.

An example of the configuration of the filter execution unit at the brightness border

FIG. 38 is a block diagram of a configuration example of a filter execution unit performing luma edge filtering.

In the example shown in FIG. 38, the filter execution unit 302 is configured to provide therein a QP calculating unit 111, a filtering determination unit 114, a filtering unit 115, a β generating unit 311, and a Tc generating unit 312.

In addition, the filter execution unit 302 is the same as the filter execution unit 102 shown in FIG. 15, in which a QP computation unit 111, a filtering determination unit 114, and a filtering unit 115 are provided. The filter execution unit 302 is different from the filter execution unit 102 shown in FIG. 15 in that the β generating unit 112 is replaced by the β generating unit 311 and the Tc generating unit 113 is replaced by the Tc generating unit 312.

That is, similar to the β generating unit 112, the β generating unit 311 generates a parameter β corresponding to the QP quantization parameter obtained from the QP average calculating unit 111b. In this case, in the β generating unit 311, the offset value (dblk_offset_div2) of the deblocking filter obtained from the configuration setting unit 301 is used as the offset value. The β generating unit 311 supplies the generated β parameter to the filter determination unit 114.

Similar to the Tc generating unit 113, the Tc generating unit 312 generates a Tc parameter corresponding to the QP quantization parameter obtained from the QP average calculating unit 111b. In this case, in the Tc generating unit 312, the offset value (dblk_offset_div2) of the deblocking filter obtained from the configuration setting unit 301 is used as the offset value. The Tc generating unit 312 supplies the generated Tc parameter to the filtering determination unit 114.

An example of the configuration of the formation block β

FIG. 39 is a block diagram of a configuration example of the β forming unit.

In the example shown in FIG. 39, the β generating unit 311 is configured to provide therein a β LUT_input calculator 321 and a β generating processing unit 322.

The QP quantization parameter obtained from the QP average calculator 111b is supplied to the β LUT_input calculator 321. The value (dblk_offset_div2) of the offset of the deblocking filter of the information set by the configuration setting unit 301 is supplied to the β LUT_input calculating unit 321.

The β LUT_input calculator 321 calculates βtmp by adding the QP quantization parameter obtained from the QP average calculator 111b and the deblocking filter offset value, and supplies the calculated βtmp value to the βtmp processing unit 322 to generate β.

The β generation processing unit 322 has, for example, a βLUT (Look Up Table) defined in the HEVC method. The β generation processing unit 322 takes the βtmp value from the β LUT_input calculator 321 as β LUT_input qp, calculates β using the βLUT, and supplies the calculated β value to the filter determination unit 114.

An example of the configuration of the Tc formation unit

FIG. 40 is a block diagram of a configuration example of a Tc generating unit.

In the example shown in FIG. 40, the Tc generating unit 312 is configured to include a TcLUT_input calculating unit 331 and a Tc generating processing unit 332.

The QP quantization parameter obtained from the QP average computation unit 111b is supplied to the TcLUT_input computation unit 331. The value (dblk_offset_div2) of the offset of the deblocking filter information set by the configuration setting unit 301 is supplied to the TcLUT_input computation unit 331.

The TcLUT_input calculator 331 calculates Tctmp by adding the quantization parameter QP obtained from the QP average calculating unit 111b and the deblocking filter offset value, and supplies the calculated Tctmp value to the Tc generation processing unit 332.

The Tc generating processing unit 332 has, for example, a TcLUT (Look Up Table) defined in the HEVC method. The Tc generation processing unit 332 receives the Tctmp value from the TcLUT_input computation unit 331 as the TcLUT_input qp, calculates the Tc using the TcLUT, and supplies the calculated Tc value to the filter determination unit 114.

Also, for example, in the configuration of the filter execution unit that performs color contrast border filtering, the Tc generating unit 113 in the filter execution unit 102 described above with reference to FIG. 22 is simply replaced by the Tc generating unit 312 shown in FIG. 38. Accordingly, no explanation is provided for its operation.

Deblocking filter operation procedure

Next, with reference to the flowchart shown in FIG. 41, the deblocking filtering process will be described. In addition, the processing in steps S302 to S308 in FIG. 41 is basically the same as the process in steps S102 to S108 in FIG. 23. However, in filtering each boundary in S305 and S306, the β generation process in S123 and the Tc generation process in S124 in FIG. 24 are different. Therefore, details will be described later with reference to FIG. 43 and 44.

For example, the information of the DisableDeblockingFilter flag (ie, on / off information) and the offset value of the deblocking filter are input through an operation block (or lossless decoder 62) not shown in the diagram.

In step S301, the configuration setting unit 301 sets the filter configuration. Details of the configuration setting process will be described later with reference to FIG. 42. The configuration is established in step S301. In this case, the information of the DisableDeblockingFilter flag is set and the value (dblk_offset_div2) of the offset of the deblocking filter is set.

The deblocking filter offset value set here is used in the β generation process in step S123 and in the Tc generation process in step S124 shown in FIG. 24, and in the Tc generation process in step S223 shown in FIG. 29, which describes the process in step S305.

In step S302, the configuration setting unit 301 determines whether the deblocking filter can be used based on the information of the DisableDeblockingFilter flag. When it is determined in step S302 that the deblocking filter cannot be used, the deblocking filtering process ends.

When it is determined in step S302 that the deblocking filter can be used, the process proceeds to step S303.

In step S303, the configuration setting unit 301 determines the boundary between the TU and the PU. In step S304, the configuration setting unit 301 determines the BS value based on the boundary information between the TU and the PU determined in step S303, prediction mode information, etc. The configuration setting unit 301 supplies the BS value information to the QP block 111 302 filter versions.

In step S305, the filter execution unit 302 performs luminance edge filtering (LUMA). Although the process is not explained as it has been described above with reference to FIG. 24, filtering is performed at the luminance edge by the process in S105.

In step S306, the filter execution unit 302 performs color contrast edge filtering (CHROMA). Although the process is not explained as it has been described above with reference to FIG. 29, filtering is performed on the border of the color contrast by the process in step S106.

In step S307, the filter execution unit 302 determines whether all boundaries have been processed. When it has been determined in step S307 that not all of the boundaries have been processed, the process returns to step S305 to repeat the subsequent processes.

When it is determined in step S307 that all boundaries have been processed, the process proceeds to step S108.

In step S308, the configuration setting unit 301 determines whether all CUs have been processed. When it has been determined in step S308 that not all boundaries have been processed, the process returns to step S303 to repeat the subsequent processes.

When it is determined in step S308 that all CUs have been processed, the deblocking filtering process ends.

Filter configuration process

Next, with reference to the flowchart shown in FIG. 42, a filter configuration setting process in S301 shown in FIG. 41.

The deblocking control flag information DisableDeblockingFilter (i.e., on / off information) and the offset value of the deblocking filter are input via an operation block (or lossless decoder 62) not shown in the diagram.

In step S321, the configuration setting unit 301 determines whether or not the deblocking control flag is set to 1. When it is determined in step S221 that the deblocking control flag is set to 1, the process proceeds to step S322.

When the deblocking control flag is set to 1, the DisableDeblockingFilter flag and the deblocking offset can be transmitted. Accordingly, in step S322, the configuration setting unit 301 reads and sets the DisableDeblockingFilter flag.

In step S323, the configuration setting unit 301 reads and sets the offset value (dblk_offset_div2) of the deblocking filter.

On the other hand, when it is determined in step S321 that the deblocking control flag is set to 0, the configuration setting process ends. That is, when it is determined in step S321 that the deblocking control flag is set to 0, the DisableDeblockingFilter flag and the deblocking offset are not transmitted. Accordingly, the DisableDeblockingFilter flag and the deblocking offset are not set and the default values are used.

Formation process β

Next, with reference to the flowchart shown in FIG. 43, the β generation process in S123 shown in FIG. 24.

The deblocking filter offset value (dblk_offset_div2) is input to the β LUT_input calculator 321 from the configuration setting unit 301. In step S331, the β LUT_input calculator 321 calculates βtmp = QP + β_offdset_div2 * 2, which is a β LUT_input qp value that is input to the β generation processing unit 322, and supplies the calculated βtmp value to the β generation processing unit 322.

In addition, similar to the offset value β described above, in practice, the offset value (dblk_offset_div2) is halved and transmitted and doubled and used when the β LUT_input qp is calculated in step S331.

In step S332, the β generating processing unit 322 takes the βtmp calculated by the β LUT_input calculator 321 as β LUT_input qp, calculates β using the βLUT, and supplies the calculated Tc value to the filter determination unit 114.

Tc formation process

Next, with reference to the flowchart shown in FIG. 44, the Tc generating process in step S124 shown in FIG. 24.

The deblocking filter offset value (dblk_offset_div2) is input to the TcLUT_input computation unit 331 from the configuration setting unit 301. In step S341, the TcLUT_input calculator 331 calculates the Tctmp = QP + dblk_offset_div2 * 2 value, which is the Tc LUT_input qp value, which is input to the Tc generation processing unit 332, and supplies the calculated Tctmp value to the Tc generation processing unit 332.

In addition, like the offset value β described above, in practice, the offset value (dblk_offset_div2) is halved and transmitted and doubled and used when the Tc LUT_input qp is calculated in step S341.

In step S342, the Tc generating processing unit 332 takes the Tctmp value calculated by the TcLUT_input calculating unit 331 as the TcLUT_input qp, calculates Tc using the TcLUT, and supplies the calculated β value to the filter determination unit 114.

Passing the value (dblk_offset_div2) of the deblocking filter offset

Next, transmission of the offset value (dblk_offset_div2) of the deblocking filter will be described with reference to FIG. 45.

In the image coding device 11, the offset value (dblk_offset_div2) of the deblocking filter is input to the deblocking filter 31a and the lossless encoder 26 via an operation unit (not shown). The offset value (dblk_offset_div2) of the deblocking filter is used for filtering in the deblocking filter 31a and is encoded with an SVLC (Sin VLC) record in the lossless encoder 26. The encoded value (dblk_offset_div2) of the deblocking filter offset is transmitted to the image decoding apparatus 51 as a parameter of the deblocking filter.

The deblocking filter offset value (dblk_offset_div2) transmitted from the picture encoding apparatus 11 is decoded by reading SVLC (Sin VLC) in the lossless encoder 62 of the picture decoding apparatus 51, and is used for filtering in the deblocking filter 31b.

Deblocking filter offset value syntax example

FIG. 46 shows an example of the syntax of a deblocking filter offset value.

In the example shown in FIG. 46, the deblocking filter offset value (dblk_offset_div2) is described in the if statement of the disaeble_deblocking_filter_flag, and the dblk_offset_div2 value is used if disaeble_deblocking_filter_flag is not 0.

In addition, this syntax is described in the slice header and PPS (picture parameter set).

As described above, using the offset value common in the β and Tc parameters, one syntax element can be shortened, and the filtering strength of the deblocking filter can be easily controlled.

An example will now be described in which, when the offsets of the β and Tc parameters are encoded, the offset of the β parameter and the difference between the offset of the β parameter and the offset of the Tc parameter are encoded and transmitted.

8. Sixth Embodiment (Second Bias Transfer Method)

Deblocking filter configuration example

FIG. 47 is a block diagram of a configuration example of a deblocking filter unit in an image encoding apparatus.

In the example shown in FIG. 47, the deblocking filter 31a is configured to contain a configuration setting unit 401 and a filter execution unit 302.

The offset value (β_offset_div2) of the β parameter and the offset value (tc_offset_div2) of the tc parameter are input to the configuration setting block 401.

The configuration setting unit 401 receives the information of the DisableDeblockingFilter flag (i.e., on / off information), the offset value of the β parameter, and the offset value of the tc parameter, which are input from, for example, an operation unit (not shown). The configuration setting unit 401 performs a configuration setting in the deblocking filter 31a based on the inputted information. That is, in the configuration setting block 401, offset values for β and tc of the deblocking filter are set. The configuration setting unit 401 supplies setting information to the filter execution unit 302.

In addition, in this case, the configuration setting unit 401 generates difference information (diff_β_tc_div2) between the offset value (β_offset_div2) of the β parameter and the offset value (tc_offset_div2) of the tc parameter. The configuration setting unit 401 supplies the generated difference information and the parameter offset value β to the lossless encoder 26.

The filter execution unit 302 is basically the same as the filter execution unit 302 shown in FIG. 37. Filter execution unit 302 calculates a quantization parameter QP used to generate parameters based on information set by configuration setting unit 401 such as deblocking filter offset value, generates β and Tc parameters, and determines whether filtering or filter strength is required.

In this case, in the filter execution unit 302 shown in FIG. 37, the offset value (dblk_offset_div2) of the deblocking filter is used. On the other hand, in the filter execution unit 302 shown in FIG. 47, the difference is that the offset value (β_offset_div2) of the β parameter and the value (tc_offset_div2) of the tc parameter are used. Therefore, the configuration example of the filter execution unit 302 in the example of FIG. 47 and a configuration example of each of the blocks constituting the filter execution block 302 is omitted.

In addition, the deblocking filter process 31a shown in FIG. 47 is basically the same as the process described above with reference to FIG. 41 except for the filter configuration setting process in step S301. Therefore, the filter configuration setting process in step S301 in FIG. 41, which is another process, will be described with reference to FIG. 48.

Filter configuration process

Next, with reference to the flowchart shown in FIG. 48, the filter configuration setting process in step S301 shown in FIG. 41.

The deblocking control flag information, the DisableDeblockingFilter flag information (i.e., on / off information), and the offset values of the β and tc parameters are input via an operation unit (not shown).

In step S411, the configuration setting unit 401 determines whether or not the deblocking control flag is set to 1. When it is determined in step S411 that the deblocking control flag is set to 1, the process proceeds to step S412.

When the deblocking control flag is set to 1, the DisableDeblockingFilter flag and offset can be passed. Accordingly, in step S412, the configuration setting unit 401 reads and sets the DisableDeblockingFilter flag.

In step S413, the configuration setting unit 401 reads and sets the offset value (dblk_offset_div2) of the β parameter.

In step S414, the configuration setting unit 401 reads and sets the offset value (tc_offset_div2) of the tc parameter.

In step S415, the configuration setting unit 401 generates a diff_β_tc_div2 value, which is the difference between the β parameter offset value and the Tc parameter offset value. The configuration setting unit 401 supplies the generated difference information diff_β_tc_div2 to the lossless encoder 26 along with the offset value of the parameter β.

On the other hand, when it is determined in step S411 that the deblocking control flag is set to 0, the configuration setting process ends. That is, when it is determined in step S411 that the deblocking control flag is set to 0, the DisableDeblockingFilter flag and the offset are not transmitted. Accordingly, the DisableDeblockingFilter flag and offset are not set and the default values are used.

Deblocking filter configuration example

FIG. 49 is a block diagram of a configuration example of a deblocking filter unit in an image decoding apparatus.

In the example shown in FIG. 49, the deblocking filter 31b is configured to include a configuration setting unit 451 and a filter execution unit 302.

The offset value (β_offset_div2) of the β parameter and and the difference information (diff_β_tc_div2) between the offset value of the β parameter and the offset value of the Tc parameter are input to the configuration setting block 451.

The configuration setting unit 451 receives information of the DisableDeblockingFilter flag (ie, on / off information), the offset value of the β parameter, and difference information, which are input from, for example, lossless decoder 62. The configuration setting unit 451 performs a configuration setting in the deblocking filter 31b based on the inputted information. That is, in the configuration setting block 451, offset values for β and tc of the deblocking filter are set. In addition, in this case, the Tc offset value is calculated and set by adding the difference information and the β offset value. The configuration setting unit 451 supplies setting information to the filter execution unit 302.

The filter execution unit 302 is basically the same as the filter execution unit 302 shown in FIG. 37. Filter execution unit 302 calculates a quantization parameter QP used to generate parameters based on information set by configuration setting unit 301 such as a deblocking filter offset value, generates β and Tc parameters, and determines whether filtering or filter strength is required.

In this case, in the filter execution unit 302 shown in FIG. 37, the offset value (dblk_offset_div2) of the deblocking filter is used. On the other hand, in the filter execution unit 302 shown in FIG. 49, the difference is that the offset value (β_offset_div2) of the β parameter and the offset value (tc_offset_div2) of the Tc parameter are used. Therefore, the configuration example of the filter execution unit 302 in the example of FIG. 49 and an example of the configuration of each of the blocks constituting the filter execution block 302 are omitted.

In addition, the deblocking filtering process of the deblocking filter 31b shown in FIG. 49 is basically the same as the process described above with reference to FIG. 41 except for the filter configuration setting process in step S301. Therefore, the filter configuration setting process in step S301 in FIG. 41, which is another process, will be described with reference to FIG. 50.

Filter configuration process

Next, with reference to the flowchart shown in FIG. 50, the filter configuration setting process in step S301 shown in FIG. 41.

The deblocking control flag information, the DisableDeblockingFilter flag information (i.e., enable / disable information), the offset value (β_offset_div2) of the β parameter, and the difference information (diff_β_tc_div2) are input through the decoder 62 without loss.

In step S421, the configuration setting unit 451 determines whether or not the deblocking control flag is set to 1. When it is determined in step S421 that the deblocking control flag is set to 1, the process proceeds to step S422.

When the deblocking control flag is set to 1, the DisableDeblockingFilter flag and offset can be passed. Accordingly, in step S422, the configuration setting unit 451 reads and sets the DisableDeblockingFilter flag.

In step S423, the configuration setting unit 451 reads and sets the offset value (β_offset_div2) of the β parameter.

In step S424, the configuration setting unit 451 calculates and sets the offset value (tc_offset_div2) of the Tc parameter by adding the offset value (β_offset_div2) of the β parameter to the difference information diff_β_tc_div2.

On the other hand, when it is determined in step S421 that the deblocking control flag is set to 0, the configuration setting process ends. That is, when it is determined in step S421 that the deblocking control flag is set to 0, the DisableDeblockingFilter flag and the offset are not transmitted. Accordingly, the DisableDeblockingFilter flag and offset are not set and the default values are used.

Passing Offset Value and Difference Information

Next, transmission of the difference information offset value (β_offset_div2) (diff_β_tc_div2) will be described with reference to FIG. 51.

In the image encoding apparatus 11, an offset value (β_offset_div2) for β and an offset value (tc_offset_div2) for Tc are input to the deblocking filter 31a via an operation unit (not shown). In the deblocking filter 31a, filtering is performed using an offset value (β_offset_div2) for β and an offset value (tc_offset_div2) for tc.

In addition, in the deblocking filter 31a, the difference information (diff_β_tc_div2) and the offset value (β_offset_div2) for β and the difference information (diff_β_tc_div2) are generated to the encoder 26 without loss. Then, the offset value (β_offset_div2) for β and the difference information (diff_β_tc_div2) are encoded by writing SVLC (Sin VLC) to the lossless encoder 26, and transmitted to the image decoding apparatus 51 as a deblocking filter parameter.

The offset value (β_offset_div2) for β and the difference information (diff_β_tc_div2) transmitted from the picture encoding device 11 are decoded by reading SVLC (Sin VLC) in the lossless encoder 62 of the picture decoding device 51, they are used in the deblocking filter 31b, and the offset value ( tc_offset_div2) for Tc. Then, in the deblocking filter 31b, the offset value (β_offset_div2) for β and the calculated offset value (tc_offset_div2) for Tc are used for filtering.

In addition, although an example has been described above in which the difference information between the offset values of the two parameters is generated in the configuring unit 401 and supplied to the lossless encoder 26, the difference information can be generated in the lossless encoder 26.

Deblocking filter offset value syntax example

FIG. 52 shows an example of the syntax of a deblocking filter offset value.

In the example shown in FIG. 52, the value (beta_offset_div2) for the offset β and the difference information (diff_beta_tc_div2) are described in an "if" statement for disaeble_deblocking_filter_flag. Accordingly, if disaeble_deblocking_filter_flag is not 0, the value (beta_offset_div2) for offset β and the difference information (diff_beta_tc_div2) are used.

In addition, this syntax is described in the slice header and PPS (picture parameter set).

In addition, although in the above explanation, the offset for β is transmitted along with the difference information, it is also possible to transmit the offset Tc. In this case, on the decoding side, the β offset is calculated from the difference information and the tc offset.

As described above, instead of transmitting the offsets of the two parameters, it is also possible to transmit the offset value of one of the parameters and the difference information between the offsets of one parameter and the other.

Since there is a correlation between the value of the offset parameter β and the value of the offset parameter Tc, these values in many cases are mutually blocked. In this case, since the difference information is small, it is possible to reduce the number of bits at the time of encoding.

As mentioned above, the HEVC method was used as the basis for the encoding method. However, the present disclosure is not limited to this, and other encoding / decoding methods comprise at least a deblocking filter that can be used as an in-loop filter.

The present method can be applied to image coding devices and image decoding devices used to receive image information (bitstream) compressed using an orthogonal transform such as discrete cosine transform and motion compensation as in the HEVC method, for example, through a media network such as such as satellite broadcasting, cable TV, the Internet, or mobile telephone devices The present disclosure may also apply to image encoding devices and image decoding devices that are used in the processing of compressed image information on a storage medium such as an optical or magnetic disk or flash memory ...

9. Seventh Embodiment (Multi-Projection Image Coding / Multi-Projection Decoding Apparatus)

Multi-projection coding / multi-projection decoding applications)

The above sequence of processes can be applied to multi-projection coding and multi-projection decoding. FIG. 53 shows an example of a method for encoding a multi-projection image.

As shown in FIG. 53, the multi-projection image contains images of a plurality of projections, and an image of one predetermined projection of the plurality of projections is designated as the base projection image. An image of each projection other than the base projection image is treated as a non-baseline projection image.

When performing multi-projection encoding as shown in FIG. 53, the deblocking filter parameters (flag, parameter offset value, difference information, etc.) can be set in each projection (the same projection). In addition, in each projection (different projections) it is also possible to share the deblocking filter parameters set in other projections.

In this case, the deblocking filter parameter set in the baseline projection is used in at least one nonbasic projection. Alternatively, for example, the deblocking filter parameter set in the non-baseline projection (view_id = i) is used in at least the baseline projection or the non-baseline projection (view_id = j).

Therefore, with deblocking filtering, the filtering process can be applied properly. That is, it is possible to suppress block noise.

Multi-projection image encoder

FIG. 54 shows a multi-projection image coding apparatus performing the multi-projection coding described above. As shown in FIG. 54, a multiple projection image encoding apparatus 600 includes encoders 601 and 602 and a multiplexer 603.

The encoder 601 encodes a base projection image to generate a base projection image coding stream. An encoder 602 encodes a non-baseline projection image to generate a non-baseline projection image coding stream. The multiplexer 603 generates a multiple projection image coding stream by multiplexing a base projection image coding stream generated by encoder 601 and a non-base projection image coding stream generated by encoder 602.

For the encoders 601 and 602 of the multi-projection image coding apparatus 600, the image coding apparatus 11 (FIG. 1) may be applied. In this case, the multi-projection encoder 600 sets and transmits the deblocking filter parameter set by the encoder 600 and the deblocking filter parameter set by the encoder 602.

In addition, a deblocking filter parameter set by encoder 601, as described above, may be set and transmitted to be shared by encoders 601 and 602. In contrast, a deblocking filter parameter set with encoder 602 may be set and transmitted to be shared encoders 601 and 602.

Multi-projection decoding device

FIG. 55 shows a multi-projection decoding apparatus performing multi-projection decoding described above. As shown in FIG. 55, a multiple projection image decoding apparatus 610 includes a demultiplexer 613 and decoders 611 and 612.

The demultiplexer 611 demultiplexes the multiplex image coding stream in which the base projection image coding stream and the non-base projection image coding stream are multiplexed to extract the base projection image coding stream and the non-base projection image coding stream. Decoder 612 obtains the base projection image by decoding the base projection image coding stream extracted by the demultiplexer 611. Decoder 613 obtains the base projection image by decoding the base projection image coding stream extracted by the demultiplexer 611.

For decoders 612 and 613 of the multi-projection image decoding apparatus 610, the image decoding apparatus 51 (FIG. 3) may be used. In this case, the multi-projection decoding apparatus 610 performs a process using a deblocking filter parameter set by encoder 601 and decoded by decoder 612, and a deblocking filter parameter set by encoder 602 and decoded by decoder 613.

In addition, a deblocking filter parameter set by encoder 601 (or encoder 602) as described above may be set and transmitted to be shared by encoders 601 and 602. In this case, the multi-projection decoding apparatus 610 performs a process using the deblocking filter parameter set by encoder 601 (or encoder 602) and decoded by decoder 612 (or decoder 613).

10. Eighth embodiment (hierarchical picture encoding / hierarchical picture decoding apparatus)

Applications for Hierarchical Image Coding and Hierarchical Image Decoding

The above sequence of processes can be applied to encoding a hierarchical image and decoding a hierarchical image. FIG. 56 shows an example of a method for encoding a multi-projection image.

As shown in FIG. 56, a hierarchical image contains images of a plurality of hierarchies (resolutions), and an image of one predetermined hierarchy of a plurality of resolutions is designated as a base level image. An image of each hierarchy other than the baseline image is treated as a non-baseline image.

When performing hierarchical image encoding (spatial scalability) as shown in FIG. 56, the deblocking filter parameter can be set at each level (the same level). In addition, at each level (different levels) it is also possible to share the deblocking filter parameters set at other levels.

In this case, the deblocking filter parameter set at the basic level is used at at least one non-basic level. Alternatively, for example, the deblocking filter parameter set in the non-basic layer (layer_id = i) is used at least in the basic layer or in the non-basic layer (layer_id = j).

Therefore, filtering can be applied properly during the release process. That is, block noise can be suppressed.

Hierarchical Image Encoder

FIG. 57 shows a hierarchical picture encoding apparatus performing the hierarchical picture encoding described above. As shown in FIG. 57, a hierarchical picture encoder 620 includes encoders 621 and 622 and a multiplexer 623.

An encoder 621 encodes a base layer picture to generate a coding stream of a base layer picture. An encoder 622 encodes a non-base layer picture to generate a non-base layer picture coding stream. A multiplexer 623 generates a hierarchical picture coding stream by multiplexing a base layer picture coding stream generated by encoder 621 and a non-base layer picture coding stream generated by encoder 622.

For the encoders 621 and 622 of the hierarchical picture coding apparatus 620, the picture coding apparatus 11 (FIG. 1) may be used. In this case, the hierarchical picture coding apparatus 620 sets and transmits the deblocking filter parameter set by the encoder 621 and the deblocking filter parameter set by the encoder 602.

In addition, a deblocking filter parameter set by encoder 621 as described above may be set and transmitted to be shared by encoders 621 and 622. In contrast, a deblocking filter parameter set by encoder 622 may be set and transmitted to be shared by encoders 621 and 622.

Hierarchical image decoding device

FIG. 58 shows a hierarchical picture decoding apparatus performing hierarchical picture decoding described above. As shown in FIG. 58, a hierarchical picture decoding apparatus 630 includes a demultiplexer 631 and decoders 632 and 633.

The demultiplexer 631 demultiplexes a hierarchical picture coding stream in which a base layer picture coding stream and a non-base layer picture coding stream are multiplexed to extract a base layer picture coding stream and a non-base layer picture coding stream. Decoder 632 obtains a base layer picture by decoding a base layer picture coding stream extracted by demultiplexer 631. Decoder 633 obtains a non-base layer picture by decoding a non-base layer picture coding stream extracted by demultiplexer 631.

For decoders 632 and 633 of the hierarchical picture decoding apparatus 630, the picture decoding apparatus 51 (FIG. 3) may be used. In this case, the hierarchical picture decoding apparatus 630 performs a process using the deblocking filter parameter set by the encoder 621 and decoded by the decoder 632 and the deblocking filter parameter set by the encoder 622 and decoded by the decoder 633.

In addition, the deblocking filter parameter set by encoder 621 (or encoder 622) as described above may be set and transmitted to be shared by encoders 621 and 622. In this case, the hierarchical picture decoding apparatus 630 performs a process using the deblocking filter parameter set by encoder 621 (or encoder 622) and decoded by decoder 632 (or decoder 633).

11. Ninth embodiment

Computer configuration example

Further, the series of processes described above may be performed by hardware or software. When the above series of processes are executed by the software, the programs constituting the software are installed on the computer. Note that the computer examples include a computer embedded in specialized hardware and a general purpose personal computer capable of performing various functions by installing various programs on it.

FIG. 59 is a block diagram of a hardware configuration example of a computer executing the sequence of the above processes using a program.

In the computer 800, the CPU (central processing unit) 801, ROM (read only memory) 802, and RAM (random access memory) 803 are connected to each other via a bus 804.

In addition, an I / O interface 805 is connected to the bus 804. Connected to the I / O interface 805 are an input device 806, an output device 807, a storage unit 808, a communication unit 809, and a drive 810.

The input device 806 is made in the form of a keyboard, mouse, microphone, etc. The output device 807 is made in the form of a display, a loudspeaker, etc. The storage unit 808 contains a hard disk, a non-volatile memory, etc. The communication unit 809 contains a network interface, etc. The drive 810 drives removable media 821, such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory.

In a computer executed as described above, the CPU 801 downloads, for example, the program stored in the memory unit 808 to the RAM 803 via the I / O interface 805 and the bus 804, and executes the program, thus executing the series of processes described above. ...

The program executed by the computer 800 (CPU 801) may be recorded on a removable medium and provided, for example, as a mounted medium. In addition, the program can be delivered via wired or wireless transmission media such as local area network, Internet, and digital satellite broadcasting.

In a computer, the program can be installed on the storage unit 808 through the I / O interface 805 by mounting removable media 821 on the drive 810. In addition, the program can be received by the communication unit 809 via wired or wireless media and installed in the storage unit 808. In addition, the program can be pre-installed in ROM 802 or in storage unit 808.

The programs to be executed by the computer may be programs for executing processes in chronological order in accordance with the sequence described herein, or programs for executing processes in parallel or whenever needed in response to a request.

In the present description, the steps describing programs to be recorded on the recording medium contain processes to be executed in parallel or independently of each other if they do not need to be executed in chronological order, as well as processes to be performed in chronological order in accordance with with the sequence described here.

As used herein, the term "system" refers to all equipment containing more than one device.

Additionally, any structure described above as one device (or one processor) can be split into two or more devices (or processors). In contrast, any structure described above as two or more devices (or processors) can be combined into one device (or one processor). Additionally, it is, of course, possible to add components other than those described above to the structure of any of the devices (or to any of the processors). Additionally, some components of a device (or processor) may be incorporated into the structure of another device (or processor) as long as the structure and function of the system as a whole remains substantially the same. That is, the present method is not limited to the above-described embodiments, and various changes can be made therein without departing from the scope of the method.

Image encoding devices and image decoding devices according to the embodiments described above can be applied to various electronic devices such as transmitters and receivers in satellite broadcasting, cable broadcasting such as cable TV, Internet distribution, cellular distribution to terminals, and the like, recording devices capable of recording images on a medium such as magnetic disks and flash memory, and reproducing devices capable of reproducing images from the data storage medium. Four examples of applications will be described below.

12. Applications

First example: TV receiver

FIG. 60 shows an example of the structure of a television device to which the above-described embodiments are applied. The television device 900 includes an antenna 901, a tuner 902, a demultiplexer 903, a decoder 904, a video processor unit 905, a display unit 906, an audio processor unit 907, a speaker 908, an external interface 909, a controller 910, a user interface 911, and a bus 912.

Tuner 902 extracts the desired channel signal from the broadcast signals received through antenna 901 and demodulates the extracted signal. The tuner 902 then outputs the encoded bitstream resulting from the demodulation to the demultiplexer 903. That is, in the television device 900, the tuner 903 serves as a transmitter that receives the encoded encoded picture stream.

The demultiplexer 903 selects the video stream and the audio stream of the program to be viewed from the encoded bitstream and outputs the divided streams to the decoder 904. The demultiplexer 903 also extracts ancillary data such as EPG (Electronic Program Guide) from the encoded bitstream and feeds the extracted data to the controller 910. If the coded bitstream is scrambled, demultiplexer 903 may descramble the coded bitstream.

Decoder 904 decodes the video stream and audio stream from demultiplexer 903. Decoder 904 then outputs video data generated by decoding to video processor 905. Decoder 904 then outputs audio data generated by decoding to audio processor 907.

The video processor section 905 reproduces the video data input from the decoder 904 and displays the video data on the display unit 906. The video processor 905 may also display an application screen supplied via the network to the display unit 906. Additionally, the video processor 905 can perform additional processing such as removing (suppressing) video noise depending on the settings. The video processor 905 can additionally generate a GUI (Graphical User Interface) image such as a menu, button or cursor, and superimpose the generated image on the output images.

The display unit 906 is driven by a trigger signal from the video processor 905 and displays video data or images on a video screen of a display device (such as a liquid crystal display, plasma display, or OELD (organic electroluminescence display, organic EL display).

The audio processor 907 performs a reproduction process such as D / A conversion and amplification of the audio data from the decoder 904 and outputs the audio signal through the speaker 908. Additionally, the audio processor 907 may perform additional processing such as removing (suppressing) audio data noise.

External interface 909 is an interface for connecting the television device 900 to an external device or network. For example, a video stream or an audio stream received via an external interface 909 can be decoded by a decoder 904. That is, an external interface 909 also serves as a transmission medium in a television device 900 that receives a coded stream of encoded pictures.

The controller 910 includes a processor, such as a CPU, and a memory device, such as a RAM and ROM. The memory device stores programs executed by the CPU, program data, EPG data, data received via a network, and the like. Programs stored in the memory device are read and executed by the CPU, for example, when the television device 900 is turned on. The CPU controls the operation of the television device 900 in accordance with control signals input from the user interface 911, for example, when executing programs.

User interface 911 connects to controller 910. User interface 911 includes buttons and switches for users to operate television device 900 and receiver unit, for example, to receive remote control signals. The user interface 911 detects the operation performed by the user through these components, generates a control signal, and outputs the generated control signal to the controller 910.

Bus 912 connects tuner 902, demultiplexer 903, decoder 904, video processor 905, audio processor 907, external interface 909, and controller 910.

In a television device 900 constructed in this manner, a decoder 904 has the functions of an image decoding device according to the embodiments described above. As a result, block noise can be suppressed when decoding pictures in the television device 900.

Second application example: Mobile phone device

FIG. 61 shows an example of the structure of a mobile telephone device to which the above-described embodiments are applied. Block 920 of the mobile telephone device contains an antenna 921, a communication unit 922, an audio codec 923, a speaker 924, a microphone 925, a camera unit 926, a video processor 927, a demultiplexer 928, a recording / playback unit 929, a display unit 930, a controller 931, an operating unit 932 and a bus 933.

Antenna 921 is connected to a communication unit 922. Loudspeaker 924 and microphone 925 are connected to audio codec 923. Operation unit 932 is connected to controller 931. Bus 933 connects communication unit 922, audio codec 923, camera unit 926, video processor 927, demultiplexer 928, recording / playback unit 929, display unit 930 to each other and controller 931.

The mobile telephone device 920 performs an operation such as transmitting / receiving audio signals, transmitting / receiving email and image data, receiving images, recording images, etc. in various operating modes, which include voice call mode, data mode, image acquisition and videophone mode.

In the voice call mode, analog audio signals generated by microphone 925 are provided to audio codec 923. Audio codec 923 converts the analog audio signal to audio data and A / D converts the converted audio data and compresses the audio data. Then, the audio codec 923 outputs the audio data after compression to the communication unit 922. Communication unit 922 encodes and modulates audio data to generate a transmission signal. Communication unit 922 then transmits the generated signal to a base station (not shown) via an antenna. 921. The communication unit 922 also amplifies and frequency converts the radio signal received through the antenna 92 to obtain a received signal. The communication unit 922 then demodulates and decodes the received signal to create audio data, and outputs the generated audio data to an audio codec 923. The audio codec 923 expands and D / A converts the audio data to generate an analog audio signal. The audio codec 923 then provides the generated audio signal to the speaker 924 for outputting the audio signal therethrough.

In the data transfer mode, the controller 931 generates text data, which, for example, must be entered into e-mail in accordance with the operation of the user and via the operation unit 932. The controller 931 also displays the text on the display unit 930. The controller 931 also generates e-mail data in response to a transmission command from a user via an operation unit 932, and outputs the generated e-mail data to a communication unit 922. Communication unit 922 encodes and modulates email data to generate a transmission signal. Communication unit 922 then transmits the generated signal to a base station (not shown) via an antenna. 921. The communication unit 922 also amplifies and frequency converts the radio signal received through the antenna 92 to obtain a received signal. The communication unit 922 then demodulates and decodes the received signal to recover the electronic data, and outputs the recovered email data to the controller 931. The controller 931 displays the contents of the email on the display unit 930 and stores the email data on the storage medium of the recording unit 929. playback.

The recording / reproducing unit 929 comprises a medium for storing read / write data. For example, the storage medium can be an internal storage medium such as RAM or flash memory, or it can be an external installable storage medium such as a hard disk, magnetic disk, magneto-optical disk, USB memory device (universal serial bus, universal serial bus) or memory card.

In the image acquisition mode, the camera unit 926 acquires an image of the object to generate image data, and outputs the generated image data to, for example, a video processor 927. The video processor 927 encodes the image data input from the camera unit 926 and stores the encoded stream on the unit's data storage medium 929 recording / playback.

In the videophone mode, the demultiplexer 928 multiplexes the video stream encoded by the video processor 927 and the audio stream input from the audio codec 923, and outputs the multiplexed stream, for example, to a communication unit 922. Communication unit 922 encodes and modulates the stream to generate a transmit signal. Communication unit 922 then transmits the generated signal to a base station (not shown) via an antenna. 921. The communication unit 922 also amplifies and frequency converts the radio signal received through the antenna 92 to obtain a received signal. The transmit signal and the received signal may contain encoded bit streams. The communication unit 922 then demodulates and decodes the received signal to reconstruct the stream, and outputs the reconstructed stream to a demultiplexer 928. The demultiplexer 928 separates the video stream and the audio stream from the input stream and outputs the video stream to a video processor 927 and the audio stream to an audio codec 923. Video processor 927 to decode the video stream video data. Video data is supplied to the display unit 930 and the image sequences are displayed by the display unit 930. Audio codec 923 expands and D / A converts audio data to generate an analog audio signal. The audio codec 923 then provides the generated audio signal to the speaker 924 for outputting the audio signal therethrough.

In a mobile telephone 920 having such a structure, the video processor 927 has the functions of image encoding apparatus and image decoding apparatus according to the embodiments described above. As a result, in the mobile telephone 920, block noise can be suppressed when encoding and decoding images.

Third application example: recording / playback device

FIG. 62 shows an example of the structure of a recording / reproducing apparatus to which the above-described embodiments are applied. The recording / reproducing apparatus 940 encodes audio data and video data of a received broadcast program and records the encoded data, for example, on a recording medium. The recording / reproducing apparatus 940 may also encode audio data and video data received from another apparatus, and write the encoded data onto, for example, a data recording medium. The recording / reproducing apparatus 940 also reproduces on a monitor and through a speaker data recorded on the data recording medium in response to, for example, a user command. In this case, the recording / reproducing apparatus 940 decodes audio data and video data.

The recorder / player 940 includes a tuner 941, an external interface 942, an encoder 943, a HDD unit 944 (hard disk drive), a disk drive 945, a switch 946, a decoder 947, an OSD unit 948, a controller 949, and a user interface 950.

Tuner 941 extracts the desired channel signal from the broadcast signals received via an antenna (not shown) and demodulates the extracted signal. The tuner 941 then outputs the encoded bitstream from the demodulation to the switch 946. That is, in the recorder / reproducing apparatus 940, the tuner 941 acts as a transmission medium.

External interface 942 is an interface for connecting the recording / reproducing apparatus 940 to an external device or network. External interface 942 may be, for example, an IEEE1394 interface, a network interface, a USB interface, or a flash memory interface. For example, video data and audio data received through the external interface 942 are input to an encoder 943. That is, in the recorder / reproduction apparatus 940, the external interface 942 acts as a transmission medium.

An encoder 943 encodes video data and audio data in the case where video data and audio data input from the external interface 942 are not encoded. Encoder 943 then outputs the encoded bitstream to switch 946.

The HDD 944 records the encoded bitstream on the internal hard disk when content data such as video data and audio data, various programs and other data is compressed. The HDD 944 reads data from the hard disk to play back video data and audio data.

The drive 945 writes to and reads data from the recording medium mounted thereon. The recording medium mounted on the 945 drive can be, for example, a DVD disc (such as DVD-Video, DVD-RAM, DVD-R, DVD-RW, DVD + R, or DVD + RW)) or Blu-ray disc ( registered trademark).

For recording video and audio data, switch 946 selects a coded bitstream that is input from tuner 941 or encoder 943, and outputs the selected coded bitstream to HDD 944 or drive 945. For video and audio data playback, switch 946 selects coded bitstream from HDD 944 or drive 945 to decoder 947.

Decoder 947 decodes the encoded bitstream and produces video data and audio data. Then, the decoder 947 outputs the generated video data to the OSD 948. In addition, the decoder 904 outputs the generated audio data to an external speaker.

OSD 948 reproduces video data that is input from decoder 947 and displays video data. In addition, the OSD 948 can overlay a GUI image such as a menu, button or cursor on the displayed video data.

The controller 949 includes a processor, such as a CPU, and a memory device, such as RAM and ROM. The memory device stores programs executable by the CPU, program data, and the like. Programs stored in the memory device are read and executed by the CPU, for example, when the recorder / reproducing apparatus 940 is turned on. The CPU controls the operation of the recording / reproducing apparatus 940 in accordance with control signals input from the user interface 950, for example, when executing programs.

User interface 950 connects to controller 949. User interface 950 includes buttons and switches for users to operate recorder / player 940 and a receiving unit, for example, to receive remote control signals. User interface 950 detects user-performed operation through these components, generates a control signal, and outputs the generated control signal to controller 949.

In the recording / reproducing apparatus 940 thus constructed, the encoder 943 has the functions of the image encoding apparatus according to the embodiments described above. Additionally, the decoder 947 has the functions of the picture decoding apparatuses corresponding to the above-described embodiments. As a result, block noise can be suppressed when encoding and decoding images in the recording / reproducing apparatus 940.

Fourth application example: imaging device

FIG. 63 shows an example of the structure of an image acquiring apparatus to which the above-described embodiments are applied. An image acquiring apparatus 960 acquires an image of an object to create an image, encode image data, and record the encoded image data on a data recording medium.

The image acquisition device 960 includes an optical unit 961, an image acquisition unit 962, a signal processor 963, a video processor 964, a display unit 965, an external interface 966, a memory device 967, a storage medium drive 968, an OSD 969, a controller 970, a user interface 971 and a bus 972 ...

The optical unit 961 is connected to the imaging unit 962. The image acquisition unit 962 is connected to the signal processor 963. The display unit 965 is connected to the video processor 964. The user interface 971 is connected to the controller 970. The bus 972 connects the video processor 964, the external interface 966, the memory device 967, the storage medium drive 968, OSD 969 and controller 970.

The optical unit 961 contains a lens, aperture, and the like. The optical unit 961 forms an optical image of an object on the image acquisition surface of the imaging unit 962. The imaging unit 962 contains an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) and converts the optical image obtained on the imaging surface into an image signal into the form of an electrical signal using photoelectric conversion. The image acquiring unit 962 then outputs the image signal to a signal processor 963.

The signal processor 963 performs various types of camera signal processing such as z-correction, gamma correction, and color correction of the image signal supplied from the imaging unit 962. The signal processor 963 outputs the image data after processing the camera signal to the video processor 964.

Video processor 964 encodes image data from signal processor 963 to generate encoded data. The video processor 964 then outputs the generated encoded data to an external interface 966 or to a storage medium drive 968. Video processor 964 also decodes encoded data from external interface 966 or storage medium drive 968 to generate image data. The video processor 964 then outputs the generated image data to the display unit 965. Video processor 964 may output image data from signal processor 963 to display unit 965 for displaying images. Video processor 964 may also superimpose display data from OSD 969 on images to be output to display unit 965.

The OSD 969 can generate a GUI image, such as a menu, button, or cursor, and output the generated image to, for example, video processor 964.

The external interface 966 is, for example, a USB I / O terminal. An external interface 966 connects to, for example, an imaging device 960 and a printer when printing the image. In addition, if necessary, the drive is connected to the external interface 966. The drive mounts a removable medium such as a magnetic disk or an optical disc, and a program that is read from the removable medium can be installed in the imaging device 960. Additionally, the external interface 966 can be a network interface connected to a network such as a LAN or the Internet. That is, the external interface 966 in the image acquiring apparatus 960 acts as a transmission medium.

The storage medium to be mounted on the storage medium drive 968 may be a read / write removable medium such as a magnetic disk, magneto-optical disk, optical disk, or semiconductor memory. Alternatively, the recordable medium can be fixedly mounted on the medium drive 968 to form a non-removable storage unit such as, for example, an internal hard disk drive or SSD (solid state drive).

Controller 970 includes a processor, such as a CPU, and a memory device, such as RAM and ROM. The memory device stores programs executed by the CPU, program data, and the like. Programs stored in the memory device are read and executed by the CPU, for example, when the imaging device 960 is turned on. The CPU controls the operation of the imaging apparatus 960 in accordance with control signals from the user interface 971, for example, when executing programs.

The user interface 971 is connected to the controller 970. The user interface 971 contains buttons and switches, for example, for operating the users with the imaging device 960. The user interface 971 detects the operation performed by the user through these components, generates a control signal, and outputs the generated control signal to the controller 970.

In the image acquiring apparatus 960 having such a structure, the video processor 964 has the functions of the image encoding apparatus and the image decoding apparatus according to the embodiments described above. As a result, block noise can be suppressed in the image acquiring apparatus 960 when encoding and decoding images.

13. Applications of scalable coding

The first system

Next, a specific example of using scalable encoded data obtained by scalable encoding (hierarchical encoding) described above will be described with reference to FIG. 56-58. For example, scalable coding is used to select data to be transmitted, as shown in the example in FIG. 64.

In the communication system 1000 shown in FIG. 64, the distribution server 1002 reads the scalable encoded data stored in the scalable encoded data storage unit 1001 and distributes the scalable encoded data to terminal devices such as personal computer 104, AV device 1005, tablet device 1006, and mobile phone device 1007 via the network 1003.

In this case, the distribution server selects and transmits the encoded data of the appropriate quality according to the capabilities of the terminal device, data transmission medium, etc. Even if the distribution server 1002 unnecessarily transmits high quality data, the terminal device does not necessarily obtain a high quality image and this may be the reason delays or overflows. In addition, it may unnecessarily consume bandwidth, or it may unnecessarily increase the load on the terminal. On the contrary, even if the distribution server 1002 unnecessarily transmits low quality data, it may not be possible to obtain an image with sufficient image quality at the terminal device. Therefore, the distribution server 1002 properly reads and transmits the scalable encoded data that is stored in the scalable encoded data storage unit 1001 as encoded data of proper quality corresponding to the capabilities of the terminal device, transmission medium, etc.

For example, it is assumed that the scalable encoded data storage unit 1001 stores scalable encoded data (BL + EL) 1011 encoded in a scalable manner. Scalable coded data (BL + EL) 1011 is coded data containing both a base layer and an enhancement layer, and data that can be decoded to obtain both a base layer image and an enhanced layer image.

The distribution server 1002 selects the proper level according to the capability of the terminal to which the data is to be transmitted, the transmission medium, etc., and reads the level data. For example, for a personal computer 1004 or tablet device 1006 with high processing capability, a distribution server 1002 reads high quality scalable encoded data (BL + EL) 1011 from a scalable encoded data storage unit 1001 and transmits it as it is. In contrast, for example, for an AV device 1005 or a mobile phone device 1007 with low processing capability, the distribution server 1002 extracts base layer data from scalable encoded data (BL + EL) 1011 and transmits the data as scalable encoded data (BL) 1012 which is data of the same content as scalable coded data (BL + EL) 1011, but of lower quality than scalable coded data (BL + EL) 1011.

Thus, using scalable encoded data, the amount of data can be easily adjusted. Accordingly, it is possible to suppress the occurrence of a delay or overflow, or it is possible to suppress an unnecessary increase in the load of the terminal device or the communication medium. Moreover, in scalable coded data (BL + EL) 1011, redundancy between layers is reduced. In this case, it is possible to reduce the amount of data as compared with the case in which the encoded data of each layer is used as individual data. Therefore, the storage area of the scalable encoded data storage unit can be more efficiently used.

In addition, various devices can be used as terminal devices, from a personal computer 1004 to a mobile telephone device 1007. Accordingly, the hardware characteristics of the terminal device differ depending on the device. In addition, since there are also different applications executed by the terminal, the software capabilities also vary. In addition, since the network 1003 is a communication medium, wired or wireless media such as the Internet, LAN (Local Area Network), or any communication network, including both, can be applied. Data transfer capabilities vary accordingly. In addition, they can change due to other means of data transmission, etc.

Therefore, the distribution server 1001 can communicate with the terminal as the destination of the data transfer before starting the data transfer to obtain information related to the capability of the terminal, such as the specification of the terminal or the characteristic of the application (software) executed by the terminal. , and information related to a communication medium, such as the available bandwidth of the network 1003. Then, the distribution server 1002 may select the appropriate level based on the information it receives.

In addition, layer extraction can be performed by the terminal. For example, the personal computer 1004 can decode the transmitted scalable coded data (BL + EL) 1011 and can display a base layer image or display an enhanced layer image. In addition, the personal computer 1004 can extract scalable base layer coded data (BL) 1012 from the transmitted scalable coded data (BL + EL) 1011 and can store scalable coded data (BL) 1012 or transmit scalable coded data (BL) 1012 to another device, or decode scalable encoded data (BL) 1012 and display a base layer image.

Of course, any number of scalable coded data storage units 1001, distribution servers 1002, networks 1003, and terminals can be used. In addition, although an example in which the distribution server 1002 transmits data to the terminal has been described above, the use examples are not limited to this only. The data communication system 1000 can be applied to any system as long as it is a system that selects and transmits the proper level appropriate to the capability of the terminal device, communication medium, etc., and as long as it transmits to the terminal device encoded data encoded in a scalable manner. ...

In addition, also in the communication system 1000 shown in FIG. 64 and described above, the same effect as the effect described above with reference to FIG. 56-58 can be obtained by applying the present technology in the same manner as the hierarchical coding and hierarchical decoding applications described above with reference to FIG. 56-58.

Second system

For example, scalable coding is used for transmission over multiple media, as shown in the example in FIG. 65.

In the communication system 1100 shown in FIG. 65, broadcast station 1101 transmits base layer scalable coded data (BL) 1121 via terrestrial broadcast 1111. In addition, broadcast station 1101 transmits (e.g., packets and transmits) enhanced layer scalable coded data (EL) 1122 through an arbitrary network 1112, which is a cable or wireless data network; or a data network containing both networks.

Terminal 1102 has a function of receiving terrestrial broadcast 1111, in which broadcast station 1101 transmits and receives scalable coded data (BL) 1121 base layer via terrestrial broadcast 1111. In addition, terminal 1102 further has a communication function for communicating via network 1112 and receives enhanced layer scalable encoded data (EL) 1122 transmitted over network 1112.

Terminal 1102 obtains a base layer image by decoding base layer scalable coded data (BL) 1121 received via terrestrial broadcast 1111 or stores scalable coded data (BL) 1121 or transmits scalable coded data (BL) 1121 to another device, for example, in response per user command, etc.

In addition, terminal 1102 obtains scalable coded data (BL + EL) by combining scalable base layer coded data (BL) 1121 received via terrestrial broadcast 1111 and scalable enhanced layer coded data (EL) 1122 received via network 1112, or obtains an enhanced layer image by decoding scalable coded data (BL + EL), or stores scalable coded data (BL + EL), or transmits scalable coded data (BL + EL) to another device, for example, in response to a user command, etc.

As described above, scalable encoded data can be transmitted over a communication medium that is different, for example, for each layer. Therefore, since it is possible to distribute the load, it is possible to suppress the occurrence of a delay or an overflow.

In addition, the transmission medium used for transmission can be selected for each layer depending on the situation. For example, scalable base layer coded data (BL) 1121 with a relatively large amount of data may be transmitted over a high bandwidth media, and scalable enhanced layer coded data (EL) 1122 with a relatively small amount of data may be transmitted over a narrow media. bandwidth. In addition, for example, regardless of whether the transmission medium for transmitting enhanced layer scalable encoded data (EL) 1122 must be network 1112 or terrestrial broadcast 1111, it may vary according to the available bandwidth of network 1112. Certainly the same true for data of any level.

By such control, it is possible to further suppress the increase in noise in the load during data transmission.

Undoubtedly, the number of layers is arbitrary and the number of data carriers used for transmission is also arbitrary. In addition, the number of data distribution destinations 1102 is also arbitrary. In addition, although broadcasting through the broadcast station 1101 has been described above as an example, the use examples are not limited to this only. Communication system 1100 can be applied to any system as long as it is a system that divides encoded data, which is scalable encoded into multiple blocks with a layer as a block, and transmits data over multiple lines.

In addition, also in the communication system 1100 shown in FIG. 65, the same effect as the effect described above with reference to FIG. 56-58 can be obtained by applying the present technology in the same manner as the hierarchical coding and hierarchical decoding applications described above with reference to FIG. 56-58.

Third system

In addition, for example, scalable coding is used to store encoded data, as shown in the example in FIG. 66.

In the imaging system 1200 shown in FIG. 66, the imaging apparatus 1201 performs scalable coding on the image data obtained by acquiring an image of the object 1211, and supplies the result to the scalable coded data storage 1202 as scalable coded data (BL + EL) 1221.

The scalable coded data storage 1202 stores scalable coded data (BL + EL) 1221 supplied from the image acquiring apparatus 1201 with a quality appropriate to the situation. For example, in a normal state, scalable coded data memory 1202 extracts base layer data from scalable coded data (BL + EL) 1221 and stores the data as scalable coded data (BL) 1222 base layer having a small amount of data with low quality. In contrast, for example, in a state of attention, the scalable encoded data memory 1202 stores scalable encoded data (BL + EL) 1221 having a large amount of data because it is of high quality.

Thus, scalable encoded data storage 1202 can store a high quality image only when needed. Therefore, it is possible to suppress the increase in the amount of data while suppressing the decrease in the image value by degrading the image quality. As a result, the efficiency of using the storage area can be improved.

For example, it is assumed that the imaging apparatus 1201 is an inspection chamber. When the target of the inspection (for example, a foreign body) is not reflected in the resulting image (in a normal state), the probability that the content of the resulting image will be irrelevant is high. In this case, the priority is given to reducing the amount of data, and the image data (scalable encoded data) is stored in low quality. In contrast, when the inspection target is reflected in the acquired image as an object 1211 (in the viewing state), the probability that the content of the acquired image is significant is high. In this case, the priority is given to reducing the amount of data, and the image data (scalable encoded data) is stored in low quality.

In addition, the determination of the normal state or the state of attention can be performed by analyzing the image, for example, the scalable encoded data storage 1202. In addition, the image acquiring apparatus 1201 can perform the determination and transmit the determination result to the scalable encoded data storage 1202.

In addition, the criterion of the normal state or the state of attention is arbitrary and the content of the image to be the criterion is arbitrary. Of course, conditions other than the content of the image can also be a criterion. For example, conditions may vary according to the magnitude, waveform, etc. of the recorded sound, or may change at predetermined time intervals, or may change in response to an external command such as a user command.

In addition, an example of switching between two states, the normal state and the attention state, was described above. However, the number of states is arbitrary and, if necessary, switching between three or more states can be set (for example, a normal state, a state of attention, a state of attention in an accident, etc.). Here, the maximum number of switching states depends on the number of levels of scalable encoded data

In addition, the imaging apparatus 1201 can determine the number of scalable coding levels according to the state. For example, in a normal state, the imaging apparatus 1201 may generate base layer scalable coded data (BL) 1222 having a small amount of data with low quality, and supply scalable coded data (BL) 1222 to scalable coded data memory 1202. In addition, for example, in a state of attention, the imaging apparatus 1201 may generate scalable base layer coded data (BL + EL) 1221 having a large amount of data with high quality, and supply scalable coded data (BL + EL) 1221 to a scalable coded data memory 1202. encoded data.

Although the viewing chamber has been described above as an example, the application of this imaging system 1200 is arbitrary and not limited to the viewing chamber.

In addition, also in the above-described imaging system 1200 shown in FIG. 66, the same effect as the effect described above with reference to FIG. 56-58 can be obtained by applying the present technology in the same manner as it is applied above in hierarchical coding and hierarchical decoding described above with reference to FIG. 56-58.

In addition, in the present description, an example has been presented in which various kinds of information such as the deblocking filter parameter and the adaptive offset filter parameter are multiplexed into an encoding stream and transmitted from the encoding side to the decoding side. However, the manner in which pieces of information are transmitted is not limited to these examples. For example, pieces of information can be transmitted or recorded as separate data associated with the coded bitstream without being multiplexed with the coded bitstream. Note that the term "associated" means allowing the images (which may be part of the images, such as slices or blocks) contained in the bitstream to be associated with the image information when decoded. That is, information can be transmitted along an information path that is different from an image (or bitstream) path. Alternatively, the information can be recorded on a recording medium other than the image recording medium (or bitstream) (or on a different area of the same recording medium). Additionally, information and pictures (or bitstream) can be associated with each other in any blocks, such as blocks of multiple frames, one frame, or part of a frame.

Although the preferred embodiments of the present disclosure have been described above with reference to the accompanying drawings, the present disclosure is not limited to these examples. It is obvious that a person skilled in the art to which the present disclosure belongs can suggest various variations and modifications within the technical idea described in the claims, and, naturally, it is understood that these variations and modifications fall within the technical scope of the present disclosure.

In addition, the present technology can also use the following structures.

(1) An image processing apparatus, comprising: a decoding unit forming an image by decoding a coded stream; and a filter unit performing filtering with a filter using a parameter obtained by expanding the filter parameter range for the filter so as to increase the filtering strength, with the block edge of the image generated by the decoding unit as a target.

(2) The image processing apparatus of (1), wherein the range of the filter parameter is expanded so as to increase the filtering strength of the filter in a state compatible with the existing range.

(3) The image processing apparatus according to (1) or (2), wherein the range of the filter parameter is adjusted to be continuous over the existing portion and the extended portion.

(4) The image processing apparatus of (1), wherein the filter parameter range is expanded by adding a new filter parameter.

(5) An image processing device according to any one of claims. (1) - (4), in which the filter parameter is a threshold value used in determining whether or not to filter. The filter is an unblocking filter and the filter parameter is a parameter used in determining whether an unblocking filter is required or not required, or when determining the choice of filter strength.

(7) The image processing apparatus of claim (6), wherein the range of the filter parameter is linearly expanded.

(8) The image processing apparatus according to (7), wherein the inclination angle of the linear shape is adjustable.

(9) An image processing device according to any one of claims. (1) - (6), in which the filter parameter range is extended along a curved line.

(10) An image processing device according to any one of claims. (1) - (5), in which the filter is an deblocking filter and the filter parameter is a parameter used in determining the choice of the filtering strength of the deblocking filter or used in limiting when filtering is performed with the filter.

(11) An image processing device according to any one of claims. (1) - (5), in which the filter block can perform filtering with a filter using a parameter extended by updating the table using the slope of the computational expression that expresses the filter parameter, with the block edge of the image generated by the decoding block as a target.

(12) An image processing device according to any one of claims. (1) - (5), in which the filter block performs filtering with a filter using a parameter extended by adding a filter parameter offset to the filter parameter, with the block edge of the image formed by the decoding block as a target.

(13) An image processing device according to any one of claims. (1) - (5), in which the filter parameter contains two types of parameters that use the common offset, and additionally provides a parameter calculator calculating the two types of parameters using the common offset.

(14) An image processing unit according to any one of claims. (1) - (5), in which the filter parameter contains two types of parameters and the first parameter calculator, calculating the first filter parameter using the first parameter offset, and the second parameter calculating block, calculating the second filter parameter using the first parameter offset, the difference between the first and second filter parameters and the offset of the second parameter.

(15) An image processing device according to any one of claims. (1) - (14), in which the filter is a deblocking filter.

(16) The image processing apparatus of (1), wherein the filter is an adaptive bias filter.

(17) An image processing device according to any one of claims. (1) - (16), in which the decoding unit decodes a coded stream in a processing unit having a hierarchical structure.

(18) An image processing method of an image processing apparatus, comprising the steps of: generating an image by decoding a coded stream; and performing filtering with a filter using the parameter obtained by expanding the range of the filter parameter to control the filter so as to increase the filtering strength, with the block edge of the image generated by the decoding unit as a target.

(19) An image processing apparatus, comprising: a filter unit performing filtering with a filter using a parameter obtained by expanding the range of a filter parameter to control the filter so as to increase the filtering strength, with a block edge of the locally decoded image as a target; an encoding unit generating a codestream by encoding an image using an image filtered by a filter of the filter unit; and a transmitting unit transmitting a coded stream generated by the coding unit.

(20) An image processing method of an image processing apparatus, comprising: performing filtering with a filter using a parameter obtained by expanding the range of a filter parameter to control the filter to increase the filtering strength, with a locally decoded image as a target; form a coded stream by encoding the image and using for this the image filtered by the filter; and transmitting the generated codestream.

List of reference positions

11 Image encoder

31, 31a, 31b De-blocking filter

51 Picture decoding device

62 Lossless Decoder

101 Configuration block

102 Filter execution unit

111 QP computation unit

111a QP receiving block

111b QP mean value calculator

112 Forming block β

113 Tc formation block

114 Filtration definition block

115 Filtration unit

121 β calculation block LUT_input

122 Limiting block

123 Block of formation of existing β

124 Extended β forming unit

131 LUT (lookup table)

141 Computation block β (qp)

153 Extended LUT

161 Computation unit Tc_LUT_input

162 Limiting block

163 Block of forming an existing vehicle

164 Block for forming extended Tc

171 LUT (lookup table)

181 Tc Computation Unit (qp)

193 Extended LUT

201 Filtration definition block

251 Formation block β

301 Configuration setting block

302 Filter execution unit

311 Formation block β

312 Tc formation block

321 Computation block β LUT_input

322 Formation block β

331 Computation block Tc_LUT_input

332 Processing unit for forming Tc

401 Block of setting configuration

451 Setting block

Claims (46)

1. An image processing device containing: a decoding module configured to generate a decoded image by decoding a coded stream; and a filter module configured to apply deblocking filtering to neighboring pixels adjacent to a border between blocks in a decoded image generated by a decoding unit using a filter parameter set in accordance with a range including a value greater than 9 and corresponding to a quantization parameter greater than or equal to 45 in the relationship between the quantization parameter and the filter parameter set when the strength of the deblocking filter is selected or when the filtered pixels are limited after the deblocking filter. 2. The image processing device according to claim 1, in which the filter module is configured to apply a deblocking filter using a filter parameter with a value greater than 9 and corresponding to a quantization parameter greater than or equal to 45 in the specified ratio within the specified range. 3. The image processing apparatus of claim 2, wherein the filter parameter has a value set in accordance with the quantization parameter. 4. The image processing apparatus of claim 3, wherein the quantization parameter is the average of the first quantization parameter and the second quantization parameter between the first block and the second block at the boundary between blocks. 5. The image processing apparatus of claim 4, wherein the filter module is configured to apply a deblocking filter to adjacent pixels in the decoded image using a filter parameter set in accordance with an extended range including a value greater than 9 and corresponding to a quantization parameter greater or equal to 45 in the indicated ratio. 6. The image processing device according to claim 5, in which the deblocking filter comprises: a weak filter having a first strength and a strong filter having a second strength greater than the first strength; and the filter parameter is the parameter used when choosing either one of the weak filter and the strong filter, or when limiting the filtered pixels after the strong filter. 7. The image processing device according to claim 6, in which the boundary between blocks is a transform unit boundary or a prediction unit boundary. 8. The image processing device according to claim 7, wherein the decoding module comprises: an arithmetic decoding unit, configured to perform an arithmetic decoding process on a coded stream to generate quantized data; an inverse quantization unit, configured to perform an inverse quantization process on the quantized data generated by the arithmetic decoding unit to generate transform coefficient data; and an inverse orthogonal transform module, configured to perform an inverse orthogonal transform process on transform coefficient data generated by the inverse transform module. 9. The image processing apparatus of claim 8, wherein the decoding module is configured to decode a coded stream in accordance with coding units obtained by recursively dividing the largest coding unit. 10. The image processing apparatus of claim 9, wherein the coding units are obtained by dividing the largest coding unit according to the quadtree structure. 11. The image processing apparatus of claim 10, wherein the largest encoding unit is an encoding unit at the top level of the quadtree structure. 12. The image processing device according to claim 11, in which the largest coding unit is a fixed size block below the sequence level; and the coding unit is a block of variable size. 13. A method of image processing, comprising the stages at which: form a decoded image by decoding the encoded stream; and apply deblocking filtering to neighboring pixels adjacent to the border between blocks in the decoded image generated by the decoding unit using a filter parameter set in accordance with a range including a value greater than 9 and corresponding to a quantization parameter greater than or equal to 45 in the ratio between the parameter quantization and a filter parameter set when choosing the strength of the deblocking filter or when limiting the filtered pixels after the deblocking filter. 14. The image processing method according to claim 13, in which a deblocking filter is applied using a filter parameter having a value greater than 9 and corresponding to a quantization parameter greater than or equal to 45 in the specified ratio within the specified range. 15. The image processing method of claim 14, wherein the filter parameter has a value set in accordance with the quantization parameter. 16. The image processing method of claim 15, wherein the quantization parameter is an average of the first quantization parameter and the second quantization parameter between the first block and the second block at a boundary between blocks. 17. The image processing method of claim 16, wherein a deblocking filter is applied to neighboring pixels in the decoded image using a filter parameter set in accordance with an extended range including a value greater than 9 and corresponding to a quantization parameter greater than or equal to 45 in said ratio. 18. The method of image processing according to claim 17, in which the deblocking filter comprises: a weak filter having a first strength and a strong filter having a second strength greater than the first strength; and the filter parameter is the parameter used when choosing either one of the weak filter and the strong filter, or when limiting the filtered pixels after the strong filter. 19. The method of image processing according to claim 18, in which the boundary between blocks is a transform unit boundary or a prediction unit boundary. 20. The image processing method according to claim 19, wherein at the decoding stage: performing an arithmetic decoding process on the coded stream to generate quantized data; performing an inverse quantization process on the quantized data generated in the arithmetic decoding process to generate transform coefficient data; and performing an inverse orthogonal transform process on the transform coefficient data generated in the inverse transform process. 21. The image processing method of claim 20, wherein the encoded stream is decoded in accordance with coding units obtained by recursively dividing the largest coding unit. 22. The image processing method of claim 21, wherein the coding units are obtained by dividing the largest coding unit according to the quadtree structure. 23. The image processing method of claim 22, wherein the largest encoding unit is an encoding unit at the top level of the quadtree structure. 24. The method of image processing according to claim 23, in which the largest coding unit is a fixed size block below the sequence level; and the coding unit is a block of variable size.

Images