WO2017208549A1

WO2017208549A1 - Image processing apparatus and image processing method

Info

Publication number: WO2017208549A1
Application number: PCT/JP2017/008270
Authority: WO
Inventors: 義崇森上
Original assignee: ソニー株式会社
Priority date: 2016-06-01
Filing date: 2017-03-02
Publication date: 2017-12-07

Abstract

[Problem] To set an offset value more efficiently. [Solution] An image processing apparatus is provided with: a setting unit that sets the range of candidates for an offset value to be applied to a pixel of a decoding image that has been decoded, in accordance with a bit depth of an image on the basis of a quantization parameter used when the image is quantized; and a filter processing unit that executes filter processing for applying the offset value selected from among the candidates included in the set range to the pixel of the decoding image.

Description

Image processing apparatus and image processing method

The present disclosure relates to an image processing apparatus and an image processing method.

H. For the purpose of further improving the encoding efficiency over H.264 / AVC, JVCVC (Joint Collaboration Team-Video Coding), a joint standardization organization of ITU-T and ISO / IEC, has made HEVC (High Efficiency Video Coding) The standardization of an image encoding method called “N” is underway.

HEVC provides not only single-layer coding but also scalable coding as well as existing image coding schemes such as MPEG2 and AVC (Advanced Video Coding). HEVC scalable coding technology is also referred to as SHVC (Scalable HEVC). Scalable encoding generally refers to a technique for hierarchically encoding a layer that transmits a coarse image signal and a layer that transmits a fine image signal.

The first edition of the standard specification of HEVC was announced at the beginning of 2013. However, in addition to the above-mentioned SHVC, the specification is continuously expanded from various viewpoints such as enhancement of encoding tools (for example, Non-Patent Document 3). In particular, a technique called a sample adaptive offset (SAO) filter is applied from HEVC. In this sample adaptive offset filter, coding efficiency is improved by selecting an optimum mode from among a plurality of modes based on a technique called band offset and edge offset. For example, Patent Document 1 discloses an example of an image processing apparatus to which a sample adaptive offset filter is applied.

JP2015-216627A

On the other hand, the sample adaptive offset filter tends to increase the processing load for setting the optimum mode and offset value. Therefore, there is a demand for a mechanism that can further improve performance by setting the offset value more efficiently.

Therefore, the present disclosure proposes an image processing apparatus and an image processing method that can set an offset value more efficiently.

According to the present disclosure, according to the bit depth of an image, a range of offset value candidates to be applied to pixels of a decoded decoded image is set based on a quantization parameter used when the image is quantized. And a filter processing unit that executes a filter process that applies the offset value selected from the candidates included in the set range to the pixels of the decoded image. A processing device is provided.

Further, according to the present disclosure, the processor applies an offset value to be applied to the decoded image pixel based on the quantization parameter used when the image is quantized according to the bit depth of the image. Setting a candidate range; and executing filtering processing for applying the offset value selected from the candidates included in the set range to the pixels of the decoded image. An image processing method is provided.

Further, according to the present disclosure, applied to the decoded image pixel according to the offset value applied to the image pixel that has already been subjected to the encoding process according to the bit depth of the image. A setting unit for setting a range of candidates for the offset value to be performed, and a filtering process for applying the offset value selected from the candidates included in the set range to the pixels of the decoded image An image processing apparatus is provided.

Further, according to the present disclosure, the setting unit configured to set a range of offset value candidates to be applied to the decoded image pixels based on the image feature amount according to the image bit depth; There is provided an image processing apparatus comprising: a filter processing unit that executes a filter process for applying the offset value selected from the candidates included in the set range to a pixel of a decoded image. .

Further, according to the present disclosure, the processor applies the decoded image pixel to the decoded image according to the offset value applied to the image pixel that has already been subjected to the encoding process according to the bit depth of the image. A filter process for setting a range of candidates for the offset value to be applied to the pixel and applying the offset value selected from the candidates included in the set range to the pixels of the decoded image Performing an image processing method.

According to the present disclosure, the processor sets a range of candidate offset values to be applied to the decoded pixel of the decoded image based on the feature amount of the image according to the bit depth of the image. And performing a filtering process that applies the offset value selected from the candidates included in the set range to the pixels of the decoded image. .

As described above, according to the present disclosure, it is possible to provide an image processing apparatus and an image processing method capable of further improving performance related to setting of an offset value.

Note that the above effects are not necessarily limited, and any of the effects shown in the present specification, or other effects that can be grasped from the present specification, together with or in place of the above effects. May be played.

It is explanatory drawing for demonstrating the outline | summary of an edge offset process. It is explanatory drawing for demonstrating the outline | summary of a band offset process. It is the block diagram which showed an example of the structure of the image coding apparatus which concerns on 1st Embodiment of this indication. It is the block diagram which showed an example of the structure of the loop filter which concerns on the same embodiment. It is explanatory drawing for demonstrating the flow of a series of processes of the image coding apparatus which concerns on the same embodiment. It is the flowchart which showed an example of the flow of a series of processes of the image coding apparatus which concerns on the same embodiment. It is the block diagram which showed an example of the structure of the sample adaptive offset filter which concerns on the same embodiment. It is the flowchart which showed an example of the flow of a series of processes of the sample adaptive offset filter concerning the embodiment. It is the flowchart shown about an example of the process which concerns on the determination of the applicability of the SAO process for every slice by the switching determination part which concerns on the same embodiment. It is the block diagram which showed an example of the structure of the offset determination part which concerns on a comparative example. It is the table which showed an example of the correspondence of each offset measurement part which concerns on a comparative example, and each offset value. It is the flowchart which showed an example of the flow of a series of processes of the offset determination part which concerns on a comparative example. It is the block diagram shown about an example of the structure of the offset determination part which concerns on the same embodiment. It is the table which showed an example of the correspondence of each offset measurement part and each offset value concerning the embodiment. It is the table which showed an example of the correspondence of each offset measurement part and each offset value concerning the embodiment. It is the table which showed an example of the correspondence of each offset measurement part and each offset value concerning the embodiment. It is the flowchart which showed an example of the flow of a series of processes of the offset determination part which concerns on this embodiment. It is an explanatory view for explaining an outline of an image coding device concerning a 2nd embodiment of this indication. 5 is a flowchart illustrating an example of a flow of a series of processes of an offset determination unit in the image encoding device according to the embodiment. 14 is a flowchart illustrating an example of a flow of a series of processes of an offset determination unit in an image encoding device according to Modification 2-1. 14 is a flowchart illustrating an example of a flow of a series of processes of an offset determination unit in an image encoding device according to a third embodiment of the present disclosure. 16 is a flowchart illustrating an example of a flow of a series of processes of an offset determination unit in an image encoding device according to Modification 3-1. 14 is a flowchart illustrating an example of a flow of a series of processes of an offset determination unit in an image encoding device according to a fourth embodiment of the present disclosure. 16 is a flowchart illustrating an example of a flow of a series of processes of an offset determination unit 350 in an image encoding device according to Modification 4-1. It is a block diagram which shows an example of the hardware constitutions of an encoder. It is a block diagram which shows an example of a schematic structure of a mobile telephone. It is a block diagram which shows an example of a schematic structure of a recording / reproducing apparatus. It is a block diagram which shows an example of a schematic structure of an imaging device. It is a block diagram which shows an example of a schematic structure of a video set. It is a block diagram which shows an example of a schematic structure of a video processor. It is a block diagram which shows the other example of the schematic structure of a video processor.

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

The description will be made in the following order.
1. 1. Overview of sample adaptive offset filter First embodiment 2-1. Example of overall configuration of image encoding device 2-2. Configuration example of loop filter 2-3. Flow of a series of processes 2-4. Sample adaptive offset filter 2-4-1. Configuration example of sample adaptive offset filter 2-4-2. Sample adaptive offset filter processing flow 2-5. Offset determination unit 2-5-1. Comparative example of offset determination unit 2-5-2. Configuration example of offset determination unit 2-5-3. Process flow of offset determination unit 2-6. Summary 3. Second embodiment 3-1. Outline 3-2. Processing 3-3. Modification 3-4. Summary 4. Third embodiment 4-1. Outline 4-2. Processing 4-3. Modification 4-4. Summary 5. Fourth embodiment 5-1. Outline 5-2. Processing 5-3. Modification 5-4. Summary 6. 6. Hardware configuration example Application example 7-1. Application to various products 7-2. Various implementation levels Conclusion

<< 1. Overview of sample adaptive offset filter >>
First, an outline of the sample adaptive offset filter will be described. A sample adaptive offset (hereinafter sometimes referred to as “SAO”) filter is an H.264 filter. It is a loop filter technology newly introduced in H.265 / HEVC, and is executed after deblocking filter processing. SAO is composed of two types of technologies called edge offset (EO: Edge Offset) and band offset (BO: Band Offset). The parameter is set to.

First, the outline of edge offset processing will be described. The edge offset process is a process in which an offset value is added to or subtracted from the pixel value for each pixel in accordance with the relative relationship between the pixel to be processed and two adjacent pixels adjacent to the pixel.

For example, FIG. 1 is an explanatory diagram for explaining an outline of the edge offset processing. EO_0 to EO_3 shown in FIG. 1 indicate pixel array candidates (classes) in the edge offset processing. In FIG. 1, a pixel indicated by reference numeral c indicates a pixel to be processed, and pixels indicated by reference numerals a and b indicate pixels adjacent to the pixel c to be processed. Note that which of the classes EO_0 to EO_3 is to be used can be selected for each CTU for both luminance and color difference using the encoding parameters “sao_eo_class_luma” and “sao_eo_class_chroma”.

Further, Category 1 to Category 4 classify the relationship of pixel values between the target pixel c and the adjacent pixels a and b. For example, category 1 shows a case where the pixel value of the target pixel c is smaller than any pixel value of the adjacent pixels a and b. Category 2 shows a case where the pixel value of the target pixel c is smaller than one of the adjacent pixels a and b and equal to the other. In the case of

categories

3 and 4, the pixel value is smoothed between the target pixel c and the adjacent pixels a and b by subtracting the offset from the target pixel c.

In addition, in the edge offset, in addition to the categories 1 to 4 described above, a category 0 indicating that the edge offset processing is not performed is provided.

As described above, in the edge offset processing, any class is selected from the classes EO_0 to EO_3, and the relationship between the pixel value of the target pixel c and the pixel values of the adjacent pixels a and b is category 0 according to the selected class. Specify which of category 4 is applicable. Then, the pixel series is smoothed by adding or subtracting the offset according to the corresponding category.

Next, the outline of the band offset process will be described. In the band offset processing, the gradation of the pixel value (that is, the maximum value that can be expressed according to the bit depth from 0) is divided into 32 bands, and pixels belonging to four consecutive bands are among them. The pixel value is changed (added or subtracted) according to the offset value set for each band.

For example, FIG. 2 is an explanatory diagram for explaining the outline of the band offset processing, and shows a case where the bit depth is 8 bits. H. In H.265 / HEVC, 8 bits (pixel value is 0 to 255) and 10 bits (pixel value is 0 to 1023) can be used as the bit depth of the pixel. That is, in the example in which the bit depth shown in FIG. 2 is 8 bits, the pixel values 0 to 255 are divided into 32 bands, and among the divided 0 to 31 bands, This shows a case where a band is selected.

As described above, in the sample adaptive offset filter, for each CTU, one of the edge offset processing and the band offset processing described above is selected, and an offset is applied to the corresponding pixel value according to the selected processing. Smoothing is performed by adding and subtracting.

<< 2. First Embodiment >>
<2-1. Overall Configuration Example of Image Encoding Device>
The image encoding device according to the first embodiment of the present disclosure will be described below. First, an example of the configuration of the image encoding device 1 according to the present embodiment will be described with reference to FIG. FIG. 3 is a block diagram showing an example of the configuration of the image encoding device 1 according to the first embodiment. As shown in FIG. 1, the image encoding device 1 includes a rearrangement buffer 11, a subtraction unit 13, an orthogonal transformation unit 14, a quantization unit 15, a lossless encoding unit 16, a storage buffer 17, a rate Control unit 18, inverse quantization unit 21, inverse orthogonal transform unit 22, addition unit 23, loop filter 24, frame memory 25,

selection units

26 and 27, intra prediction unit 30, and inter prediction unit 35.

The rearrangement buffer 11 rearranges images included in a series of image data. The rearrangement buffer 11 rearranges the images according to the GOP (Group of Pictures) structure related to the encoding process, and then subtracts the rearranged image data, the intra prediction unit 30, the inter prediction unit 35, and the loop. Output to the filter 24.

The subtraction unit 13 is supplied with image data input from the rearrangement buffer 11 and predicted image data input from the intra prediction unit 30 or the inter prediction unit 35 described later. The subtraction unit 13 calculates prediction error data that is a difference between the image data input from the rearrangement buffer 11 and the prediction image data, and outputs the calculated prediction error data to the orthogonal transformation unit 14.

The orthogonal transform unit 14 performs orthogonal transform on the prediction error data input from the subtraction unit 13. The orthogonal transformation performed by the orthogonal transformation part 14 may be discrete cosine transformation (Discrete Cosine Transform: DCT) or Karoonen-Labe transformation, for example. Orthogonal transformation is performed for each TU (transformation unit) formed by dividing a CU. The size of the TU is adaptively selected from 4 × 4 pixels, 8 × 8 pixels, 16 × 16 pixels, and 32 × 32 pixels. The orthogonal transform unit 14 outputs transform coefficient data acquired by the orthogonal transform process to the quantization unit 15.

The quantization unit 15 is supplied with transform coefficient data input from the orthogonal transform unit 14 and a rate control signal from the rate control unit 18 described later. The quantization unit 15 quantizes the transform coefficient data in a quantization step determined according to the rate control signal. The quantization unit 15 outputs the quantized transform coefficient data (hereinafter referred to as quantized data) to the lossless encoding unit 16 and the inverse quantization unit 21. The quantized data corresponds to an example of “bit stream”.

The lossless encoding unit 16 generates an encoded stream by performing a lossless encoding process on the quantized data input from the quantization unit 15. Further, the lossless encoding unit 16 encodes various parameters referred to by the decoder, and inserts the encoded parameters into the header area of the encoded stream. The parameters encoded by the lossless encoding unit 16 may include the above-described parameters that specify the Quad-Tree structure, and information related to intra prediction and information related to inter prediction, which will be described later. Then, the lossless encoding unit 16 outputs the generated encoded stream to the accumulation buffer 17.

The accumulation buffer 17 temporarily accumulates the encoded stream input from the lossless encoding unit 16 using a storage medium such as a semiconductor memory. Then, the accumulation buffer 17 outputs the accumulated encoded stream to a transmission unit (not shown) (for example, a communication interface or a connection interface with a peripheral device) at a rate corresponding to the bandwidth of the transmission path.

The rate control unit 18 monitors the free capacity of the accumulation buffer 17. Then, the rate control unit 18 generates a rate control signal according to the free capacity of the accumulation buffer 17 and outputs the generated rate control signal to the quantization unit 15. For example, the rate control unit 18 generates a rate control signal for reducing the bit rate of the quantized data when the free capacity of the storage buffer 17 is small. For example, when the free capacity of the accumulation buffer 17 is sufficiently large, the rate control unit 18 generates a rate control signal for increasing the bit rate of the quantized data.

The inverse quantization unit 21, the inverse orthogonal transform unit 22, and the addition unit 23 constitute a local decoder. The local decoder corresponds to an example of a “composite unit” that restores quantized data and generates decoded image data.

The inverse quantization unit 21 inversely quantizes the quantized data in the same quantization step as that used by the quantization unit 15 and restores transform coefficient data. Then, the inverse quantization unit 21 outputs the restored transform coefficient data to the inverse orthogonal transform unit 22.

The inverse orthogonal transform unit 22 restores the prediction error data by performing an inverse orthogonal transform process on the transform coefficient data input from the inverse quantization unit 21. Similar to the orthogonal transform, the inverse orthogonal transform is performed for each TU. Then, the inverse orthogonal transform unit 22 outputs the restored prediction error data to the addition unit 23.

The adder 23 adds the decoded prediction error data input from the inverse orthogonal transform unit 22 and the predicted image data input from the intra prediction unit 30 or the inter prediction unit 35, thereby obtaining decoded image data (reconstruction). Image). Then, the adder 23 outputs the generated decoded image data to the loop filter 24 and the frame memory 25.

The loop filter 24 includes a filter group such as a deblocking filter (DF), a sample adaptive offset (SAO) filter, and an adaptive loop filter (ALF) for the purpose of improving image quality. The loop filter 24 performs a filtering process on the decoded image data input from the adding unit 23 based on the original image data supplied from the rearrangement buffer 11 and outputs the decoded image data after filtering to the frame memory 25. The details of the loop filter 24 will be described later separately.

The frame memory 25 stores the decoded image data before filtering input from the adding unit 23 and the decoded image data after filtering input from the loop filter 24 using a storage medium.

The selection unit 26 reads out the decoded image data before filtering used for intra prediction from the frame memory 25, and supplies the read decoded image data to the intra prediction unit 30 as reference image data. Also, the selection unit 26 reads out the decoded image data after filtering used for inter prediction from the frame memory 25 and supplies the read out decoded image data to the inter prediction unit 35 as reference image data.

In the intra prediction mode, the selection unit 27 outputs predicted image data as a result of the intra prediction output from the intra prediction unit 30 to the subtraction unit 13 and outputs information related to the intra prediction to the lossless encoding unit 16. In addition, in the inter prediction mode, the selection unit 27 outputs prediction image data as a result of the inter prediction output from the inter prediction unit 35 to the subtraction unit 13 and outputs information related to the inter prediction to the lossless encoding unit 16. To do. The selection unit 27 switches between the intra prediction mode and the inter prediction mode according to the cost.

The intra prediction unit 30 performs an intra prediction process for each PU (prediction unit) formed by dividing a CU based on the original image data and the decoded image data. For example, the intra prediction unit 30 evaluates the prediction result of each candidate mode in the prediction mode set using a predetermined cost function. Next, the intra prediction unit 30 selects the prediction mode with the lowest cost, that is, the prediction mode with the highest compression rate, as the optimal prediction mode. Further, the intra prediction unit 30 generates predicted image data according to the optimal prediction mode. Then, the intra prediction unit 30 outputs information related to intra prediction including prediction mode information representing the selected optimal prediction mode, cost, and predicted image data to the selection unit 27.

The inter prediction unit 35 performs inter prediction processing for each PU formed by dividing the CU based on the original image data and the decoded image data. For example, the inter prediction unit 35 evaluates the prediction result of each candidate mode in the prediction mode set using a predetermined cost function. Next, the inter prediction unit 35 selects the prediction mode with the lowest cost, that is, the prediction mode with the highest compression rate, as the optimal prediction mode. Further, the inter prediction unit 35 generates predicted image data according to the optimal prediction mode. Then, the inter prediction unit 35 outputs to the selection unit 27 information, cost, and predicted image data regarding inter prediction including the prediction mode information representing the selected optimal prediction mode and motion information.

<2-2. Example of loop filter configuration>
Next, an example of the configuration of the loop filter 24 according to the present embodiment will be described with reference to FIG. FIG. 4 is a block diagram showing an example of the configuration of the loop filter 24 according to the present embodiment. As shown in FIG. 4, the loop filter 24 includes an original image holding unit 100, a deblocking filter 200, a sample adaptive offset (SAO) filter 300, and an adaptive loop filter 400.

The original image holding unit 100 is a holding unit for holding original image data supplied from the rearrangement buffer 11 to the loop filter 24. Each component in the loop filter 24 executes each filter process by appropriately referring to the original image data held in the original image holding unit 100.

Also, the decoded image data (reconstructed image) decoded by the inverse quantization unit 21, the inverse orthogonal transform unit 22, and the addition unit 23 (that is, the local decoder) is first supplied to the deblock filter 200.

The deblocking filter 200 removes block distortion of the decoded image data by appropriately performing a deblocking filter process. In this description, a detailed description of the deblocking filter process is omitted. The deblocking filter 200 outputs the filter processing result to the sample adaptive offset filter 300.

The sample adaptive offset filter 300 determines the SAO mode to be applied and the offset value to be applied to the decoded image data after the filtering by the deblocking filter 200 for each CTU.

Note that the SAO mode indicates which of the edge offset and the band offset shown in FIGS. 1 and 2 is selected. When the edge offset is selected, the SAO mode is selected from among the edge offset classes EO_0 to EO_3 shown in FIG. 1, and among the categories 0 to 4 corresponding to the class. It shows which category was selected. When the band offset is selected, the SAO mode indicates which band is selected from the bands of the band offset shown in FIG.

Then, the sample adaptive offset filter 300 performs filter processing (hereinafter, may be referred to as “SAO processing”) for each CTU on the decoded image data based on the determined mode and offset value.

In particular, the sample adaptive offset filter 300 according to the present embodiment reduces the processing load of the SAO processing based on a quantization step (QP: Quantization Parameter) when generating quantized data that is a decoding source of the decoded image data. . Note that the acquisition source is not particularly limited as long as the sample adaptive offset filter 300 can acquire the quantization step of the quantized data that is the decoding source of the decoded image data. As a specific example, the sample adaptive offset filter 300 may acquire the quantization step from the quantization unit 15. Details of the sample adaptive offset filter 300 according to this embodiment will be described later.

The sample adaptive offset filter 300 outputs the decoded image data after the SAO processing to the adaptive loop filter 400. Further, the sample adaptive offset filter 300 outputs the SAO mode and offset value determined for each CTU to the lossless encoding unit 16 as encoding parameters. Receiving this output, the lossless encoding unit 16 encodes the supplied SAO mode and offset value with respect to the generated encoded stream, and inserts it into the header area of the encoded stream.

The adaptive loop filter 400 performs an adaptive loop filter (ALF: Adaptive Loop Filter) process on the decoded image data after the SAO process supplied from the sample adaptive offset filter 300. In the adaptive loop filter 400, for example, a two-dimensional Wiener filter is used as a filter. Of course, filters other than the Wiener filter may be used.

The adaptive loop filter 400 has a plurality of filters with different tap sizes, and performs adaptive loop filter processing. The adaptive loop filter 400 outputs the filter processing result to the frame memory 25.

The example of the configuration of the loop filter 24 according to the present embodiment has been described above with reference to FIG.

<2-3. Flow of a series of processing>
Next, a flow of a series of processes of the image encoding device according to the present embodiment will be described with reference to FIGS. 5 and 6 are explanatory diagrams for explaining a flow of a series of processes of the image encoding device according to the present embodiment.

For example, FIG. 5 shows a flow of processing in which the image encoding apparatus divides the picture P1 into a plurality of blocks (CTU) and encodes the blocks. As illustrated in FIG. 5, the image encoding apparatus performs processing by dividing the picture P1 into a plurality of CTUs having a fixed block size by performing raster scan from the upper left to the lower right. That is, in the example shown in FIG. 5, the image encoding apparatus divides and processes the picture P1 in the order of blocks U11, U12,..., U1m, U21, U22, U23, and U24.

In the following description, focusing on the operation when the block U23 is a processing target, the image from when the decoded image data of the block U23 is generated and filtered by the loop filter 24 is described. A series of processing flow of the encoding apparatus will be described. In this case, the blocks U11 to U1m, U21, and U22 are processed blocks, and among the processed blocks, the blocks U13 and U22 in particular are adjacent blocks to the processing target block U23.

Next, referring to FIG. 6 together with FIGS. 3 and 4, the flow of processing will be described until decoded image data of one CTU in the picture P <b> 1 is generated and filtered by the loop filter 24. . FIG. 6 is a flowchart showing an example of a flow of a series of processes of the image encoding device according to the present embodiment. In particular, decoded image data of one CTU in the picture P1 is generated, and the loop filter 24 performs filtering. The flow of processing is shown until processing is performed. Note that in this description, attention is focused on the generation of decoded image data, and description of processing for generating and outputting an encoded stream by lossless encoding of quantized data is omitted.

(Step S11)
The rearrangement buffer 11 (see FIG. 3) rearranges the images included in the series of image data. The rearrangement buffer 11 rearranges the images according to the GOP (Group of Pictures) structure related to the encoding process, and then subtracts the rearranged image data, the intra prediction unit 30, the inter prediction unit 35, and the loop. Output to the filter 24.

The selection unit 26 reads decoded image data before filtering used for intra prediction from the frame memory 25 and supplies the read decoded image data to the intra prediction unit 30 as reference image data. Also, the selection unit 26 reads out the decoded image data after filtering used for inter prediction from the frame memory 25 and supplies the read out decoded image data to the inter prediction unit 35 as reference image data.

The intra prediction unit 30 performs an intra prediction process for each PU (prediction unit) formed by dividing a CU based on the supplied original image data and decoded image data.

At this time, when the block U23 in FIG. 5 is the target of the intra prediction process, the intra prediction unit 30 and the original image data of the block U23 and the decoded image data of the processed adjacent blocks U13 and U22 Intra prediction processing is executed based on the above.

The intra prediction unit 30 evaluates the prediction result of each candidate mode in the prediction mode set using a predetermined cost function, and selects the optimal prediction mode based on the evaluation result. Further, the intra prediction unit 30 generates predicted image data according to the optimal prediction mode. Then, the intra prediction unit 30 outputs information related to intra prediction including prediction mode information representing the selected optimal prediction mode, cost, and predicted image data to the selection unit 27.

Also, the inter prediction unit 35 performs inter prediction processing for each PU formed by dividing the CU based on the original image data and the decoded image data.

The inter prediction unit 35 evaluates the prediction result of each candidate mode in the prediction mode set using a predetermined cost function, and selects the optimum prediction mode based on the evaluation result. Further, the inter prediction unit 35 generates predicted image data according to the optimal prediction mode. Then, the inter prediction unit 35 outputs to the selection unit 27 information, cost, and predicted image data regarding inter prediction including the prediction mode information representing the selected optimal prediction mode and motion information.

(Step S12)
The subtraction unit 13 is supplied with image data input from the rearrangement buffer 11 and predicted image data input from the intra prediction unit 30 or the inter prediction unit 35. The subtraction unit 13 calculates prediction error data that is a difference between the image data input from the rearrangement buffer 11 and the prediction image data, and outputs the calculated prediction error data to the orthogonal transformation unit 14.

(Step S13)
The orthogonal transform unit 14 performs orthogonal transform on the prediction error data input from the subtraction unit 13. The orthogonal transform unit 14 outputs transform coefficient data acquired by the orthogonal transform process to the quantization unit 15.

The quantization unit 15 is supplied with transform coefficient data input from the orthogonal transform unit 14 and a rate control signal from the rate control unit 18. The quantization unit 15 quantizes the transform coefficient data in a quantization step determined according to the rate control signal. The quantization unit 15 outputs the quantized transform coefficient data (that is, quantized data) to the lossless encoding unit 16 and the inverse quantization unit 21.

(Step S14)
The inverse quantization unit 21 inversely quantizes the quantized data in the same quantization step as that used by the quantization unit 15 to restore transform coefficient data. Then, the inverse quantization unit 21 outputs the restored transform coefficient data to the inverse orthogonal transform unit 22.

(Step S15)
The adder 23 adds the decoded prediction error data input from the inverse orthogonal transform unit 22 and the predicted image data input from the intra prediction unit 30 or the inter prediction unit 35, thereby obtaining decoded image data (reconstruction). Image). Then, the adder 23 outputs the generated decoded image data to the loop filter 24 and the frame memory 25.

(Step S16)
The decoded image data output to the loop filter 24 is supplied to the deblock filter 200 (see FIG. 4). The deblocking filter 200 removes block distortion of the decoded image data by appropriately performing a deblocking filter process. The deblocking filter 200 outputs the filter processing result to the sample adaptive offset filter 300.

(Step S17)
The sample adaptive offset filter 300 determines, for each CTU, the SAO mode to be applied and the offset value to be applied to the decoded image data after filtering by the deblocking filter 200. The sample adaptive offset filter 300 performs filter processing (that is, SAO processing) on the decoded image data for each CTU based on the determined mode and offset value. That is, at the opportunity shown in step S17, SAO processing is performed on the processing target block U23 shown in FIG.

The sample adaptive offset filter 300 outputs the decoded image data after the SAO processing to the adaptive loop filter 400. Further, the sample adaptive offset filter 300 outputs the SAO mode and the offset value determined for each CTU (that is, for each LCU) to the lossless encoding unit 16 as encoding parameters. Receiving this output, the lossless encoding unit 16 encodes the supplied SAO mode and offset value with respect to the generated encoded stream, and inserts it into the header area of the encoded stream.

(Step S18)
The adaptive loop filter 400 performs an adaptive loop filter (ALF: Adaptive Loop Filter) process on the decoded image data after the SAO process supplied from the sample adaptive offset filter 300.

(Step S19)
The adaptive loop filter 400 outputs the filter processing result to the frame memory 25. As described above, the decoded image data of the block U 23 shown in FIG. 5 is generated and stored in the frame memory 25. The decoded image data of the block U23 stored in the frame memory 25 is used, for example, for intra prediction and inter prediction of a block adjacent to the block U23 (for example, the block U24) among the unprocessed blocks shown in FIG. It is done.

As described above, with reference to FIGS. 5 and 6, the flow of a series of processes of the image encoding device according to the present embodiment has been described.

<2-4. Sample adaptive offset filter>
<2-4-1. Sample Adaptive Offset Filter Configuration>
Next, details of the sample adaptive offset filter 300 according to the present embodiment will be described. First, an example of the configuration of the sample adaptive offset filter 300 according to the present embodiment will be described with reference to FIG. FIG. 7 is a block diagram showing an example of the configuration of the sample adaptive offset filter 300 according to the first embodiment.

As shown in FIG. 7, the sample adaptive offset filter 300 according to the present embodiment includes a control unit 310 and a filter processing unit 390. Control unit 310 includes an analysis unit 320 and a switching determination unit 360. The control unit 310 corresponds to an example of a “setting unit”.

The switching determination unit 360 determines whether or not the SAO process can be applied to each of the luminance and chromaticity for each slice of the decoded image data, and controls the operation of the analysis unit 320 based on the determination result.

Specifically, when the slice to be processed is either an I slice or a P slice, the switching determination unit 360 instructs the analysis unit 320 to apply the SAO process to the slice.

In addition, when the slice to be processed is a B slice, the switching determination unit 360 counts the number of CTUs to which SAO processing has been applied (hereinafter, the I slice or the P slice) immediately before the B slice. , It may be referred to as “number of modes”), to determine whether or not the SAO processing can be applied to the B slice.

As a specific example, when the number of modes in the immediately preceding slice is equal to or greater than a predetermined threshold TH11 (for example, half or more), the switching determination unit 360 applies to the analysis unit 320 the application of SAO processing to the B slice to be processed. Instruct. On the other hand, when the number of modes in the immediately preceding slice is less than the threshold value TH11, the switching determination unit 360 analyzes the analysis unit 320 so as to limit the application of the SAO process to the processing target B slice (that is, do not apply). To instruct.

Also, the switching determination unit 360 outputs the applicability of SAO processing for each slice to the lossless encoding unit 16 as a parameter at the time of encoding. Note that examples of parameters for controlling whether to apply SAO processing for each slice include “slice_sao_luma_flag” corresponding to luminance and “slice_sao_chroma_flag” corresponding to chromaticity.

The analysis unit 320 includes a statistic acquisition unit 330 and a mode determination unit 340. The analysis unit 320 is based on the original image data held in the original image holding unit 100 (see FIG. 4) and the decoded image data output from the deblocking filter 200, and the SAO mode and offset for performing SAO processing. A value is determined for each CTU (ie, for each LCU). Note that whether or not the analysis unit 320 can be applied is controlled in units of slices based on an instruction from the switching determination unit 360. Below, the detail of each structure of the analysis part 320 is demonstrated.

The statistic acquisition unit 330 calculates a statistic for specifying the SAO mode and offset based on the original image data and the decoded image data.

As a specific example, the statistic acquisition unit 330 analyzes the correspondence between the target pixel and adjacent pixels for each of the edge offset classes EO_0 to EO_3 illustrated in FIG. 1 in units of pixels in the decoded image data. The frequency of occurrence for each of categories 0 to 4 is counted. Further, at this time, the statistic acquisition unit 330 cumulatively counts the difference in pixel value between the decoded image data and the original image data for each of the counted categories. Then, the statistic acquisition unit 330 calculates the appearance frequency of each of the categories 0 to 4 counted for each of the classes EO_0 to EO_3 and the pixel value accumulated for each category as a statistic.

Also, the statistic acquisition unit 330 counts the frequency of the corresponding pixel for each of the band offset bands 0 to 31 shown in FIG. 2, and calculates the pixel value between the decoded image data and the original image data at that pixel. Accumulate the difference for each band. Then, the statistic acquisition unit 330 calculates the frequency counted for each band and the difference between the pixel values accumulated for each band as the statistic.

Statistic acquisition unit 330 outputs the statistic calculated as described above to mode determination unit 340.

The mode determination unit 340 includes an offset determination unit 350. Based on the statistic supplied from the statistic acquisition unit 330, the mode determination unit 340 calculates an offset value and a cost corresponding to the offset value for each SAO mode in CTU units in the decoded image data. 350.

Specifically, the mode determination unit 340 sends the offset value and the cost corresponding to the offset value to the offset determination unit 350 for each combination of the edge offset classes EO_0 to EO_3 and the categories 1 to 4. Let it be calculated. Similarly, the mode determination unit 340 causes the offset determination unit 350 to calculate an offset value and a cost corresponding to the offset value for each band of the band offset. The mode determination unit 340 also calculates a cost when the SAO process is not applied.

The offset determination unit 350 calculates a cost for each offset value candidate for the mode instructed by the mode determination unit 340. Then, the offset determination unit 350 compares the calculated costs to identify the offset value that can improve the coding efficiency most and the cost corresponding to the offset value. Details of the offset determination unit 350 will be described later.

The mode determination unit 340 compares the costs calculated for each mode of SAO, and based on the comparison result, the mode of SAO that can improve the coding efficiency most, and the offset value corresponding to the mode. Is specified for each CTU (ie, for each LCU). And the mode determination part 340 outputs the mode specified for every CTU and the offset value corresponding to the said mode to the filter process part 390 mentioned later. At this time, if it is determined that the SAO processing is not applied as a result of the cost comparison, the mode determination unit 340 instructs the filter processing unit 390 not to apply the SAO processing to the target CTU. .

Also, the mode determination unit 340 outputs the SAO mode and offset value specified for each CTU to the lossless encoding unit 16 as parameters at the time of encoding. Note that examples of the parameters at the time of encoding include “sao_type_idx_luma” corresponding to luminance and “sao_typ_idx_chroma” corresponding to chromaticity as parameters indicating either edge offset or band offset. Further, when edge offset is applied, “sao_eo_class_luma” corresponding to luminance and “sao_eo_class_chroma” corresponding to chromaticity can be cited as parameters indicating the class of edge offset. In addition, when a band offset is applied, “sao_band_position” can be cited as a parameter indicating the position of the band. Further, “sao_offset_abs” indicating the absolute value of the offset value and “sao_offset_sign” indicating the positive / negative of the offset value can be cited as parameters for notifying the offset value.

The filter processing unit 390 performs SAO processing on each CTU of the decoded image data based on the SAO mode and the offset value supplied from the mode determination unit 340. When the mode determination unit 340 instructs that the SAO processing is not applied, the filter processing unit 390 does not perform the SAO processing on the target CTU. Similarly, for a slice that is determined not to be subjected to SAO processing based on the determination result of the switching determination unit 360, the filter processing unit 390 should not perform SAO processing on the corresponding slice based on the determination result. Needless to say.

Then, the filter processing unit 390 outputs the decoded image data subjected to the SAO process to the adaptive loop filter 400 (see FIG. 4).

<2-4-2. Sample adaptive offset filter processing flow>
Next, an example of a flow of a series of processes of the sample adaptive offset filter according to the present embodiment will be described with reference to FIG. FIG. 8 is a flowchart illustrating an example of a flow of a series of processes of the sample adaptive offset filter according to the present embodiment.

(Step S100)
First, the switching determination unit 360 determines whether or not the SAO process can be applied to each of the luminance and chromaticity for each slice of the decoded image data, and controls the operation of the analysis unit 320 based on the determination result. The operation related to the determination of whether or not the SAO process can be applied by the switching determination unit 360 will be described later in detail.

(Step S500)
When it is determined that the SAO process is applied to the slice (step S200, YES), the analysis unit 320 performs the SAO process based on the original image data and the decoded image data (after the deblocking filter process). The SAO mode and the offset value for performing the above are determined for each CTU.

Specifically, the statistic acquisition unit 330 calculates a statistic for specifying the SAO mode and offset based on the original image data and the decoded image data, and sends the calculated statistic to the mode determination unit 340. Output.

Based on the statistic supplied from the statistic acquisition unit 330, the mode determination unit 340 provides the offset determination unit 350 with the cost of each value that can be taken as an offset value for each mode of SAO in the decoded image data. Let it be calculated.

The mode determination unit 340 compares the costs calculated for each mode of SAO, and selects the SAO mode that can improve the coding efficiency most based on the comparison result for each CTU (ie, for each LCU). To be specific. And the mode determination part 340 outputs the mode specified for every CTU and the offset value corresponding to the said mode to the filter process part 390 mentioned later. Also, the mode determination unit 340 outputs the SAO mode and offset value specified for each CTU to the lossless encoding unit 16 as encoding parameters.

(Step S600)
The filter processing unit 390 performs SAO processing on each CTU of the decoded image data based on the SAO mode and the offset value supplied from the mode determination unit 340.

(Step S700)
The above process is executed for a series of CTUs in the target slice (step S700, NO), and upon completion of the process for the series of CTUs (step S700, YES), the application of the SAO process to the target slice is completed. To do.

(Step S200)
When the SAO process is not applied to the target slice (step S200, NO), the processes shown in steps S300 to S700 are not executed for the slice.

Next, with reference to FIG. 9, an example of the process shown in step S <b> 100 in FIG. 8, that is, a process related to determination of whether or not to apply the SAO process for each slice by the switching determination unit 360 will be described. FIG. 9 is a flowchart illustrating an example of processing related to determination of whether or not to apply SAO processing for each slice by the switching determination unit according to the present embodiment.

(Steps S101 and S104)
If the slice to be processed is not a B slice, that is, if it is either an I slice or a P slice (step S101, NO), the switching determination unit 360 analyzes the application of the SAO process to the slice. (Step S104).

(Steps S101 and S102)
In addition, when the slice to be processed is a B slice (YES in step S101), the switching determination unit 360 applies the SAO process to the slice immediately before the B slice (that is, the I slice or the P slice). The number of CTUs (number of modes) is acquired (step S102).

(Steps S103 and S104)
If the acquired mode number is equal to or greater than the predetermined threshold TH11 (step S103, NO), the switching determination unit 360 instructs the analysis unit 320 to apply the SAO process to the slice (step S104).

(Steps S103 and S105)
On the other hand, when the acquired mode number is less than the predetermined threshold value TH11 (step S103, YES), the switching determination unit 360 restricts the application of the SAO process to the slice (that is, does not apply). To the analysis unit 320 (step S105).

As described above, the switching determination unit 360 determines whether the SAO process can be applied for each slice, and controls the operation of the analysis unit 320 based on the determination result.

Heretofore, an example of a series of processing flows of the sample adaptive offset filter 300 according to the present embodiment has been described with reference to FIGS. 8 and 9.

<2-5. Offset judgment unit>
Next, details of the offset determination unit 350 will be described. The offset determination unit 350 calculates the cost for each possible value as an offset value based on the statistic generated by the statistic acquisition unit 330, and compares the calculated costs to improve the coding efficiency most. Identify possible offset values.

As mentioned above, H. In H.265 / HEVC, 8 bit (pixel value is 0 to 255) and 10 bit (pixel value is 0 to 1023) can be used as the bit depth of the pixel, and an applicable offset value can be determined according to each bit depth. Candidates are different. Specifically, when the bit depth is 8 bits, the offset value can be 0 to 7, and when the bit depth is 10 bits, the offset value can be 0 to 31.

Therefore, in the conventional method, when the bit depth of the pixel is 10 bits, the number of offset value candidates is four times that in the case where the bit depth is 8 bits, so the amount of processing for cost calculation increases, It may cause an increase in circuit scale and power consumption. Therefore, in this description, in order to make the characteristics of the offset determination unit 350 according to the present embodiment easier to understand, first, an overview of the offset determination unit based on the conventional method is described as a comparative example, and then, according to the present embodiment. The offset determination unit 350 will be described.

<2-5-1. Comparative Example of Offset Determination Unit>
The offset determination unit according to the comparative example will be described with reference to FIGS. Hereinafter, when the offset determination unit according to the comparative example and the offset determination unit 350 according to the present embodiment are explicitly distinguished, the offset determination unit according to the comparative example is referred to as an “offset determination unit 350a”. May be described.

First, an example of the configuration of the offset determination unit 350a according to the comparative example will be described with reference to FIG. FIG. 10 is a block diagram illustrating an example of the configuration of the offset determination unit 350a according to the comparative example.

As illustrated in FIG. 10, the offset determination unit 350 a includes one or more offset measurement units 351 and an offset determination unit 353.

The offset measurement unit 351 is based on the statistic generated by the statistic acquisition unit 330 (for example, the appearance frequency of the pixel to be processed or the difference in pixel value between the decoded image data and the original image data). For each offset value, the cost when the offset value is applied is calculated. Note that the cost calculated at this time means, for example, each offset value in order to reduce an error (for example, quantization error) between the original image data generated in the decoded image data and bring it closer to the original image data. The amount of data required for the offset processing based on that (that is, the amount of data transmitted to the decoder side) and the like. Detailed description of the cost calculation is omitted in this specification.

In the example shown in FIG. 10, in order to calculate the cost in parallel for each of the offset values 0 to 31 when the bit depth is 10 bits, as the offset measuring unit 351, the Offset [0] measuring unit to Offset [31] measurement are performed. Is provided.

For example, the table indicated by the reference sign d40 in FIG. 11 corresponds to each offset measurement unit 351 shown in FIG. 10 (ie, Offset [0] measurement unit to Offset [31] measurement unit) and each offset value. Showing the relationship. As illustrated in FIG. 11, the Offset [0] measurement unit calculates the cost when the offset value is 0. Similarly, Offset [1] measurement unit to Offset [31] measurement unit are associated with offset values 1 to 31, respectively.

That is, in the example shown in FIG. 10, when the bit depth is 8 bits, the Offset [0] measurement unit indicated by reference numeral 351a among the Offset [0] measurement unit to Offset [31] measurement unit Offset [7] measurement part is used. Further, when the bit depth is 10 bits, all of the Offset [0] measurement unit to the Offset [31] measurement unit indicated by reference numeral 351b are used.

The offset determination unit 353 compares the costs calculated by the respective offset measurement units 351, and determines the offset value with the lowest cost from the offset value candidates based on the comparison result. Then, the offset determination unit 353 outputs the determined offset value and the cost corresponding to the offset value. Upon receiving this output, the mode determination unit 340 (see FIG. 7) compares the costs calculated for each mode of the SAO, and based on the comparison result, the SAO that can improve the coding efficiency most. The mode and the offset value corresponding to the mode are specified for each CTU (that is, for each LCU).

Next, an example of a flow of a series of processes performed by the offset determination unit 350a according to the comparative example will be described with reference to FIG. FIG. 12 is a flowchart illustrating an example of a flow of a series of processes of the offset determination unit 350a according to the comparative example.

(Steps S611 to S614)
First, the offset determination unit 350a initializes Count, which is a counter value, with 0 (step S611). Then, the cost is calculated for the offset value candidate indicated by the counter value Count. For example, when the counter value Count is 0, the offset determination unit 350a calculates the cost of the offset value “0”. At this time, in the case of the example shown in FIG. 10, the Offset [0] measurement unit among the offset measurement units 351 calculates the cost of the offset value “0”. Similarly, when the counter value Count is 1, the offset determination unit 350a calculates the cost of the offset value “1” (step S613). When the calculation of the cost is completed, the offset determination unit 350a increments the counter value Count (step S614).

As described above, the offset determination unit 350a calculates the cost at each offset value according to the bit depth of the pixel (step S612, NO). For example, when the bit depth is 8 bits, the offset determination unit 350a calculates the cost for each of the offset values “0” to “7”. When the bit depth is 10 bits, the offset determination unit 350a calculates the cost for each of the offset values “0” to “31”.

(Step S615)
When the calculation of the cost of each counter value is completed according to the bit depth (step S612, YES), the offset determination unit 350a compares the cost of each calculated offset value, and based on the comparison result, the offset with the lowest cost Determine the value.

As described above with reference to FIGS. 10 to 12, when the bit depth of the pixel is 8 bits, possible values of the offset value are 0 to 7, and the offset determination unit 350a uses the offset values 0 to 7 For each of these, a total of eight cost calculations are performed. In addition, when the bit depth is 10 bits, the offset value can be 0 to 31, so that the offset determination unit 350a performs the cost calculation a total of 32 times for each of the offset values 0 to 31. Become.

That is, when the bit depth of the pixel is 10 bits, the number of offset value candidates is four times that of the case where the bit depth is 8 bits. The amount of processing is four times that of the case. Therefore, when processing an image with a bit depth of 10 bits, as described with reference to FIG. 12, the number of cost calculations (processing amount) for each offset value is smaller than when the bit depth is 8 bits. 4 times, which may increase power consumption.

Also, when processing an image having a bit depth of 10 bits, as shown in FIG. 10, the number of offset measurement units 351 (that is, Offset [ 0] measuring unit to Offset [31] measuring unit) are required, and the circuit scale is often increased. In addition, when processing an image having a bit depth of 10 bits, a circuit for calculating the cost of offset values 8 to 31 that is additionally required, that is, Offset [8] measurement unit to Offset [31] The measurement unit is not used when the pixel bit depth is 8 bits. In other words, as long as an image with a pixel bit depth of 8 bits is processed, the Offset [8] measurement unit to the Offset [31] measurement unit can have a redundant configuration.

Therefore, when the bit depth is 10 bits, the offset determination unit 350 according to the present embodiment limits the offset value candidates from the values that can be taken as offset values (that is, 0 to 31). The offset value to be applied is specified from the candidates. With such a configuration, the offset determination unit 350 according to the present embodiment shares the processing amount for cost calculation of each offset value between the case where the bit depth is 8 bits and the case where the bit depth is 10 bits. And reduce power consumption. Therefore, details of the offset determination unit 350 according to the present embodiment will be described below.

<2-5-2. Configuration Example of Offset Determination Unit>
First, an example of the configuration of the offset determination unit 350 according to the present embodiment will be described with reference to FIG. FIG. 13 is a block diagram illustrating an example of the configuration of the offset determination unit 350 according to the first embodiment.

As shown in FIG. 13, the offset determination unit 350 according to the present embodiment includes a candidate control unit 355, and includes only an Offset [0] measurement unit to an Offset [7] measurement unit as the offset measurement unit 351. Different from the offset determination unit 350a according to the comparative example shown in FIG. Therefore, in this description, each configuration of the offset determination unit 350 according to the present embodiment will be described mainly by focusing on a different part from the offset determination unit 350a.

The candidate control unit 355 determines offset value candidates based on the bit depth of the pixels and the quantization step (QP) when generating the quantized data that is the decoding source of the decoded image data, and each of the determined candidates Each is assigned to the Offset [0] measurement unit to the Offset [7] measurement unit. Note that the acquisition source of the quantization step is not particularly limited as described above. As a specific example, the candidate control unit 355 acquires, from the quantization unit 15, the quantization step applied when the quantization unit 15 generates the quantization data that is the decoding source of the decoded image data. Also good.

Here, the tendency of the relationship between the quantization step and the offset value applied for the SAO processing will be described. Generally, when quantized data is generated by quantizing transform coefficient data (prediction error data after orthogonalization), the smaller the quantization step, the difference between the original image data and the decoded image data. Tends to be small (that is, the quantization error tends to be small). Therefore, when the quantization step is small, a relatively small value is often selected as the offset value applied for the SAO processing.

Also, when the pixel value in the image changes (for example, when the texture is finer), a larger quantization step tends to be selected. Also, the larger the quantization step, the greater the difference between the original image data and the decoded image data (that is, the quantization error tends to increase). When the quantization error is large in this way, even if the cost is calculated for each offset value candidate, the difference in the calculated cost does not change significantly between the candidates located in the vicinity.

Using the above characteristics, the candidate control unit 355 selects the offset value candidate from the values that can be taken as the offset value (that is, 0 to 31) when the bit depth of the pixel is 10 bits. Limit based on quantization step. Then, the candidate control unit 355 assigns each of the limited offset value candidates to the Offset [0] measurement unit to the Offset [7] measurement unit.

Here, an example of the correspondence between each offset value candidate and the Offset [0] measurement unit to Offset [7] measurement unit will be described with reference to FIGS. FIG. 14 to FIG. 16 are explanatory diagrams for explaining an example of a control table showing the correspondence between each offset value candidate and the Offset [0] measurement unit to Offset [7] measurement unit. 14 to 16, a constant N (N is an integer of 0 ≦ N) is a value determined according to the number of offset value candidates (in other words, the number of offset measuring units 351), and N + 1 is This corresponds to the number of offset value candidates.

For example, in the control table d51 shown in FIG. 14, the pixel bit depth is 8 bits, the pixel bit depth is 10 bits, and the quantization step is less than the first threshold TH21 (QP <TH21). Corresponding to either case.

In the control table d51 shown in FIG. 14, Offset [0] measurement unit to Offset [N] measurement unit are associated with 0 to N as offset value candidates. That is, when the candidate control unit 355 assigns an offset value candidate to each offset measurement unit 351 based on the control table d51, the Offset [i] measurement unit (i is an integer of 0 ≦ i ≦ N) I is assigned as an offset value. For example, when N = 7, as shown in FIG. 13, the offset [0] measurement unit, the Offset [1] measurement unit,... 1,..., 7 are assigned.

Next, the control table d52 shown in FIG. 15 will be described. The control table d52 corresponds to the case where the bit depth of the pixel is 10 bits and the quantization step is greater than the first threshold TH21 and less than or equal to the second threshold TH22 (TH21 <QP ≦ TH22). Needless to say, the second threshold value TH22 is larger than the first threshold value TH21.

In the control table d52 shown in FIG. 15, 0 to 2 × N are associated with Offset [0] measurement unit to Offset [N] measurement unit as offset value candidates. That is, when the candidate control unit 355 assigns an offset value candidate to each offset measurement unit 351 based on the control table d52, the Offset [i] measurement unit (i is an integer of 0 ≦ i ≦ N) 2 × i is assigned as the offset value. For example, in the case of N = 7, as shown in FIG. 15, the offset [0] measurement unit, the Offset [1] measurement unit,... 2,..., 14 are assigned.

Next, the control table d53 shown in FIG. 16 will be described. The control table d53 corresponds to the case where the bit depth of the pixel is 10 bits and the quantization step is larger than the second threshold value TH22 (TH22 <QP).

In the control table d53 shown in FIG. 14, 0 to 4 × N are associated with Offset [0] measurement unit to Offset [N] measurement unit as offset value candidates. That is, when the candidate control unit 355 assigns an offset value candidate to each offset measurement unit 351 based on the control table d52, the Offset [i] measurement unit (i is an integer of 0 ≦ i ≦ N) 4 × i is assigned as an offset value. For example, when N = 7, as shown in FIG. 16, the offset [0] measurement unit, the Offset [1] measurement unit,... 4,..., 28 are assigned.

In this way, the candidate control unit 355 selects any one of the control tables d51 to d53 based on the bit depth of the pixel and the acquired quantization step, and the Offset [0] measurement unit according to the selected control table. ~ Offset [7] Offset value candidates are assigned to each measurement unit.

Specifically, when the bit depth of the pixel is 10 bits, the candidate control unit 355 sets the offset value candidate so that the maximum value of the offset value is smaller as the quantization step is smaller. Is set to be smaller (ie, the difference between adjacent candidates). In addition, when the bit depth of the pixel is 10 bits, the candidate control unit 355 sets the maximum offset value candidate to be larger as the quantization step is larger, and the interval between adjacent candidates ( That is, it is set so that the difference between adjacent candidates) becomes larger. With the configuration as described above, the candidate control unit 355 determines the number of offset value candidates when the pixel bit depth is 10 bits as the number of offset value candidates when the pixel bit depth is 8 bits (in other words, the offset value in the case of 8 bits). Range of possible values).

Note that the values of the first threshold value TH21 and the second threshold value TH22 are determined in advance by performing an experiment in advance according to the configuration of the image encoding device and various parameters (for example, the resolution of the original image data). do it. The configuration described above is merely an example, and the number of offset measurement units 351 and the number of control tables for specifying offset value candidates may be changed as appropriate. Of course, when changing the number of offset measuring units 351 and the number of control tables, the candidates for offset values set in each control table and the threshold value for switching each control table are determined in advance experiments or the like. An appropriate value determined based on this may be set in advance.

The subsequent processing is the same as that of the offset determination unit 350a according to the comparative example described above. That is, each offset measurement unit 351 (that is, the Offset [0] measurement unit to the Offset [7] measurement unit) calculates a cost for the assigned offset value candidate and outputs the cost to the offset determination unit 353.

The example of the configuration of the offset determination unit 350 according to the present embodiment has been described above with reference to FIGS. With the configuration described above, the offset determination unit 350 according to the present embodiment can limit the processing amount of the cost calculation when the bit depth of the pixel is 10 bits to the processing amount equivalent to the case of 8 bits. It becomes. Therefore, even when the bit depth is 10 bits, the offset determination unit 350 according to the present embodiment has the same configuration as that of the 8-bit offset measurement unit 351 (that is, Offset [0] measurement unit to Offset [7] The offset value can be determined using the measurement unit. That is, according to the offset determination unit 350 according to the present embodiment, the circuit scale can be reduced as compared with the offset determination unit 350a according to the comparative example. In addition, the offset determination unit 350 according to the present embodiment can reduce power consumption as compared with the offset determination unit 350a according to the comparative example as the processing amount of the cost calculation is reduced.

The configuration of the offset determination unit 350 described above with reference to FIG. 13 is merely an example, and is not necessarily limited to the configuration illustrated in FIG. For example, the offset determination unit 350 according to the present embodiment is provided with Offset [0] measurement unit to Offset [31] measurement unit as the offset measurement unit 351, and a mode in which only part is used and a mode in which all are used. The offset determination unit 350 may be configured to be switched.

For example, when it is necessary to limit power consumption as in battery driving, as described above, only the Offset [0] measurement unit to the Offset [7] measurement unit may be used as the offset measurement unit 351. Good. Further, when the processing amount can be further improved as in the case of power supply driving, all of the Offset [0] measurement unit to the Offset [31] measurement unit are used as the offset measurement unit 351, and the comparison described above. The offset determination unit 350a according to the example may operate similarly. As described above, the offset measurement unit 351 according to the present embodiment switches to a mode in which only a part of the offset measurement units 351 is used according to a predetermined condition, such as a situation in which the image coding apparatus operates. Thus, the configuration may be such that the processing amount of the cost calculation can be reduced.

<2-5-3. Process flow of offset determination unit>
Next, an example of a flow of a series of processes of the offset determination unit 350 according to the present embodiment will be described with reference to FIG. FIG. 17 is a flowchart illustrating an example of a flow of a series of processes of the offset determination unit 350 according to the present embodiment. In FIG. 17, the constant N (N is an integer of 0 ≦ N) is a value determined according to the number of offset value candidates (in other words, the number of offset measuring units 351), as described above. N + 1 corresponds to the number of offset value candidates.

(Step S621)
First, the offset determination unit 350 initializes Count, which is a counter value, with 0 (step S611). Then, the cost is calculated for the offset value candidate indicated by the counter value Count. At this time, the offset determination unit 350 restricts offset value candidates based on the bit depth of the pixel and the quantization step (QP) when generating the quantized data that is the decoding source of the decoded image data.

(Steps S623 and S624)
Specifically, one of a case where the bit depth of the pixel is 8 bits and a case where the bit depth of the pixel is 10 bits and the quantization step is equal to or less than the first threshold TH21 (QP ≦ TH21). In such a case (step S623, NO), the offset determination unit 350 selects the table 1 corresponding to the condition. A specific example of the table 1 is a control table d51 shown in FIG.

Then, the offset determination unit 350 specifies offset value candidates indicated by the counter value Count based on the selected table 1, and calculates a cost for the specified candidates. In this case, when the counter value Count = i (i is an integer of 0 ≦ i ≦ N), i is specified as a candidate offset value.

(Step S625)
Further, when the bit depth of the pixel is 10 bits and the quantization step is larger than the first threshold value TH21 (step S624, YES), the offset determination unit 350 determines that the quantization step is the second threshold value. It is determined whether it is greater than TH22 (TH22> TH21).

(Step S626)
When the quantization step is equal to or smaller than the second threshold TH22 (step S625, NO), the offset determination unit 350 selects the table 2 corresponding to the condition. A specific example of the table 2 is a control table d52 shown in FIG.

Then, the offset determination unit 350 specifies offset value candidates indicated by the counter value Count based on the selected table 2, and calculates a cost for the specified candidates. In this case, when the counter value Count = i (i is an integer of 0 ≦ i ≦ N), 2 × i is specified as a candidate offset value.

(Step S627)
When the quantization step is larger than the second threshold TH22 (step S625, YES), the offset determination unit 350 selects the table 3 corresponding to the condition. A specific example of the table 3 is a control table d53 shown in FIG.

Then, the offset determination unit 350 identifies offset value candidates indicated by the counter value Count based on the selected table 3, and calculates a cost for the identified candidates. In this case, when the counter value Count = i (i is an integer satisfying 0 ≦ i ≦ N), 4 × i is specified as a candidate offset value.

(Step S628)
The offset determination unit 350 identifies an offset value candidate indicated by the counter value Count, and increments the counter value Count when cost calculation is completed for the identified candidate.

(Step S622)
As described above, the offset determination unit 350 specifies offset value candidates until the counter value Count exceeds a predetermined constant N, and calculates the cost for the specified candidates (NO in step S622).

(Step S629)
For each of the counter values Count from 0 to N, an offset value candidate is specified, and when cost calculation is completed for the specified candidate (YES in step S622), the offset determination unit 350 calculates each offset value candidate. Compare costs. And the offset determination part 350 determines a candidate with the lowest cost as an offset value based on the said comparison result.

In the foregoing, with reference to FIG. 17, an example of a flow of a series of processes of the offset determination unit 350 according to the present embodiment has been described.

<2-6. Summary>
As described above, in the image coding apparatus according to the present embodiment, the offset determination unit 350 generates a bit depth of a pixel and a quantization step when generating quantized data that is a decoding source of the decoded image data ( QP) and limit offset value candidates for SAO processing.

With such a configuration, the offset determination unit 350 according to the present embodiment can limit the processing amount of the cost calculation when the bit depth of the pixel is 10 bits to a processing amount equivalent to the case of 8 bits. Become. Therefore, according to the offset determination unit 350 according to the present embodiment, even when the bit depth is 10 bits, the same configuration as in the case of 8 bits is used as the configuration for calculating the cost of each candidate offset value. Thus, the offset value can be determined. That is, according to the image coding apparatus according to the present embodiment, the circuit scale of the offset determination unit 350 is reduced as compared with the case where the cost is calculated for each offset value that can be taken when the bit depth of the pixel is 10 bits. It becomes possible. In addition, the image coding apparatus according to the present embodiment can further reduce power consumption as the amount of cost calculation decreases when the bit depth of a pixel is 10 bits.

In the above description, the example of limiting the offset value candidates based on the quantization step has been described. However, the criterion for limiting the offset value candidates is not necessarily limited to the quantization step. As a specific example, the offset determination unit 350 may limit offset value candidates based on the feature amount of the original image data.

As a specific example of the feature amount of the original image data, for example, there is a variance value that serves as an index of the amount of change in the pixel value in the original image data.

For example, when the variance value of the original image data is small, the change in the pixel value is small. A specific example of the original image data having a small variance value is a flat image with little shading or a gradual change in shading. In such a case, the similarity between blocks (CTU) is high, and the difference between original image data and decoded image data tends to be small. Therefore, when the variance of the original image data is small, the offset determination unit 350 may limit the offset candidates so that the offset value becomes smaller as in the control table d51 illustrated in FIG. .

In addition, when the variance value of the original image data is large, the case where the change of the pixel value is large is shown. As a specific example of the original image data having a large variance value, an image having a finer texture can be cited. In such a case, the similarity between blocks (CTU) is low, and the difference between original image data and decoded image data tends to be large. Therefore, when the variance value of the original image data is large, the offset determination unit 350 sets the offset candidates so that the offset value becomes larger as in the control tables d52 and d53 shown in FIGS. You may restrict.

Note that the feature amount of the original image data may be calculated as the feature amount by the statistic acquisition unit 330 based on the acquired original image data, for example.

As described above, if a condition is set in advance according to the tendency of the applied offset value and the offset determination unit 350 can limit the offset value candidates according to the condition, a determination criterion for limiting the offset value candidates Is not particularly limited. Of course, as long as conditions can be set according to the tendency of the applied offset value, it goes without saying that parameters other than the variance value may be used as the feature amount of the original image data.

<< 3. Second Embodiment >>
<3-1. Overview>
Subsequently, an image encoding device according to the second embodiment will be described. In the first embodiment described above, the description has been made mainly focusing on the case where an image having a bit depth of 10 bits or less is encoded as an input image. On the other hand, in recent years, it is also assumed that an image having a bit depth exceeding 10 bits is encoded as an input image, and the range of values that can be taken by the offset value becomes wider as the bit depth increases. The number of candidates tends to increase. As a specific example, when the bit depth is 12 bits, possible values of the offset value are 0 to 124.

In view of this situation, HEVC is considering the introduction of so-called Log2OffsetScale parameters such as “log2_sao_offset_scale_luma” and “log2_sao_offset_scale_chroma”. The Log2OffsetScale parameter is a parameter for scaling (ie, enlarging or reducing) the offset value when the bit depth is 10 bits as necessary when the bit depth of the pixel of the input image exceeds 10 bits. It is possible to set in units of pictures. Here, “log2_sao_offset_scale_luma” is a Log2OffsetScale parameter for scaling the offset value with respect to luminance, and “log2_sao_offset_scale_chroma” is a Log2OffsetScale parameter for scaling the offset value with respect to color difference.

Specifically, when the bit depth of the pixel of the input image is 12 bits, any value of 0, 1, and 2 can be set as the Log2OffsetScale parameter. As a more specific example, offset value candidates when Log2OffsetScale is 0 to 2 are as follows.
・ Log2OffSetScale = 0, Offset = {0, 1, 2, 3, ..., 31}
・ Log2OffSetScale = 1, Offset = {0, 2, 4, 6, ..., 62}
-Log2OffSetScale = 2, Offset = {0, 4, 8, 16, ..., 124}

In this way, by providing the Log2OffsetScale parameter, the offset value candidates from 0 to 31 when the bit depth is 10 bits are scaled according to the situation to select the offset value when exceeding 10 bits. Can be set automatically. In addition, with the introduction of the Log2OffsetScale parameter, it is not necessary to transmit data for all offset value candidates, so that it is possible to suppress a decrease in data transmission efficiency accompanying an increase in bit depth.

As a specific example, when the peak signal-to-noise ratio (PSNR) of the input image is high (that is, when the image quality is good), the difference between the original image data and the decoded image data is It tends to be smaller. Therefore, in such a case, for example, by setting Log2OffSetScale to be smaller, the maximum value that can be taken as an offset value is limited, but the interval between offset value candidates (that is, adjacent to each other) It is possible to set so that the difference between candidates is smaller. That is, the offset value can be applied with higher resolution according to the difference between the original image data and the decoded image data.

As another example, when the PSNR of the input image is low (that is, when the image quality is poor), the difference between the original image data and the decoded image data tends to be larger. In such a case, for example, by setting Log2OffSetScale to be larger, it is possible to set the interval between offset value candidates (that is, the difference between adjacent candidates) to be larger. Become. Such control makes it possible to apply a larger value as the offset value without changing the number of offset value candidates.

Therefore, in the present embodiment, by setting the Log2OffsetScale parameter in a more preferable manner, it is possible to suppress a decrease in data transmission efficiency accompanying an increase in bit depth and to further improve the performance related to the setting of an offset value ( As a result, an example of a mechanism capable of reducing the processing load related to the setting of the offset value is proposed.

Specifically, the image encoding device according to the present embodiment, when the bit depth of the pixel of the input image exceeds 10 bits, according to the parameter (that is, QP) for quantizing the input image, Set the Log2OffsetScale parameter. For example, FIG. 18 is an explanatory diagram for describing an overview of the image coding apparatus according to the present embodiment, and illustrates an example of a relationship between a quantized value and a quantized value. In FIG. 18, the horizontal axis indicates the quantized value, and the vertical axis indicates the quantized value. A graph indicated by a broken line corresponds to before quantization, and a graph indicated by a solid line corresponds to after quantization.

As can be seen from the above-described characteristics, SAO is a mechanism for improving the difference between an original image and a restored image by an offset value. For example, a value quantized by applying an offset value is encoded. To the value of the target image (that is, the original image). On the other hand, as shown in FIG. 18, when an image is quantized, the larger the quantized value (that is, the quantization step (QP)), the larger the quantized value (that is, the data before quantization). ) And the quantized bits (ie, the quantized data) tend to be large. The image encoding apparatus according to the present embodiment is applied in the SAO processing by setting the Log2OffsetScale parameter according to the QP when the input image (original image) is quantized by using the above-described characteristics. Control the offset value.

As a more specific example, the image encoding device controls so that a larger value is set as the Log2OffsetScale parameter when the QP is equal to or greater than the threshold value. As a result, the range of offset value candidates becomes wider and the interval between adjacent candidates is controlled to be larger. Therefore, the image encoding apparatus can apply a larger value as the offset value even under a situation where the error before and after the quantization becomes larger by setting a larger value as the QP. It is possible to eliminate a larger error.

In addition, when the QP is less than the threshold value, the image coding apparatus performs control so that a smaller value is set as the Log2OffsetScale parameter. Accordingly, the range of offset value candidates is narrowed, and the interval between adjacent candidates is controlled to be smaller. Therefore, the image coding apparatus can set the interval between offset value candidates to be smaller in a situation where an error before and after quantization becomes smaller by setting a smaller value as QP. As a result, the image can be restored more precisely.

That is, the image coding apparatus according to the present embodiment adapts the Log2OffsetScale parameter (and thus the range of offset value candidates and the interval between the candidates) in units of pictures according to the characteristics (ie, QP) of the picture. To set. Therefore, the image encoding apparatus according to the present embodiment performs processing related to the setting of the offset value by setting the offset value more efficiently even in a situation where the bit depth of the pixel of the input image becomes larger. Further, it is possible to further reduce the data transmission efficiency accompanying the increase in bit depth.

The outline of the sample adaptive offset filter in the image encoding device according to the present embodiment has been described above with reference to FIG.

<3-2. Processing>
Subsequently, an example of a flow of a series of processing of the sample adaptive offset filter in the image encoding device according to the present embodiment will be described with particular attention paid to the processing of the offset determination unit 350. For example, FIG. 19 is a flowchart illustrating an example of a flow of a series of processes of the offset determination unit 350 in the image encoding device according to the present embodiment.

(Steps S711, S713, S717)
As shown in FIG. 19, first, the offset determination unit 350 obtains a quantization step (QP) when generating quantized data that is a decoding source of a target picture (decoded image data) (S711). The acquired QP is sequentially compared with each of predetermined threshold values TH30 and TH31. Note that the magnitude relationship between the thresholds TH30 and TH31 is TH30 <TH31.

(Step S715)
When the acquired QP is less than the threshold value TH30 (S713, YES), the offset determination unit 350 sets 0 as the Log2OffsetScale parameter. In this case, offset value candidates are {0, 1, 2, 3,..., 31} as described above.

(Step S719)
On the other hand, when the acquired QP is greater than or equal to the threshold TH30 (S713, NO) and less than the threshold TH31 (S717, YES), the offset determination unit 350 sets 1 as the Log2OffsetScale parameter. In this case, as described above, the offset value candidates are scaled based on the Log2OffsetScale parameter and become {0, 2, 4, 6,..., 62}.

(Step S721)
If the acquired QP is equal to or greater than the threshold TH31 (S717, NO), the offset determination unit 350 sets 2 as the Log2OffsetScale parameter. In this case, as described above, the offset value candidates are scaled based on the Log2OffsetScale parameter and become {0, 4, 8, 12,..., 124}.

It should be noted that an appropriate value determined based on a prior experiment or the like may be set in advance for the thresholds HT30 and TH31 for switching the setting of the Log2OffsetScale parameter based on QP.

As described above, with reference to FIG. 19, an example of a flow of a series of processing of the sample adaptive offset filter in the image encoding device according to the present embodiment has been described, particularly focusing on the processing of the offset determination unit 350. The offset determination unit 350 corresponds to an example of a “setting unit” that sets a range of offset value candidates.

<3-3. Modification>
Subsequently, as a modification of the present embodiment (hereinafter referred to as “Modification 2-1”), the image coding apparatus according to the present embodiment is the same as the image coding apparatus according to the first embodiment described above. Next, an example in which the number of offset value candidates is limited to the same number as when the bit depth is 8 bits will be described. Specifically, the image encoding device according to the modified example 2-1 sets the Log2OffsetScale parameter according to the QP when the input image (original image) is quantized, and sets the offset value candidate as the candidate. Set the number within a limited range.

For example, FIG. 20 is a flowchart illustrating an example of a flow of a series of processes of the offset determination unit 350 in the image encoding device according to the modified example 2-1. Note that the image encoding device according to the modification 2-1 is described assuming that, for example, one of the following settings is applied as a Log2OffsetScale parameter and an offset value candidate.
・ Log2OffSetScale = 0, Offset = {0, 1, 2, 3, 4, 5, 6, 7}
・ Log2OffSetScale = 1, Offset = {0, 2, 4, 6, 8, 10, 12, 14}
・ Log2OffSetScale = 2, Offset = {0, 4, 8, 12, 16, 20, 24, 28}
・ Log2OffSetScale = 2 、 Offset = {0, 8, 16, 24, 32, 40, 48, 56}

(Steps S731, S733, S737, S741)
As shown in FIG. 20, first, the offset determination unit 350 obtains a quantization step (QP) when generating quantized data that is a decoding source of a target picture (decoded image data) (S731). The acquired QP is sequentially compared with predetermined threshold values TH35, TH36, and TH37 (S733, S737, S741). It is assumed that the threshold values TH35, TH36, and TH37 have a magnitude relationship of TH35 <TH36 <TH37.

(Step S735)
If the acquired QP is less than the threshold TH35 (S713, YES), the offset determination unit 350 sets 0 as the Log2OffsetScale parameter. At this time, the offset determination unit 350 sets {0, 1, 2, 3, 4, 5, 6, 7} as offset value candidates.

(Step S739)
On the other hand, when the acquired QP is greater than or equal to the threshold TH35 (S733, NO) and less than the threshold TH36 (S737, YES), the offset determination unit 350 sets 1 as the Log2OffsetScale parameter. At this time, the offset determination unit 350 sets {0, 2, 4, 6, 8, 10, 12, 14} as offset value candidates.

(Step S743)
When the acquired QP is equal to or greater than the threshold TH36 (S735, NO) and is less than the threshold TH37 (S741, YES), the offset determination unit 350 sets 2 as the Log2OffsetScale parameter. At this time, the offset determination unit 350 sets {0, 4, 8, 12, 16, 20, 24, 28} as offset value candidates.

(Step S745)
When the acquired QP is equal to or greater than the threshold TH37 (S741, NO), the offset determination unit 350 sets 2 as the Log2OffsetScale parameter. At this time, the offset determination unit 350 sets {0, 8, 16, 24, 32, 40, 48, 56} as offset value candidates.

It should be noted that appropriate values determined based on prior experiments or the like may be set in advance in the thresholds HT35, TH36, and TH37 for setting the Log2OffsetScale parameter and switching the offset value candidates based on the QP.

Through the control as described above, the image coding apparatus according to the modified example 2-1 has the feature (that is, QP) of the picture in units of pictures as in the image coding apparatus according to the second embodiment described above. Accordingly, it is possible to adaptively set the Log2OffsetScale parameter (and thus the range of offset value candidates and the interval between the candidates). In addition, the image encoding device according to the modified example 2-1 uses the same configuration as the 8-bit configuration as the configuration for calculating the cost of each candidate offset value even when the bit depth exceeds 10 bits. Thus, the offset value can be determined. That is, according to the image encoding device according to the modified example 2-1, the circuit scale of the offset determination unit 350 is reduced and the offset value is reduced as in the image encoding device according to the first embodiment described above. It becomes possible to reduce the processing amount concerning each cost calculation.

In the image encoding device according to the modified example 2-1, the maximum number of offset candidates is 56, which is different from the image encoding device according to the second embodiment described above. This is because the ratio at which a value greater than 56 is selected as the offset value tends to be small. Specifically, as the QP increases, the error between the original image and the restored image increases, but the coefficient (lambda) used in the cost calculation for selecting the offset value to be applied tends to increase. is there. Therefore, it is known from experimental data that the ratio of selecting a value larger than 56 as the offset value is small. Therefore, the maximum value of offset value candidates is set to 56 in the image encoding device according to Modification 2-1. Of course, the setting shown above is only an example, and it is needless to say that the setting may be appropriately changed according to the system characteristics, use case, and the like.

<3-4. Summary>
As described above, in the image encoding device according to the present embodiment, when the bit depth of the pixel of the input image exceeds 10 bits, the input image is quantized according to the parameter (that is, QP) for quantizing the input image. Set the Log2OffsetScale parameter in units of pictures. Thereby, for example, when the QP is larger than the threshold value, the range of candidate offset values is wider, and the interval between adjacent candidates is controlled to be larger. Therefore, even under a situation where the error before and after the quantization becomes larger, a larger value can be applied as the offset value, and the larger error can be eliminated. In addition, when the QP is smaller than the threshold value, the range of offset value candidates is narrowed, and the interval between adjacent candidates is controlled to be smaller. For this reason, in a situation where the error before and after quantization becomes smaller, the interval between offset value candidates can be made smaller, and as a result, the image can be restored more precisely.

As described above, the image coding apparatus according to the present embodiment can perform the Log2OffsetScale parameter (as a result, the range of offset value candidates and the interval between the candidates in accordance with the feature (ie, QP) of the picture in units of pictures. ) Is set adaptively. In particular, in an encoder or camcorder, there is a case where control is performed such that an encoded stream (hereinafter also referred to as “bit”) is included in a bit rate at the time of data transfer by changing QP. Even in such a case, according to the image coding apparatus according to the present embodiment, it is possible to expect an improvement in coding efficiency in units of pictures. That is, the image encoding apparatus according to the present embodiment performs processing related to the setting of the offset value by setting the offset value more efficiently even in a situation where the bit depth of the pixel of the input image becomes larger. Further, it is possible to further reduce the data transmission efficiency accompanying the increase in bit depth.

<< 4. Third Embodiment >>
<4-1. Overview>
Next, an image encoding device according to the third embodiment will be described. In the second embodiment described above, the Log2OffsetScale parameter is set in units of pictures in accordance with the parameter (QP) for quantizing the input image. On the other hand, the encoding apparatus according to the present embodiment sets the Log2OffsetScale parameter and offset value candidates for each picture in accordance with the result of applying the offset value to the encoded picture (frequency of using the offset value).

For example, when the correlation between adjacent pictures is high, such as when similar scenes continue, the difference between pictures tends to be small even if the offset value is increased. That is, in such a case, the tendency of the offset value applied to a certain picture tends to be similar to the tendency of the offset value applied to the picture encoded immediately before. Therefore, the image coding apparatus according to the present embodiment uses such characteristics and adaptively switches the Log2OffsetScale parameter and offset value candidates in units of pictures according to the frequency of use of the offset value for the encoded picture. .

In this description, as in the case of the image encoding device according to Modification Example 2-1, focusing on the case of switching between the Log2OffsetScale parameter and the offset value candidate, the mode (Mode) is set as follows. It will be explained as a thing.
・ Mode = 0, Log2OffSetScale = 0, Offset = {0, 1, 2, 3, 4, 5, 6, 7}
・ Mode = 1, Log2OffSetScale = 1, Offset = {0, 2, 4, 6, 8, 10, 12, 14}
・ Mode = 2, Log2OffSetScale = 2, Offset = {0, 4, 8, 12, 16, 20, 24, 28}
・ Mode = 3, Log2OffSetScale = 2, Offset = {0, 8, 16, 24, 32, 40, 48, 56}

As a specific example, in a picture encoded immediately before, attention is paid to a case where a lower offset value (for example, an offset value smaller than a threshold) is frequently applied among offset value candidates set based on a certain mode. To do. In this case, the image encoding apparatus according to the present embodiment is one mode lower than the mode corresponding to the immediately encoded picture as the mode for setting the offset value candidate of the next encoded picture. Set the mode. More specifically, when Mode = 1 is set in the picture encoded immediately before, the image encoding apparatus sets Mode = 0 for the next picture to be encoded.

As another example, in a picture encoded immediately before, a larger offset value (for example, an offset value larger than a threshold value) is frequently applied among offset value candidates set based on a certain mode. Pay attention. In this case, the image encoding apparatus according to the present embodiment is one level higher than the mode corresponding to the immediately encoded picture as a mode for setting a candidate offset value for the next encoded picture. Set the mode. More specifically, when Mode = 1 is set in the picture encoded immediately before, the image encoding apparatus sets Mode = 2 for the picture to be encoded next.

Note that the above description focuses on the case where the Log2OffsetScale parameter and the offset value candidate are switched in the same manner as in the image encoding device according to the modified example 2-1, depending on the application result of the offset value to the encoded picture. However, it is not necessarily limited to the same control. As a specific example, the Log2OffsetScale parameter may be switched in the same manner as in the image encoding device (see FIG. 19) according to the second embodiment described above, depending on the application result of the offset value to the encoded picture.

Through the control as described above, the image coding apparatus according to the present embodiment can efficiently set the offset value in a situation where the correlation between adjacent pictures is high, such as when similar scenes continue. It is possible to control the range of offset value candidates (that is, to set offset value candidates) so as to be set (that is, the frequency of use of each offset value candidate is further improved).

The outline of the sample adaptive offset filter in the image encoding device according to the present embodiment has been described above.

<4-2. Processing>
Subsequently, an example of a flow of a series of processing of the sample adaptive offset filter in the image encoding device according to the present embodiment will be described with particular attention paid to the processing of the offset determination unit 350. For example, FIG. 21 is a flowchart illustrating an example of a flow of a series of processes of the offset determination unit 350 in the image encoding device according to the present embodiment.

(Steps S751, S753, S757)
As shown in FIG. 21, first, the offset determination unit 350 acquires the application result of the offset value for the encoded picture (S751), and evaluates the application frequency of each offset value (S753, S757).

(Step S755)
For example, attention is paid to a case where the frequency of applying an offset value smaller than a predetermined threshold is higher (S753, YES). In this case, the offset determination unit 350 sets a mode one level lower than the mode corresponding to the picture encoded immediately before as a mode for setting a candidate offset value of a picture to be encoded next.

(Step S759)
As another example, when the application frequency of the offset value smaller than the predetermined threshold is lower (S753, NO) and the application frequency of the offset value larger than the predetermined threshold is higher (S757, YES). Pay attention. In this case, the offset determination unit 350 sets a mode that is one step higher than the mode corresponding to the picture encoded immediately before, as a mode for setting a candidate offset value of the picture to be encoded next.

In addition, the threshold value used for each determination of step S753 and S757 does not necessarily need to be the same value. Under such circumstances, for example, the application frequency of the offset value smaller than the predetermined threshold is lower (S753, NO), and the application frequency of the offset value larger than the predetermined threshold is also low (S757, NO). ) Can be envisaged. In such a case, the offset determination unit 350 may maintain the mode corresponding to the picture encoded immediately before and apply the same to the picture to be encoded next.

In addition, for the threshold values used for the determinations in steps S753 and S757, appropriate values determined based on prior experiments or the like may be set in advance for each mode.

As described above, with reference to FIG. 21, an example of a flow of a series of processing of the sample adaptive offset filter in the image encoding device according to the present embodiment has been described, particularly focusing on the processing of the offset determination unit 350.

<4-3. Modification>
Subsequently, as a modification of the present embodiment (hereinafter referred to as “Modification 3-1”), the image coding apparatus according to the present embodiment and the image coding apparatus according to the second embodiment described above are included. An example of control when combined will be described. Specifically, the image encoding device according to the modified example 3-1 quantizes the input image in the same manner as the encoding device according to the second embodiment described above (for example, FIGS. 19 and 20). The Log2OffsetScale parameter and offset value candidates are set according to the parameters (ie, QP). On the other hand, the image coding apparatus according to the modified example 3-1 is the second implementation in that the threshold for evaluating the QP is dynamically controlled according to the application result of the offset value to the encoded picture. Different from form. Therefore, the image encoding device according to the modified example 3-1 will be described by focusing attention on differences from the image encoding device according to the second embodiment.

For example, FIG. 22 is a flowchart illustrating an example of a flow of a series of processes of the offset determination unit 350 in the image encoding device according to the modified example 3-1, for controlling a threshold for evaluating the QP. An example of processing is shown.

(Steps S761, S763, S767)
As shown in FIG. 22, first, the offset determination unit 350 acquires the application result of the offset value to the encoded picture (S761), and evaluates the application frequency of each offset value (S763, S767).

(Step S765)
For example, attention is paid to a case where the frequency of applying an offset value smaller than a predetermined threshold is higher (S753, YES). In this case, the offset determination unit 350 adds the offset to at least a part (particularly, a threshold having a smaller value) of the thresholds for evaluating the QP, so that the threshold becomes a larger value. To control. As a more specific example, in the example shown in FIG. 19, the offset determination unit 350 may add an offset to each of the threshold values TH30 to TH37. Such control facilitates selection of a setting in which the range of offset value candidates is narrow and the offset value candidate value is smaller (that is, a setting in which the value of the Log2OffsetScale parameter is smaller).

(Step S769)
As another example, when the application frequency of the offset value smaller than the predetermined threshold is lower (S763, NO) and the application frequency of the offset value larger than the predetermined threshold is higher (S767, YES). Pay attention. In this case, the offset determination unit 350 subtracts the offset from at least a part (particularly, a threshold having a larger value) of the thresholds for evaluating the QP so that the threshold becomes a smaller value. Control. As a more specific example, in the example shown in FIG. 19, the offset determination unit 350 may subtract the offset from each of the threshold values TH30 to TH37. Such control makes it easy to select a setting in which the range of offset value candidates is wide and a larger value can be selected as the offset value (that is, a setting in which the value of the Log2OffsetScale parameter is larger).

In addition, the threshold value used for each determination of step S763 and S767 does not necessarily need to be the same value. Under such circumstances, for example, when the application frequency of the offset value smaller than the predetermined threshold is lower (S763, NO), and the application frequency of the offset value larger than the predetermined threshold is low (S767, NO). ) Can be envisaged. In such a case, the offset determination unit 350 may not control the threshold value for evaluating the QP.

For the threshold values used for the determinations in steps S763 and S767, an appropriate value determined based on a prior experiment or the like is set in advance for each offset value candidate setting (in other words, for each mode described above). Just keep it.

Through the control as described above, the image encoding device according to the modified example 3-1 can set the offset value efficiently (that is, the offset value in a situation where the correlation between adjacent pictures is high). Log2OffsetScale parameter and offset value candidates can be set (so that the frequency of use of each candidate is improved). Further, according to the image encoding device according to the modified example 3-1, since the Log2OffsetScale parameter is set according to the QP, even when a prediction error shifts due to a rapid variation of the QP, the prediction error It becomes possible to mitigate the influence accompanying the deviation.

<4-4. Summary>
As described above, the image coding apparatus according to the present embodiment determines the Log2OffsetScale parameter and the offset value candidates for each picture according to the application result of the offset value to the encoded picture (frequency of use of the offset value). Set. As a result, the image encoding apparatus according to the present embodiment can set the offset value efficiently in a situation where the correlation between adjacent pictures is high (that is, the frequency of use of each candidate offset value). It is possible to set Log2OffsetScale parameter and offset value candidates.

In particular, in encoders and camcorders, the stream (bit) after encoding is transferred by changing the QP in accordance with the accuracy of various predictions such as intra prediction and inter prediction (in other words, whether or not the prediction is successful). There is a case where control is performed so as to be within the bit rate of the hour. At this time, in a scene that is difficult to predict (in other words, a scene having low correlation between pictures or low correlation in the screen), the difference between the original image and the restored image becomes larger, and the target bit There may be more bits that do not fit in the rate. Even in such a case, according to the image coding apparatus according to the present embodiment, Log2OffsetScale is adaptively adapted in a more preferable manner according to the characteristics of the scene (for example, whether or not the scene is likely to generate bits). Parameters and offset value candidates can be set.

<< 5. Fourth Embodiment >>
<5-1. Overview>
Subsequently, an image encoding device according to the fourth embodiment will be described. In the above-described image encoding device according to the third embodiment, the Log2OffsetScale parameter and offset value candidates are set for each picture in accordance with the application result of the offset value to the encoded picture. On the other hand, the image coding apparatus according to the present embodiment sets Log2OffsetScale parameter and offset value candidates in units of pictures according to the feature amount of the target input image (picture).

For example, an image including a complex texture or a noisy image has a large change in the screen (in a picture), so the accuracy of various predictions such as intra prediction and inter prediction is low. There is a tendency that the difference between them also becomes larger (in other words, bits tend to occur). Therefore, in such a case, for example, it is desirable to scale the offset value by setting a larger value as the Log2OffsetScale parameter so that a larger value can be set as the offset value.

On the other hand, when the change in the screen (in the picture) is small, the accuracy of various predictions such as intra prediction and inter prediction is high, and the difference between the original image and the restored image tends to be larger (in other words, Then, it tends to be hard to generate a bit). Therefore, in such a case, for example, by setting a smaller value as the Log2OffsetScale parameter, the maximum value that can be taken as an offset value is limited, but the interval between offset value candidates is made smaller. It tends to be desirable to set to.

Therefore, the image encoding apparatus according to the present embodiment uses the characteristics as described above, and the Log2OffsetScale parameter and offset value candidates according to the feature amount (for example, variance) of the target input image (picture). Are switched adaptively in units of pictures.

As a specific example, attention is paid to the case where the variance of the input image is evaluated as the feature amount of the input image. An image including a complex texture, a noisy image, or the like has a large change in the image (in the picture) (that is, a low correlation in the screen) and tends to have a higher variance. On the other hand, an image having a small change in the screen (in the picture) (that is, an image having a high correlation in the screen) tends to have a lower variance.

For this reason, the image coding apparatus according to the present embodiment regards an image with higher variance (for example, an image with variance equal to or greater than a threshold value), for example, as having low correlation within the screen and low accuracy of various predictions. Set a larger value for the Log2OffsetScale parameter. As another example, the image coding apparatus according to the present embodiment, for an image with lower variance (for example, an image with variance less than a threshold), has high correlation in the screen, for example, and accuracy of various predictions. Is assumed to be high, and a smaller value is set as the Log2OffsetScale parameter.

In the above-described example, the case where “distribution” is used as the feature amount of the image has been described. However, the parameter used as the feature amount is not necessarily limited to “distribution”. As a specific example, “variance”, “Hadamard”, “DCT”, “Tv norm”, and the like may be used as image feature amounts.

Through the control as described above, the image coding apparatus according to the present embodiment can adjust the characteristics of the target input image (picture) (for example, whether or not the difference between the original image and the restored image becomes larger). Accordingly, the Log2OffsetScale parameter and offset value candidates can be adaptively switched on a picture-by-picture basis. That is, according to the image coding apparatus according to the present embodiment, the offset value is efficiently set according to the characteristics of the target input image (that is, the frequency of use of each candidate offset value is more As a result, offset value candidates can be set.

<5-2. Processing>
Subsequently, an example of a flow of a series of processing of the sample adaptive offset filter in the image encoding device according to the present embodiment will be described with particular attention paid to the processing of the offset determination unit 350. For example, FIG. 23 is a flowchart illustrating an example of a flow of a series of processes of the offset determination unit 350 in the image encoding device according to the present embodiment.

Note that, in the present description, as in the case of the image encoding device according to Modification Example 2-1, one of the following settings is applied as a Log2OffsetScale parameter and offset value candidate.
・ Log2OffSetScale = 0, Offset = {0, 1, 2, 3, 4, 5, 6, 7}
・ Log2OffSetScale = 1, Offset = {0, 2, 4, 6, 8, 10, 12, 14}
・ Log2OffSetScale = 2, Offset = {0, 4, 8, 12, 16, 20, 24, 28}
・ Log2OffSetScale = 2 、 Offset = {0, 8, 16, 24, 32, 40, 48, 56}

(Steps S771, S773, S777, S781)
As shown in FIG. 23, first, the offset determination unit 350 acquires the feature amount of the target picture (S751), and sequentially compares the acquired feature amount with each of predetermined threshold values TH40, TH41, and TH42 ( S773, S777, S781). It is assumed that the threshold values TH40, TH41, and TH42 have a magnitude relationship of TH40 <TH41 <TH42.

(Step S775)
When the acquired feature amount is less than the threshold value TH40 (S773, YES), the offset determination unit 350 sets 0 as the Log2OffsetScale parameter. At this time, the offset determination unit 350 sets {0, 1, 2, 3, 4, 5, 6, 7} as offset value candidates.

(Step S779)
On the other hand, when the acquired feature amount is equal to or greater than the threshold value TH40 (S773, NO) and less than the threshold value TH41 (S777, YES), the offset determination unit 350 sets 1 as the Log2OffsetScale parameter. At this time, the offset determination unit 350 sets {0, 2, 4, 6, 8, 10, 12, 14} as offset value candidates.

(Step S783)
When the acquired feature amount is equal to or greater than the threshold value TH41 (S775, NO) and less than the threshold value TH42 (S781, YES), the offset determination unit 350 sets 2 as the Log2OffsetScale parameter. At this time, the offset determination unit 350 sets {0, 4, 8, 12, 16, 20, 24, 28} as offset value candidates.

(Step S785)
When the acquired feature amount is equal to or greater than the threshold TH42 (S781, NO), the offset determination unit 350 sets 2 as the Log2OffsetScale parameter. At this time, the offset determination unit 350 sets {0, 8, 16, 24, 32, 40, 48, 56} as offset value candidates.

It should be noted that appropriate values determined based on prior experiments are set in advance in the thresholds HT40, TH41, and TH42 for setting the Log2OffsetScale parameter and switching the offset value candidates based on the feature amount of the image (picture). Just keep it.

In the above, with reference to FIG. 23, an example of the flow of a series of processing of the sample adaptive offset filter in the image encoding device according to the present embodiment has been described, particularly focusing on the processing of the offset determination unit 350.

<5-3. Modification>
Subsequently, as a modification of the present embodiment (hereinafter referred to as “Modification 4-1”), the image coding apparatus according to the present embodiment and the image coding apparatus according to the second embodiment described above are included. An example of control when combined will be described. Specifically, the image encoding device according to the modified example 4-1 is for quantizing an input image in the same manner as the encoding device according to the second embodiment described above (for example, FIGS. 19 and 20). The Log2OffsetScale parameter and offset value candidates are set according to the parameters (ie, QP). On the other hand, the image coding apparatus according to the modified example 4-1 has the second feature in that the threshold for evaluating the QP is dynamically controlled according to the feature amount of the target input image (picture). Different from the embodiment. Therefore, the image encoding device according to the modified example 4-1 will be described by focusing attention on differences from the image encoding device according to the second embodiment.

For example, FIG. 24 is a flowchart illustrating an example of a flow of a series of processes of the offset determination unit 350 in the image encoding device according to the modified example 4-1, for controlling a threshold for evaluating the QP. An example of processing is shown.

(Steps S791, S793, S797)
As shown in FIG. 24, first, the offset determination unit 350 acquires a feature amount of a target picture (S791), and sequentially compares the acquired feature amount with predetermined threshold values TH45 and TH46, respectively (S793, S797). ).

(Step S795)
For example, attention is paid to a case where the acquired feature amount is less than the threshold value TH45 (S793, YES). This example corresponds to a case where the correlation of the target picture is high within the screen and the difference between the original image and the restored image is smaller. In this case, the offset determination unit 350 adds the offset to at least a part (particularly, a threshold having a smaller value) of the thresholds for evaluating the QP, so that the threshold becomes a larger value. To control. As a more specific example, in the example shown in FIG. 19, the offset determination unit 350 may add an offset to each of the threshold values TH30 to TH37. Such control facilitates selection of a setting in which the range of offset value candidates is narrow and the offset value candidate value is smaller (that is, a setting in which the value of the Log2OffsetScale parameter is smaller).

(Step S769)
As another example, attention is focused on a case where the acquired feature amount is equal to or greater than the threshold value TH45 (S793, NO) and the feature amount exceeds the threshold value TH46 (S797, YES). This example corresponds to a case where the correlation of the target picture in the screen is low and the difference between the original image and the restored image becomes larger. In this case, the offset determination unit 350 subtracts the offset from at least a part (particularly, a threshold having a larger value) of the thresholds for evaluating the QP so that the threshold becomes a smaller value. Control. As a more specific example, in the example shown in FIG. 19, the offset determination unit 350 may subtract the offset from each of the threshold values TH30 to TH37. Such control makes it easy to select a setting in which the range of offset value candidates is wide and a larger value can be selected as the offset value (that is, a setting in which the value of the Log2OffsetScale parameter is larger).

When the acquired feature value is equal to or greater than the threshold value TH45 (S793, NO) and the feature value is equal to or less than the threshold value TH46 (S797, NO), the offset determination unit 350 uses the threshold value for evaluating the QP. Need not be controlled.

In addition, for the threshold values TH45 and TH46 used for the determinations in steps S793 and S797, appropriate values determined based on prior experiments or the like may be set in advance.

Through the control as described above, the image coding apparatus according to the modified example 4-1 can set the offset value efficiently according to the characteristics of the target input image (that is, each offset value candidate). It is possible to set offset value candidates so that the frequency of use is further improved. Further, according to the image encoding device according to the modified example 4-1, since the Log2OffsetScale parameter is set according to the QP, even when the prediction error is deviated due to a rapid fluctuation of the QP, the prediction error It becomes possible to mitigate the influence accompanying the deviation.

<5-4. Summary>
As described above, the image coding apparatus according to the present embodiment sets the Log2OffsetScale parameter and offset value candidates in units of pictures according to the feature amount of the target input image (picture). As a result, the image encoding apparatus according to the present embodiment can more efficiently set the offset value according to the characteristics of the target input image (that is, the frequency of use of each candidate offset value is further improved). As a result, it is possible to set offset value candidates. That is, according to the image encoding device according to the present embodiment, the Log2OffsetScale parameter and the offset value are adaptively adapted in a more preferable manner according to the scene characteristics (for example, whether the scene is likely to generate bits). Candidates can be set.

<< 6. Hardware configuration example >>
The above-described embodiments may be realized using any of software, hardware, and a combination of software and hardware. When the image encoding device 10 uses software, a program constituting the software is stored in advance in a storage medium (non-transitory media) provided inside or outside the device, for example. Each program is read into a RAM (Random Access Memory) at the time of execution and executed by a processor such as a CPU (Central Processing Unit).

FIG. 25 is a block diagram illustrating an example of a hardware configuration of an encoder to which the above-described embodiment can be applied. Referring to FIG. 25, the encoder 800 includes a system bus 810, an image processing chip 820, and an off-chip memory 890. The image processing chip 820 includes n (n is 1 or more) processing circuits 830-1, 830-2,..., 830-n, a reference buffer 840, a system bus interface 850, and a local bus interface 860.

The system bus 810 provides a communication path between the image processing chip 820 and an external module (for example, a central control function, an application function, a communication interface, or a user interface). The processing circuits 830-1, 830-2,..., 830-n are connected to the system bus 810 via the system bus interface 850 and to the off-chip memory 890 via the local bus interface 860. The processing circuits 830-1, 830-2,..., 830-n can also access a reference buffer 840 that may correspond to an on-chip memory (eg, SRAM). The off-chip memory 890 may be a frame memory that stores image data processed by the image processing chip 820, for example.

As an example, the processing circuit 830-1 is the intra prediction unit 30, the processing circuit 830-2 is the inter prediction unit 35, the other processing circuits are the orthogonal transform unit 14, and the other processing circuit is a lossless code. It can correspond to the conversion unit 16. Another processing circuit may correspond to the loop filter 24. In addition, each of the deblock filter 200, the sample adaptive offset filter 300, and the adaptive loop filter 400 constituting the loop filter 24 may be configured as a separate processing circuit. Note that these processing circuits may be formed not on the same image processing chip 820 but on separate chips.

<< 7. Application example >>
<7-1. Application to various products>
In the above-described embodiment, a transmission device that transmits an encoded video stream using a satellite line, a cable TV line, the Internet, a cellular communication network, or the like, The present invention can be applied to various electronic devices such as a recording device for recording on a medium. Hereinafter, three application examples will be described.

(1) First Application Example FIG. 26 illustrates an example of a schematic configuration of a mobile phone to which the above-described embodiment is applied. A cellular phone 920 includes an antenna 921, a communication unit 922, an audio codec 923, a speaker 924, a microphone 925, a camera unit 926, an image processing unit 927, a demultiplexing unit 928, a recording / reproducing unit 929, a display unit 930, a control unit 931, an operation Part 932, sensor part 933, bus 934, and battery 935.

The antenna 921 is connected to the communication unit 922. The speaker 924 and the microphone 925 are connected to the audio codec 923. The operation unit 932 is connected to the control unit 931. The bus 934 connects the communication unit 922, the audio codec 923, the camera unit 926, the image processing unit 927, the demultiplexing unit 928, the recording / reproducing unit 929, the display unit 930, the control unit 931, and the sensor unit 933 to each other.

The mobile phone 920 has various operation modes including a voice call mode, a data communication mode, a shooting mode, and a videophone mode, and is used for sending and receiving voice signals, sending and receiving e-mail or image data, taking images, and recording data. Perform the action.

In the voice call mode, the analog voice signal generated by the microphone 925 is supplied to the voice codec 923. The audio codec 923 converts an analog audio signal into audio data, A / D converts the compressed audio data, and compresses it. Then, the audio codec 923 outputs the compressed audio data to the communication unit 922. The communication unit 922 encodes and modulates the audio data and generates a transmission signal. Then, the communication unit 922 transmits the generated transmission signal to a base station (not shown) via the antenna 921. In addition, the communication unit 922 amplifies a radio signal received via the antenna 921 and performs frequency conversion to acquire a received signal. Then, the communication unit 922 demodulates and decodes the received signal to generate audio data, and outputs the generated audio data to the audio codec 923. The audio codec 923 expands the audio data and performs D / A conversion to generate an analog audio signal. Then, the audio codec 923 supplies the generated audio signal to the speaker 924 to output audio.

Further, in the data communication mode, for example, the control unit 931 generates character data constituting the e-mail in response to an operation by the user via the operation unit 932. In addition, the control unit 931 causes the display unit 930 to display characters. In addition, the control unit 931 generates e-mail data in response to a transmission instruction from the user via the operation unit 932, and outputs the generated e-mail data to the communication unit 922. The communication unit 922 encodes and modulates email data and generates a transmission signal. Then, the communication unit 922 transmits the generated transmission signal to a base station (not shown) via the antenna 921. In addition, the communication unit 922 amplifies a radio signal received via the antenna 921 and performs frequency conversion to acquire a received signal. Then, the communication unit 922 demodulates and decodes the received signal to restore the email data, and outputs the restored email data to the control unit 931. The control unit 931 displays the content of the electronic mail on the display unit 930 and stores the electronic mail data in the storage medium of the recording / reproducing unit 929.

The recording / reproducing unit 929 has an arbitrary readable / writable storage medium. For example, the storage medium may be a built-in storage medium such as a RAM or a flash memory, or an externally mounted storage medium such as a hard disk, a magnetic disk, a magneto-optical disk, an optical disk, a USB memory, or a memory card. May be.

In the shooting mode, for example, the camera unit 926 images a subject to generate image data, and outputs the generated image data to the image processing unit 927. The image processing unit 927 encodes the image data input from the camera unit 926 and stores the encoded stream in the storage medium of the recording / playback unit 929.

Further, in the videophone mode, for example, the demultiplexing unit 928 multiplexes the video stream encoded by the image processing unit 927 and the audio stream input from the audio codec 923, and the multiplexed stream is the communication unit 922. Output to. The communication unit 922 encodes and modulates the stream and generates a transmission signal. Then, the communication unit 922 transmits the generated transmission signal to a base station (not shown) via the antenna 921. In addition, the communication unit 922 amplifies a radio signal received via the antenna 921 and performs frequency conversion to acquire a received signal. These transmission signal and reception signal may include an encoded bit stream. Then, the communication unit 922 demodulates and decodes the received signal to restore the stream, and outputs the restored stream to the demultiplexing unit 928. The demultiplexing unit 928 separates the video stream and the audio stream from the input stream, and outputs the video stream to the image processing unit 927 and the audio stream to the audio codec 923. The image processing unit 927 decodes the video stream and generates video data. The video data is supplied to the display unit 930, and a series of images is displayed on the display unit 930. The audio codec 923 decompresses the audio stream and performs D / A conversion to generate an analog audio signal. Then, the audio codec 923 supplies the generated audio signal to the speaker 924 to output audio.

Sensor unit 933 includes a sensor group such as an acceleration sensor and a gyro sensor, and outputs an index representing the movement of mobile phone 920. The battery 935 includes a communication unit 922, an audio codec 923, a camera unit 926, an image processing unit 927, a demultiplexing unit 928, a recording / reproducing unit 929, a display unit 930, and a control via a power supply line which is omitted in the drawing. Power is supplied to the unit 931 and the sensor unit 933.

In the mobile phone 920 configured as described above, the image processing unit 927 has the function of the image encoding device 10 according to the above-described embodiment. Thereby, in the mobile phone 920, the amount of cost calculation for determining the SAO mode to be applied and the offset value to be applied can be reduced, and consequently the power consumption and circuit scale of the mobile phone 920 can be reduced. It becomes possible.

(2) Second Application Example FIG. 27 shows an example of a schematic configuration of a recording / reproducing apparatus to which the above-described embodiment is applied. For example, the recording / reproducing device 940 encodes audio data and video data of a received broadcast program and records the encoded data on a recording medium. In addition, the recording / reproducing device 940 may encode audio data and video data acquired from another device and record them on a recording medium, for example. In addition, the recording / reproducing device 940 reproduces data recorded on the recording medium on a monitor and a speaker, for example, in accordance with a user instruction. At this time, the recording / reproducing device 940 decodes the audio data and the video data.

The recording / reproducing apparatus 940 includes a tuner 941, an external interface 942, an encoder 943, an HDD (Hard Disk Drive) 944, a disk drive 945, a selector 946, a decoder 947, an OSD (On-Screen Display) 948, a control unit 949, and a user interface. 950.

Tuner 941 extracts a signal of a desired channel from a broadcast signal received via an antenna (not shown), and demodulates the extracted signal. Then, the tuner 941 outputs the encoded bit stream obtained by the demodulation to the selector 946. That is, the tuner 941 has a role as a transmission unit in the recording / reproducing apparatus 940.

The external interface 942 is an interface for connecting the recording / reproducing apparatus 940 to an external device or a network. The external interface 942 may be, for example, an IEEE 1394 interface, a network interface, a USB interface, or a flash memory interface. For example, video data and audio data received via the external interface 942 are input to the encoder 943. That is, the external interface 942 serves as a transmission unit in the recording / reproducing device 940.

The encoder 943 encodes video data and audio data when the video data and audio data input from the external interface 942 are not encoded. Then, the encoder 943 outputs the encoded bit stream to the selector 946.

The HDD 944 records an encoded bit stream in which content data such as video and audio is compressed, various programs, and other data on an internal hard disk. Also, the HDD 944 reads out these data from the hard disk when playing back video and audio.

The disk drive 945 performs recording and reading of data to and from the mounted recording medium. The recording medium loaded in the disk drive 945 may be, for example, a DVD disk (DVD-Video, DVD-RAM, DVD-R, DVD-RW, DVD + R, DVD + RW, etc.) or a Blu-ray (registered trademark) disk. .

The selector 946 selects an encoded bit stream input from the tuner 941 or the encoder 943 when recording video and audio, and outputs the selected encoded bit stream to the HDD 944 or the disk drive 945. In addition, the selector 946 outputs the encoded bit stream input from the HDD 944 or the disk drive 945 to the decoder 947 during video and audio reproduction.

The decoder 947 decodes the encoded bit stream and generates video data and audio data. Then, the decoder 947 outputs the generated video data to the OSD 948. The decoder 904 outputs the generated audio data to an external speaker.

The OSD 948 reproduces the video data input from the decoder 947 and displays the video. Further, the OSD 948 may superimpose a GUI image such as a menu, a button, or a cursor on the video to be displayed.

The control unit 949 includes a processor such as a CPU and memories such as a RAM and a ROM. The memory stores a program executed by the CPU, program data, and the like. The program stored in the memory is read and executed by the CPU when the recording / reproducing apparatus 940 is activated, for example. The CPU controls the operation of the recording / reproducing device 940 according to an operation signal input from the user interface 950, for example, by executing the program.

The user interface 950 is connected to the control unit 949. The user interface 950 includes, for example, buttons and switches for the user to operate the recording / reproducing device 940, a remote control signal receiving unit, and the like. The user interface 950 detects an operation by the user via these components, generates an operation signal, and outputs the generated operation signal to the control unit 949.

In the recording / reproducing apparatus 940 configured in this way, the encoder 943 has the function of the image encoding apparatus 10 according to the above-described embodiment. Thereby, in the recording / reproducing apparatus 940, the processing amount of the cost calculation for determining the SAO mode to be applied and the offset value to be applied is reduced, and consequently, the power consumption and circuit scale of the recording / reproducing apparatus 940 are reduced. It becomes possible.

(3) Third Application Example FIG. 28 illustrates an example of a schematic configuration of an imaging apparatus to which the above-described embodiment is applied. The imaging device 960 images a subject to generate an image, encodes the image data, and records it on a recording medium.

The imaging device 960 includes an optical block 961, an imaging unit 962, a signal processing unit 963, an image processing unit 964, a display unit 965, an external interface 966, a memory 967, a media drive 968, an OSD 969, a control unit 970, a user interface 971, and a sensor 972. , A bus 973 and a battery 974.

The optical block 961 is connected to the imaging unit 962. The imaging unit 962 is connected to the signal processing unit 963. The display unit 965 is connected to the image processing unit 964. The user interface 971 is connected to the control unit 970. The bus 973 connects the image processing unit 964, the external interface 966, the memory 967, the media drive 968, the OSD 969, the control unit 970, and the sensor 972 to each other.

The optical block 961 includes a focus lens and a diaphragm mechanism. The optical block 961 forms an optical image of the subject on the imaging surface of the imaging unit 962. The imaging unit 962 includes an image sensor such as a CCD or a CMOS, and converts an optical image formed on the imaging surface into an image signal as an electrical signal by photoelectric conversion. Then, the imaging unit 962 outputs the image signal to the signal processing unit 963.

The signal processing unit 963 performs various camera signal processing such as knee correction, gamma correction, and color correction on the image signal input from the imaging unit 962. The signal processing unit 963 outputs the image data after the camera signal processing to the image processing unit 964.

The image processing unit 964 encodes the image data input from the signal processing unit 963 and generates encoded data. Then, the image processing unit 964 outputs the generated encoded data to the external interface 966 or the media drive 968. The image processing unit 964 also decodes encoded data input from the external interface 966 or the media drive 968 to generate image data. Then, the image processing unit 964 outputs the generated image data to the display unit 965. In addition, the image processing unit 964 may display the image by outputting the image data input from the signal processing unit 963 to the display unit 965. Further, the image processing unit 964 may superimpose display data acquired from the OSD 969 on an image output to the display unit 965.

The OSD 969 generates a GUI image such as a menu, a button, or a cursor, for example, and outputs the generated image to the image processing unit 964.

The external interface 966 is configured as a USB input / output terminal, for example. The external interface 966 connects the imaging device 960 and a printer, for example, when printing an image. Further, a drive is connected to the external interface 966 as necessary. For example, a removable medium such as a magnetic disk or an optical disk is attached to the drive, and a program read from the removable medium can be installed in the imaging device 960. Further, the external interface 966 may be configured as a network interface connected to a network such as a LAN or the Internet. That is, the external interface 966 has a role as a transmission unit in the imaging device 960.

The recording medium mounted on the media drive 968 may be any readable / writable removable medium such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory. Further, a recording medium may be fixedly attached to the media drive 968, and a non-portable storage unit such as an internal hard disk drive or an SSD (Solid State Drive) may be configured.

The control unit 970 includes a processor such as a CPU and memories such as a RAM and a ROM. The memory stores a program executed by the CPU, program data, and the like. The program stored in the memory is read and executed by the CPU when the imaging device 960 is activated, for example. The CPU controls the operation of the imaging device 960 according to an operation signal input from the user interface 971, for example, by executing the program.

The user interface 971 is connected to the control unit 970. The user interface 971 includes, for example, buttons and switches for the user to operate the imaging device 960. The user interface 971 detects an operation by the user via these components, generates an operation signal, and outputs the generated operation signal to the control unit 970.

The sensor 972 includes a sensor group such as an acceleration sensor and a gyro sensor, and outputs an index representing the movement of the imaging device 960. The battery 974 supplies power to the imaging unit 962, the signal processing unit 963, the image processing unit 964, the display unit 965, the media drive 968, the OSD 969, the control unit 970, and the sensor 972 via a power supply line that is omitted in the drawing. Supply.

In the imaging device 960 configured as described above, the image processing unit 964 has the function of the image encoding device 10 according to the above-described embodiment. Thereby, in the imaging device 960, it is possible to reduce the amount of cost calculation processing for determining the SAO mode to be applied and the offset value to be applied, and thus reduce the power consumption and circuit scale of the imaging device 960. It becomes possible.

<7-2. Various implementation levels>
The technology according to the present disclosure includes various implementation levels such as, for example, a processor such as a system LSI (Large Scale Integration), a module using a plurality of processors, a unit using a plurality of modules, and a set in which other functions are further added to the unit. May be implemented.

(1) Video Set An example of realizing the technology according to the present disclosure as a set will be described with reference to FIG. FIG. 29 is a block diagram illustrating an example of a schematic configuration of a video set.

In recent years, electronic devices have become multifunctional. Development or manufacture of an electronic device is performed for each function, and then proceeds to a stage where a plurality of functions are integrated. Accordingly, there are businesses that manufacture or sell only a part of electronic devices. The operator provides a component having a single function or a plurality of functions related to each other, or provides a set having an integrated function group. The video set 1300 shown in FIG. 29 is a set that includes components for encoding and / or decoding images (which may be either) and components having other functions related to these functions. is there.

Referring to FIG. 29, the video set 1300 includes a module group including a video module 1311, an external memory 1312, a power management module 1313, and a front end module 1314, and a related function including a connectivity module 1321, a camera 1322, and a sensor 1323. A device group.

A module is a component formed by aggregating parts for several functions related to each other. The module may have any physical configuration. As an example, the module may be formed by integrally arranging a plurality of processors having the same or different functions, electronic circuit elements such as resistors and capacitors, and other devices on a circuit board. Another module may be formed by combining another module or a processor with the module.

In the example of FIG. 29, in the video module 1311, parts for functions related to image processing are collected. The video module 1311 includes an application processor 1331, a video processor 1332, a broadband modem 1333, and a baseband module 1334.

The processor may be, for example, an SOC (System On a Chip) or a system LSI (Large Scale Integration). The SoC or the system LSI may include hardware that implements predetermined logic. The SoC or the system LSI may include a CPU and a non-transitory tangible medium that stores a program for causing the CPU to execute a predetermined function. The program is stored in, for example, a ROM, and can be executed by the CPU after being read into a RAM (Random Access Memory) at the time of execution.

Application processor 1331 is a processor that executes an application related to image processing. An application executed in the application processor 1331 may control, for example, the video processor 1332 and other components in addition to some calculation for image processing. The video processor 1332 is a processor having functions relating to image encoding and decoding. Note that the application processor 1331 and the video processor 1332 may be integrated into one processor (see a dotted line 1341 in the figure).

The broadband modem 1333 is a module that performs processing related to communication via a network such as the Internet or a public switched telephone network. For example, the broadband modem 1333 performs digital modulation for converting a digital signal including transmission data into an analog signal, and digital demodulation for converting an analog signal including reception data into a digital signal. Transmission data and reception data processed by the broadband modem 1333 may include arbitrary information such as image data, an encoded stream of image data, application data, an application program, and setting data, for example.

The baseband module 1334 is a module that performs baseband processing for an RF (Radio Frequency) signal transmitted / received via the front end module 1314. For example, the baseband module 1334 modulates a transmission baseband signal including transmission data, converts the frequency into an RF signal, and outputs the RF signal to the front end module 1314. In addition, the baseband module 1334 frequency-converts and demodulates the RF signal input from the front end module 1314 to generate a reception baseband signal including reception data.

The external memory 1312 is a memory device provided outside the video module 1311 and accessible from the video module 1311. When large-scale data such as video data including a large number of frames is stored in the external memory 1312, the external memory 1312 includes a relatively inexpensive and large-capacity semiconductor memory such as a DRAM (Dynamic Random Access Memory). obtain.

The power management module 1313 is a module that controls power supply to the video module 1311 and the front end module 1314.

The front end module 1314 is a module that is connected to the baseband module 1334 and provides a front end function. In the example of FIG. 29, the front end module 1314 includes an antenna unit 1351, a filter 1352, and an amplification unit 1353. The antenna unit 1351 includes one or more antenna elements that transmit or receive radio signals and related components such as an antenna switch. The antenna unit 1351 transmits the RF signal amplified by the amplification unit 1353 as a radio signal. Further, the antenna unit 1351 outputs an RF signal received as a radio signal to the filter 1352 and causes the filter 1352 to filter the RF signal.

The connectivity module 1321 is a module having a function related to the external connection of the video set 1300. The connectivity module 1321 may support any external connection protocol. For example, the connectivity module 1321 is a sub-module that supports a wireless connection protocol such as Bluetooth (registered trademark), IEEE 802.11 (for example, Wi-Fi (registered trademark)), NFC (Near Field Communication), or IrDA (InfraRed Data Association). And a corresponding antenna. In addition, the connectivity module 1321 may include a sub module that supports a wired connection protocol such as USB (Universal Serial Bus) or HDMI (registered trademark) (High-Definition Multimedia Interface) and a corresponding connection terminal. .

In addition, the connectivity module 1321 writes and stores data to a storage medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, or a storage device such as an SSD (Solid State Drive) or NAS (Network Attached Storage). A drive for reading data from the medium may be included. The connectivity module 1321 may include these storage media or storage devices. In addition, the connectivity module 1321 may provide connectivity to a display that outputs an image or a speaker that outputs sound.

The camera 1322 is a module that acquires a captured image by imaging a subject. A series of captured images acquired by the camera 1322 constitutes video data. Video data generated by the camera 1322 may be encoded by the video processor 1332 as necessary and stored by the external memory 1312 or a storage medium connected to the connectivity module 1321, for example.

The sensor 1323 is, for example, a GPS sensor, an audio sensor, an ultrasonic sensor, an optical sensor, an illuminance sensor, an infrared sensor, an angular velocity sensor, an angular acceleration sensor, a velocity sensor, an acceleration sensor, a gyro sensor, a geomagnetic sensor, an impact sensor, or a temperature sensor. A module that may include one or more of them. The sensor data generated by the sensor 1323 can be used by the application processor 1331 to execute an application, for example.

In the video set 1300 configured as described above, the technology according to the present disclosure can be used in the video processor 1332, for example. In this case, the video set 1300 is a set to which the technology according to the present disclosure is applied.

Note that the video set 1300 may be realized as various types of devices that process image data. For example, the video set 1300 may correspond to the mobile phone 920, the recording / reproducing device 940, or the imaging device 960 described with reference to FIGS.

(2) Video Processor FIG. 30 is a block diagram illustrating an example of a schematic configuration of the video processor 1332. The video processor 1332 encodes an input video signal and an input audio signal to generate video data and audio data, and decodes the encoded video data and audio data to generate an output video signal and an output audio signal. And a function to perform.

Referring to FIG. 30, the video processor 1332 includes a video input processing unit 1401, a first scaling unit 1402, a second scaling unit 1403, a video output processing unit 1404, a frame memory 1405, a memory control unit 1406, an encoding / decoding engine 1407, Video ES (Elementary Stream) buffers 1408A and 1408B,

audio ES buffers

1409A and 1409B, an audio encoder 1410, an audio decoder 1411, a multiplexing unit (MUX) 1412, a demultiplexing unit (DEMUX) 1413, and a stream buffer 1414 .

The video input processing unit 1401 converts, for example, a video signal input from the connectivity module 1321 into digital image data. The first scaling unit 1402 performs format conversion and scaling (enlargement / reduction) on the image data input from the video input processing unit 1401. The second scaling unit 1403 performs format conversion and scaling (enlargement / reduction) on the image data output to the video output processing unit 1404. The format conversion in the first scaling unit 1402 and the second scaling unit 1403 is, for example, conversion between 4: 2: 2 / Y-Cb-Cr system and 4: 2: 0 / Y-Cb-Cr system. It may be. The video output processing unit 1404 converts the digital image data into an output video signal and outputs the output video signal to, for example, the connectivity module 1321.

The frame memory 1405 is a memory device that stores image data shared by the video input processing unit 1401, the first scaling unit 1402, the second scaling unit 1403, the video output processing unit 1404, and the encoding / decoding engine 1407. The frame memory 1405 may be realized using a semiconductor memory such as a DRAM, for example.

The memory control unit 1406 controls access to the frame memory 1405 according to the access schedule for the frame memory 1405 stored in the access management table 1406A based on the synchronization signal input from the encode / decode engine 1407. The access management table 1406A is updated by the memory control unit 1406 depending on processing executed in the encoding / decoding engine 1407, the first scaling unit 1402, the second scaling unit 1403, and the like.

The encoding / decoding engine 1407 performs an encoding process for encoding image data to generate an encoded video stream, and a decoding process for decoding image data from the encoded video stream. For example, the encoding / decoding engine 1407 encodes the image data read from the frame memory 1405 and sequentially writes the encoded video stream to the video ES buffer 1408A. Also, for example, the encoded video stream is sequentially read from the video ES buffer 1408B, and the decoded image data is written in the frame memory 1405. The encoding / decoding engine 1407 can use the frame memory 1405 as a work area in these processes. For example, the encoding / decoding engine 1407 outputs a synchronization signal to the memory control unit 1406 at the timing of starting processing of each LCU (Largest Coding Unit).

The video ES buffer 1408A buffers the encoded video stream generated by the encoding / decoding engine 1407. The encoded video stream buffered by the video ES buffer 1408A is output to the multiplexing unit 1412. The video ES buffer 1408B buffers the encoded video stream input from the demultiplexer 1413. The encoded video stream buffered by the video ES buffer 1408B is output to the encoding / decoding engine 1407.

The audio ES buffer 1409A buffers the encoded audio stream generated by the audio encoder 1410. The encoded audio stream buffered by the audio ES buffer 1409A is output to the multiplexing unit 1412. The audio ES buffer 1409B buffers the encoded audio stream input from the demultiplexer 1413. The encoded audio stream buffered by the audio ES buffer 1409B is output to the audio decoder 1411.

The audio encoder 1410 digitally converts the input audio signal input from the connectivity module 1321, for example, and encodes the input audio signal according to an audio encoding method such as an MPEG audio method or an AC3 (Audio Code number 3) method. The audio encoder 1410 sequentially writes the encoded audio stream to the audio ES buffer 1409A. The audio decoder 1411 decodes audio data from the encoded audio stream input from the audio ES buffer 1409B and converts it into an analog signal. The audio decoder 1411 outputs an audio signal to the connectivity module 1321, for example, as a reproduced analog audio signal.

The multiplexing unit 1412 multiplexes the encoded video stream and the encoded audio stream to generate a multiplexed bit stream. The format of the multiplexed bit stream may be any format. The multiplexing unit 1412 may add predetermined header information to the bit stream. Further, the multiplexing unit 1412 may convert the stream format. For example, the multiplexing unit 1412 can generate a transport stream (a bit stream in a transfer format) in which an encoded video stream and an encoded audio stream are multiplexed. Further, the multiplexing unit 1412 can generate file data (recording format data) in which the encoded video stream and the encoded audio stream are multiplexed.

The demultiplexing unit 1413 demultiplexes the encoded video stream and the encoded audio stream from the multiplexed bit stream by a method reverse to the multiplexing performed by the multiplexing unit 1412. That is, the demultiplexer 1413 extracts (or separates) the video stream and the audio stream from the bit stream read from the stream buffer 1414. The demultiplexer 1413 may convert the stream format (inverse conversion). For example, the demultiplexing unit 1413 may acquire a transport stream that can be input from the connectivity module 1321 or the broadband modem 1333 via the stream buffer 1414, and convert the transport stream into a video stream and an audio stream. . Further, the demultiplexing unit 1413 may acquire file data read from the storage medium by the connectivity module 1321 via the stream buffer 1414 and convert the file data into a video stream and an audio stream.

Stream buffer 1414 buffers the bit stream. For example, the stream buffer 1414 buffers the transport stream input from the multiplexing unit 1412 and outputs the transport stream to, for example, the connectivity module 1321 or the broadband modem 1333 at a predetermined timing or in response to an external request. To do. Further, for example, the stream buffer 1414 buffers the file data input from the multiplexing unit 1412 and records the file data to the connectivity module 1321, for example, at a predetermined timing or in response to an external request. Output to. Further, the stream buffer 1414 buffers a transport stream acquired through, for example, the connectivity module 1321 or the broadband modem 1333, and demultiplexes the transport stream at a predetermined timing or in response to an external request. Output to the unit 1413. Also, the stream buffer 1414 buffers file data read from the storage medium by the connectivity module 1321, for example, and outputs the file data to the demultiplexing unit 1413 at a predetermined timing or in response to an external request. To do.

In the video processor 1332 configured as described above, the technology according to the present disclosure can be used in the encode / decode engine 1407, for example. In this case, the video processor 1332 is a chip or a module to which the technology according to the present disclosure is applied.

FIG. 31 is a block diagram showing another example of a schematic configuration of the video processor 1332. In the example of FIG. 31, the video processor 1332 has a function of encoding and decoding video data by a predetermined method.

Referring to FIG. 31, the video processor 1332 includes a control unit 1511, a display interface 1512, a display engine 1513, an image processing engine 1514, an internal memory 1515, a codec engine 1516, a memory interface 1517, a multiplexing / demultiplexing unit 1518, a network. An interface 1519 and a video interface 1520 are included.

The control unit 1511 controls operations of various processing units in the video processor 1332 such as the display interface 1512, the display engine 1513, the image processing engine 1514, and the codec engine 1516. The control unit 1511 includes, for example, a main CPU 1531, a sub CPU 1532, and a system controller 1533. The main CPU 1531 executes a program for controlling the operation of each processing unit in the video processor 1332. The main CPU 1531 supplies a control signal generated through execution of the program to each processing unit. The sub CPU 1532 plays an auxiliary role of the main CPU 1531. For example, the sub CPU 1532 executes a child process and a subroutine of a program executed by the main CPU 1531. The system controller 1533 manages execution of programs by the main CPU 1531 and the sub CPU 1532.

The display interface 1512 outputs the image data to, for example, the connectivity module 1321 under the control of the control unit 1511. For example, the display interface 1512 outputs an analog image signal converted from digital image data or the digital image data itself to a display connected to the connectivity module 1321. Under the control of the control unit 1511, the display engine 1513 executes format conversion, size conversion, color gamut conversion, and the like for the image data so that the attributes of the image data match the specifications of the output display. The image processing engine 1514 performs image processing that may include filtering processing having an object such as image quality improvement on the image data under the control of the control unit 1511.

The internal memory 1515 is a memory device provided inside the video processor 1332 that is shared by the display engine 1513, the image processing engine 1514, and the codec engine 1516. The internal memory 1515 is used when inputting / outputting image data among the display engine 1513, the image processing engine 1514, and the codec engine 1516, for example. The internal memory 1515 may be any type of memory device. For example, the internal memory 1515 may have a relatively small memory size for storing block unit image data and associated parameters. The internal memory 1515 may be a memory having a small capacity but a fast response speed such as SRAM (Static Random Access Memory) (for example, relative to the external memory 1312).

The codec engine 1516 performs an encoding process for encoding image data to generate an encoded video stream, and a decoding process for decoding image data from the encoded video stream. The image encoding scheme supported by the codec engine 1516 may be any one or a plurality of schemes. In the example of FIG. 31, the codec engine 1516 includes an MPEG-2 Video block 1541, an AVC / H. H.264 block 1542, HEVC / H. H.265 block 1543, HEVC / H. 265 (scalable) block 1544, HEVC / H. 265 (multi-view) block 1545 and MPEG-DASH block 1551. Each of these functional blocks encodes and decodes image data according to a corresponding image encoding method.

The MPEG-DASH block 1551 is a functional block that enables image data to be transmitted according to the MPEG-DASH system. The MPEG-DASH block 1551 executes generation of a stream conforming to the standard specification and control of transmission of the generated stream. The encoding and decoding of the image data to be transmitted may be performed by other functional blocks included in the codec engine 1516.

The memory interface 1517 is an interface for connecting the video processor 1332 to the external memory 1312. Data generated by the image processing engine 1514 or the codec engine 1516 is output to the external memory 1312 via the memory interface 1517. Data input from the external memory 1312 is supplied to the image processing engine 1514 or the codec engine 1516 via the memory interface 1517.

The multiplexing / demultiplexing unit 1518 multiplexes and demultiplexes the encoded video stream and the related bit stream. At the time of multiplexing, the multiplexing / demultiplexing unit 1518 may add predetermined header information to the multiplexed stream. Also, at the time of demultiplexing, the multiplexing / demultiplexing unit 1518 may add predetermined header information to each separated stream. That is, the multiplexing / demultiplexing unit 1518 can perform format conversion together with multiplexing or demultiplexing. For example, the multiplexing / demultiplexing unit 1518 performs conversion and inverse conversion between a plurality of bit streams and a transport stream, which is a multiplexed stream having a transfer format, and a plurality of bit streams and a recording format. You may support conversion and reverse conversion to and from file data.

The network interface 1519 is an interface for connecting the video processor 1332 to the broadband modem 1333 or the connectivity module 1321, for example. The video interface 1520 is an interface for connecting the video processor 1332 to the connectivity module 1321 or the camera 1322, for example.

In the video processor 1332 configured as described above, the technology according to the present disclosure may be used in, for example, the codec engine 1516. In this case, the video processor 1332 is a chip or a module to which the technology according to the present disclosure is applied.

Note that the configuration of the video processor 1332 is not limited to the two examples described above. For example, the video processor 1332 may be realized as a single semiconductor chip, or may be realized as a plurality of semiconductor chips. Further, the video processor 1332 may be realized as a three-dimensional stacked LSI formed by stacking a plurality of semiconductors, or a combination of a plurality of LSIs.

<< 8. Conclusion >>
Heretofore, the image encoding device according to each embodiment of the present disclosure has been described in detail with reference to FIGS. 1 to 31.

Note that the technique according to the present disclosure may be applied to a scalable coding technique. HEVC scalable coding technology is also referred to as SHVC (Scalable HEVC). For example, the above-described embodiments can be applied to individual layers (base layer and enhancement layer) included in a multi-layer encoded stream. Information regarding the determined SAO mode and offset value may be generated and encoded for each layer, or may be reused between layers. Further, the technology according to the present disclosure may be applied to a multi-view encoding technology. For example, the above-described embodiments can be applied to individual views (base view and non-base view) included in a multi-view encoded stream. Information regarding the determined SAO mode and offset value may be generated and encoded for each view, or may be reused between views.

The terms CU, PU, and TU described in this specification mean a logical unit that also includes syntax associated with individual blocks in HEVC. When focusing only on individual blocks as a part of an image, these may be replaced by the terms CB (Coding Block), PB (Prediction Block), and TB (Transform Block), respectively. The CB is formed by hierarchically dividing a CTB (Coding Tree Block) into a quad-tree shape. An entire quadtree corresponds to CTB, and a logical unit corresponding to CTB is called CTU (Coding Tree Unit).

Also, in this specification, an example in which information regarding the determined SAO mode and offset value is multiplexed in the header of the encoded stream and transmitted from the encoding side to the decoding side has been mainly described. However, the method for transmitting such information is not limited to such an example. For example, these pieces of information may be transmitted or recorded as separate data associated with the encoded bitstream without being multiplexed into the encoded bitstream. Here, the term “associate” means that an image (which may be a part of an image such as a slice or a block) included in the bitstream and information corresponding to the image can be linked at the time of decoding. Means. That is, information may be transmitted on a transmission path different from that of the image (or bit stream). Information may be recorded on a recording medium (or another recording area of the same recording medium) different from the image (or bit stream). Furthermore, the information and the image (or bit stream) may be associated with each other in an arbitrary unit such as a plurality of frames, one frame, or a part of the frame.

The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that it belongs to the technical scope of the present disclosure.

In addition, the effects described in this specification are merely illustrative or illustrative, and are not limited. That is, the technology according to the present disclosure can exhibit other effects that are apparent to those skilled in the art from the description of the present specification in addition to or instead of the above effects.

The following configurations also belong to the technical scope of the present disclosure.
(1)
A setting unit that sets a range of candidate offset values to be applied to pixels of a decoded image based on a quantization parameter used when the image bit depth is quantized according to a threshold value; ,
A filter processing unit that executes a filter process for applying the offset value selected from the candidates included in the set range to the pixels of the decoded image;
An image processing apparatus comprising:
(2)
The image processing apparatus according to (1), wherein the setting unit controls an interval between the adjacent candidates according to the quantization parameter.
(3)
The setting unit sets a range of candidates for the offset value by setting a parameter for enlarging or reducing the offset value according to the bit depth of the image based on the quantization parameter. The image processing apparatus according to 1) or (2).
(4)
The image according to any one of (1) to (3), wherein the setting unit sets the range such that the range of the candidate offset value is wider as the quantization parameter is larger. Processing equipment.
(5)
The setting unit restricts the number of offset value candidates according to the bit depth of the image, and sets the offset value candidates based on the quantization parameter, any of (1) to (4) An image processing apparatus according to claim 1.
(6)
The setting unit sets the number of offset value candidates in the case of a first bit depth that is the bit depth of the image to the offset value in the case of a second bit depth that is smaller than the first bit depth. The image processing apparatus according to (5), wherein the number is limited to the number of candidates.
(7)
The setting unit sets a range of candidates for the offset value based on the quantization parameter when the bit depth of the image is larger than a threshold, according to any one of (1) to (6), The image processing apparatus described.
(8)
The said setting part is an image processing apparatus as described in said (7) which sets the range of the said candidate of an offset value based on the said quantization parameter, when the bit depth of the said image is larger than 10 bits.
(9)
The setting unit sets a range of candidates for the offset value applied to the pixel of the decoded image according to the offset value applied to the pixel of the image that has already been subjected to the encoding process. The image processing apparatus according to any one of (1) to (8).
(10)
In the offset value applied to the pixel of the image that has already been subjected to the encoding process, the setting unit applies to the pixel of the decoded image as the application frequency of the offset value that is smaller than the threshold is higher. The image processing apparatus according to (9), wherein the range is set so that a range of candidates for the offset value to be applied becomes narrower.
(11)
The image processing apparatus according to any one of (1) to (10), wherein the setting unit sets a range of candidates for the offset value applied to the image based on a feature amount of the image.
(12)
The image processing apparatus according to (11), wherein the setting unit sets the range such that the larger the feature amount of the image is, the wider the range of candidates for the offset value applied to the image is. .
(13)
Processor
According to the bit depth of the image, based on the quantization parameter used when quantizing the image, setting a range of candidate offset values to be applied to the decoded image pixels;
Executing a filtering process that applies the offset value selected from the candidates included in the set range to the pixels of the decoded image;
Including an image processing method.
(14)
The range of candidate offset values to be applied to the decoded image pixels in accordance with the offset value applied to the image pixels that have already been encoded according to the image bit depth. A setting section for setting
A filter processing unit that executes a filter process for applying the offset value selected from the candidates included in the set range to the pixels of the decoded image;
An image processing apparatus comprising:
(15)
A setting unit that sets a range of candidate offset values to be applied to the decoded pixels of the decoded image based on the feature amount of the image according to the bit depth of the image;
A filter processing unit that executes a filter process for applying the offset value selected from the candidates included in the set range to the pixels of the decoded image;
An image processing apparatus comprising:
(16)
Processor
The range of candidate offset values to be applied to the decoded image pixels in accordance with the offset value applied to the image pixels that have already been encoded according to the image bit depth. Setting
Executing a filtering process that applies the offset value selected from the candidates included in the set range to the pixels of the decoded image;
Including an image processing method.
(17)
Processor
Setting a range of candidate offset values to be applied to the decoded pixels of the decoded image based on the feature amount of the image according to the bit depth of the image;
Executing a filtering process that applies the offset value selected from the candidates included in the set range to the pixels of the decoded image;
Including an image processing method.

DESCRIPTION OF SYMBOLS 10 Image coding apparatus 11 Rearrangement buffer 13 Subtraction part 14 Orthogonal transformation part 15 Quantization part 16 Lossless encoding part 17 Accumulation buffer 18 Rate control part 21 Inverse quantization part 22 Inverse orthogonal transformation part 23 Adder part 24 Loop filter 25 Frame Memory 26 Selection Unit 27 Selection Unit 30 Intra Prediction Unit 35 Inter Prediction Unit 100 Original Image Holding Unit 200 Deblock Filter 300 Sample Adaptive Offset Filter 310 Control Unit 320 Analysis Unit 330 Statistics Acquisition Unit 340 Mode Determination Unit 350 Offset Determination Unit 351 Offset Measurement unit 353 Offset determination unit 355 Candidate control unit 360 Determination unit 390 Filter processing unit 400 Adaptive loop filter

Claims

A setting unit that sets a range of candidate offset values to be applied to the decoded image pixels based on the quantization parameter used when the image is quantized according to the bit depth of the image;
A filter processing unit that executes a filter process for applying the offset value selected from the candidates included in the set range to the pixels of the decoded image;
An image processing apparatus comprising:
The image processing device according to claim 1, wherein the setting unit controls an interval between the adjacent candidates according to the quantization parameter.
The setting unit sets a range of candidates for the offset value by setting a parameter for enlarging or reducing the offset value according to the bit depth of the image based on the quantization parameter. The image processing apparatus according to 1.
The image processing apparatus according to claim 1, wherein the setting unit sets the range such that the range of the candidate offset value is wider as the quantization parameter is larger.
The image processing apparatus according to claim 1, wherein the setting unit limits the number of offset value candidates according to a bit depth of the image, and sets the offset value candidates based on the quantization parameter.
The setting unit sets the number of offset value candidates in the case of a first bit depth that is the bit depth of the image to the offset value in the case of a second bit depth that is smaller than the first bit depth. The image processing apparatus according to claim 5, wherein the number is limited to the number of candidates.
The image processing apparatus according to claim 1, wherein the setting unit sets a range of candidates for the offset value based on the quantization parameter when the bit depth of the image is larger than a threshold value.
The image processing apparatus according to claim 7, wherein the setting unit sets a range of candidates for the offset value based on the quantization parameter when the bit depth of the image is larger than 10 bits.
The setting unit sets a range of candidates for the offset value applied to the pixel of the decoded image according to the offset value applied to the pixel of the image that has already been subjected to the encoding process. The image processing apparatus according to claim 1.
In the offset value applied to the pixel of the image that has already been subjected to the encoding process, the setting unit applies to the pixel of the decoded image as the application frequency of the offset value that is smaller than the threshold is higher. The image processing apparatus according to claim 9, wherein the range is set so that the range of the offset value candidates to be applied becomes narrower.
The image processing apparatus according to claim 1, wherein the setting unit sets a range of candidates for the offset value to be applied to the image based on the feature amount of the image.
12. The image processing apparatus according to claim 11, wherein the setting unit sets the range such that the larger the feature amount of the image, the wider the range of candidates for the offset value applied to the image.
Processor
According to the bit depth of the image, based on the quantization parameter used when quantizing the image, setting a range of candidate offset values to be applied to the decoded image pixels;
Executing a filtering process that applies the offset value selected from the candidates included in the set range to the pixels of the decoded image;
Including an image processing method.
The range of candidate offset values to be applied to the decoded image pixels in accordance with the offset value applied to the image pixels that have already been encoded according to the image bit depth. A setting section for setting
A filter processing unit that executes a filter process for applying the offset value selected from the candidates included in the set range to the pixels of the decoded image;
An image processing apparatus comprising:
A setting unit that sets a range of candidate offset values to be applied to the decoded pixels of the decoded image based on the feature amount of the image according to the bit depth of the image;
A filter processing unit that executes a filter process for applying the offset value selected from the candidates included in the set range to the pixels of the decoded image;
An image processing apparatus comprising:
Processor
The range of candidate offset values to be applied to the decoded image pixels in accordance with the offset value applied to the image pixels that have already been encoded according to the image bit depth. Setting
Executing a filtering process that applies the offset value selected from the candidates included in the set range to the pixels of the decoded image;
Including an image processing method.
Processor
Setting a range of candidate offset values to be applied to the decoded pixels of the decoded image based on the feature amount of the image according to the bit depth of the image;
Executing a filtering process that applies the offset value selected from the candidates included in the set range to the pixels of the decoded image;
Including an image processing method.