TW201937929A - Image processing apparatus, image processing method, and image processing program - Google Patents

Abstract

An image processing apparatus of the disclosure includes: a blocking unit configured to divide image information into blocks; a map information acquisition unit configured to acquire at least one piece of map information in which pieces of reference pixel information and pieces of positional information are associated with a target pixel in each of the blocks, the pieces of reference pixel information denoting at least one reference pixel referenced by the target pixel, each of the pieces of positional information denoting a positional relationship between the target pixel and the at least one reference pixel; and a reference pixel information selecting unit configured to select one piece of reference pixel information of the pieces of reference pixel information included in the at least one piece of map information based on a pixel value of the target pixel and a pixel value of the at least one reference pixel.

Description

Image processing device, image processing method, and image processing program

The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

As an image coding technology, as disclosed in Patent Document 1, S3TC (S3 Texture Compression, also referred to as DXTC) is known as a texture coding technology for 3DCG. Here, S3TC divides the image information into 4 × 4 blocks. Next, two representative colors and two intermediate colors between the representative colors are set for each block, and each pixel is replaced with these four colors (replacement image). Specifically, index information indicating which of the four colors is used is stored. According to the above configuration, the replacement image can be reproduced (reproduced) by retaining the representative color and index information of each block. Thereby, image information can be compressed.

Patent Document 1: Japanese Public Table Patent Gazette "Special Table 2001-519608".

However, depending on the image, the degree of deterioration of the decoded image may increase. In view of the foregoing problems, an object of the present invention is to achieve an image processing device capable of preventing degradation of an image after decoding, as compared with the prior art.

The image processing device of the present invention includes: a segmentation section that divides image information into a plurality of blocks; and a mapping information acquisition section for acquiring mapping information, which is a reference pixel to be referred to by a display target pixel A plurality of reference pixel information, and information relating to position information indicating a positional relationship between the target pixel and the reference pixel and the target pixel in the block; a reference pixel information selection unit, based on the pixels of the target pixel Value and the pixel value of the reference pixel, and among the plurality of reference pixel information included in the mapping information, one reference pixel information is selected.

In the image processing method of the present invention, the image information is divided into a plurality of blocks, and mapping information is obtained. The mapping information is a plurality of reference pixel information indicating a reference pixel to be referred to by the target pixel, and the target pixel is indicated. Position information related to the positional relationship between the reference pixels and information associated with the target pixel within the block; and based on the pixel value of the target pixel and the pixel value of the reference pixel, the From among the plurality of reference pixel information, select one reference pixel information.

The image processing program of the present invention enables a computer to implement the functions of the following sections: a block-based section that divides image information into a plurality of blocks; a mapping information obtaining section for obtaining mapping information that maps pixels to be displayed A plurality of reference pixel information of a reference pixel to be referred to, and information relating to position information indicating a positional relationship between the target pixel and the reference pixel and the target pixel in the block; and a reference pixel information selection section, Based on the pixel value of the target pixel and the pixel value of the reference pixel, one of the reference pixel information is selected from the plurality of reference pixel information included in the mapping information.

The following parameter diagrams describe preferred embodiments of the present invention in detail. The dimensions, materials, and other specific numerical values shown in the embodiments are merely examples to facilitate understanding of the present invention, and the present invention is not limited unless otherwise specified. In addition, in this specification and the drawings, elements having substantially the same functions and structures are denoted by the same symbols, and redundant descriptions are omitted, and elements not directly related to the present invention are omitted.

FIG. 1 is a schematic diagram illustrating an example of a configuration of an image processing apparatus according to the present invention. As shown in FIG. 1, the image processing device 11 of the present invention functionally includes, for example, a block generation unit 12, a mapping information acquisition unit 13, a reference pixel information selection unit 14, a mapping selection unit 15, a quantization unit 16, and a storage unit. The unit 17, the computing unit 18, and the coding unit 19.

In addition, the image processing device 11 includes, as a hardware configuration, a control section composed of, for example, a CPU, an MPU, and the like, and operates according to a program stored in the storage section 17. In addition, the configuration is only an example. For example, it may be configured that mapping information (reference pixel information, position-related information, bit allocation information, and mode information) shown in FIG. 4B described later is stored in one LSI. Wait). The control unit realizes the block generation unit 12, the mapping information acquisition unit 13, the reference pixel information selection unit 14, the mapping selection unit 15, the quantization unit 16, the calculation unit 18, and the encoding unit 19. The storage unit 17 is composed of an information storage medium such as a ROM or a RAM, and is an information storage medium for storing a program executed by the control unit. The storage unit 17 also functions as a working memory of the control unit. In addition, the program may be provided by downloading through a network, or provided by various information storage media readable by a computer such as a CD-ROM, DVD-ROM, or semiconductor memory.

Here, the image information input to the image processing device 11 is, for example, as shown in FIG. 2, the image information Vmn is m pixel information arranged in the horizontal direction X (the left-right direction in the figure) and is in the vertical direction. N pixel information is arranged on Y (up and down direction in the figure) (m and n are natural numbers). The image information is formed of, for example, various colors of RGB. In addition to RGB, the image information may also be formed from image information in different forms such as YUV. In the following examples, RGB is used as an example for description.

For example, as shown in FIG. 3, the block division unit 12 divides the input image information into a plurality of blocks including 16 pixels P0 to P15 in a 4 × 4 line. The block is not limited to 4 × 4 lines, as long as it includes a plurality of pixels, it is not particularly limited.

The pixel value of each pixel of the block is quantized by the quantization unit 16 and a quantized block is generated. The quantized block is stored in the storage unit 17.

The quantization unit 16 calculates, as a representative value, the maximum value and the minimum value of the RGB colors of the image information based on the pixel values of the pixels in the block, for example. Then, the pixel value of each pixel in the block is quantized by replacing the maximum value and the minimum value with a plurality of intermediate values represented by a predetermined bit width (for example, 12 bits). The blocks and the quantized blocks are stored in the storage unit 17. In the above examples, the maximum value and the minimum value are taken as the representative values, but it is not limited to this.

The mapping information acquisition unit 13 obtains mapping information, and the mapping information is information related to a plurality of reference pixel information indicating a reference pixel to be referred to by the target pixel and position information indicating a positional relationship between the target pixel and the reference pixel. Specifically, description is made using the mapping information shown in FIG. 4B. In FIG. 4B, the target pixels are Q1 to Q4. As shown on the right side of FIG. 4B, reference pixels P1, P4, P1 and P13, P4 and P7 are associated with the target pixel Q1 as reference pixel information. As shown in the left side of FIG. 4B, the positional relationship between each reference pixel and the target pixel is also associated. Further, as shown on the right side of FIG. 4B, each reference pixel information (target pixels Q1 and P1, P4, P1 and P13, P4 and P7, etc.) is associated with the mode information, or the mode information indicates whether the reference pixel is directly referenced or not. 1: 2 internal classification. A detailed description of the direct reference and internal division will be described later. Further, in the mapping information, the bit width of the target pixel and the pixel bit width other than the target pixel are also related. In addition, the mapping information is only an example, and the mapping information is not limited to the above configuration.

In addition, each mapping information is set in advance corresponding to each image in which the degree of degradation of the image during decoding in the conventional technique described above is large. Specifically, for example, when the color of pixels in a block continuously changes like a gradient, the mapping information represented by code 6 or code 7 can suppress the degree of image degradation during decoding.

The reference pixel information selection unit 14 selects one reference pixel information based on the pixel value of the target pixel and the pixel value of the reference pixel. Specifically, the case of the target pixel Q1 will be described as an example. As described above, Q1 is associated with P1, P4, P1 and P13, P4, and P7, and thus obtains candidate pixel values, respectively. More specifically, for example, in the reference pixel information in the first row of FIG. 4B, a direct reference is associated with the mode information, so the candidate pixel value is the pixel value of P1. Similarly, the reference pixel value of the second row is the pixel value of P4. On the other hand, regarding the pixel values of the third row, since P1 and P13 are associated as reference pixel information, candidate pixel information is obtained based on the distance between the target pixel Q1 and the reference pixels P1 and P13. Specifically, the ratio of the distance between Q1 and P1 and the distance between Q1 and P13 is 1: 2, so the candidate pixel information of Q1 is obtained according to the pixel value of P1 × 2/3 + the pixel value of P13 × 1/3. . As for the reference pixel value of the fourth row, P4 and P7 are associated as the reference pixel information. Therefore, similarly to the reference pixel information of the third row, the pixel value of P4 × 2/3 + the pixel value of P7 × 1/3 To get the reference pixel value. When expressed by a formula, the formula for calculating the reference pixel value based on the distance corresponding to Q1 and P1 and the distance between Q1 and P13 is as follows.

[Formula 1]

The calculation method of the reference pixel value is only an example, and the calculation method is not limited. The reference pixel information selection unit 14 further causes the arithmetic unit 18 to calculate a difference between the target pixel value and the reference pixel value, and selects, as the reference pixel information, a reference pixel corresponding to a reference pixel value whose difference is the smallest. Although an example of selecting by calculating a difference is shown here, it is not limited to this, and any other method may be used as long as it is a selection method that reduces the error between the pixel value of the target pixel and the reference pixel value. For example, the difference value is calculated using an approximate difference value or the like.

The reference pixel information selection unit 14 similarly selects reference pixel information on the target pixels Q2 to Q4 shown in FIG. 4B. When the mapping information shown in FIG. 4B to FIG. 4H is acquired, the same processing as described above is performed, and therefore description thereof is omitted.

The mapping selection unit 15 selects one of the plurality of mapping information based on a replacement block and a block before replacement, wherein the replacement block is a reference selected by the reference pixel information selection unit 14 for each mapping information. The pixel value calculated by the pixel information is replaced. The mapping selection unit 15 replaces the reference pixel corresponding to each target pixel with the obtained reference pixel value based on the mapping information including the reference pixel information of each target pixel, and obtains a replacement block. The mapping selection unit 15 calculates a difference between the obtained replacement block and the block before replacement, and selects mapping information having the smallest difference. The calculation method of the difference is not particularly limited, and a direct difference may be calculated, or an approximate value difference may be calculated. An example of calculating the difference and selecting is shown here. The method for selecting the mapping information is not limited to this, and any method may be used as long as the error between the replacement block and the block before replacement is reduced.

For example, the encoding unit 19 outputs as encoded information: quantized data of the selected mapping information (for example, code 1), and reference pixel information (for example, the reference pixel P1 indicating Q1) corresponding to the selected mapping information. (Information) quantized data, quantized data of pixel values (eg, P0) other than the target pixel, and quantized data of the maximum and minimum values. As described above, the mapping information is associated with the bit allocation information. For example, in accordance with the mapping information, the quantization unit 16 quantizes the reference pixel information and pixel values other than the target pixel. In addition, it may be configured to determine whether it is RGB-based data or YUV-based data according to an order included in the data of the maximum value and the minimum value.

The image processing apparatus of the present invention may be further configured to include a decoding unit (not shown), and the encoded information is decoded by the decoding unit to generate decoded image data.

In addition, the communication processing device 11 may further include a monochrome determination unit for determining whether or not the image information is composed of monochrome. In this case, the monochrome determination unit determines whether the pixels in the block are represented by monochrome. When it is determined that each pixel in the block is monochrome, the remaining bits are also allocated to the bit width at the time of quantization by setting the maximum / minimum values of the RGB colors independently. For example, processing is performed using the mapping information of code 0 as shown in FIG. 4A. As shown in FIG. 4A, since there is no reference pixel, the information of the block is used as it is during encoding. For example, a block having a bit width of 12 bits is allocated using pixels P0 to 15 in the block shown in FIG. 4A. Moreover, in this case, processing can also be performed using the mapping information shown in the above-mentioned FIGS. 4B to 4H. In addition, in the case of a single color, since it can be represented by one color, there is no need to assign a bit width to other colors. Therefore, it may be configured to further allocate the remaining bit width to the single color. In addition, for example, even if the values R, G, B, R, and G of each pixel are the same within a certain range, it can be processed like a single color. The combination is not limited to this. In the case where monochrome judgment is provided, it is possible to judge at the time of decoding by adding additional bits that determine whether or not the image is monochrome.

Hereinafter, an image processing flow executed by the image processing device 11 according to the present embodiment will be described with reference to a flowchart illustrated in FIG. 5.

As shown in FIG. 5, image information is input to the image processing apparatus 11 (S1). The block division unit 12 divides the input image information into a plurality of blocks including 16 pixels P0 to P15 in a 4 × 4 line (S2). Based on the pixel values of the pixels in the block, the maximum and minimum values of the RGB colors of the image information are calculated as representative values (S3).

The mapping information acquisition unit 13 obtains mapping information, and the mapping information is information associated with a plurality of reference pixel information indicating a reference pixel to be referred to by the target pixel and position information indicating a positional relationship between the target pixel and the reference pixel (S4) . The reference pixel information selection unit 14 further causes the arithmetic unit 18 to calculate a difference between the target pixel value and the reference pixel value, and selects, as the reference pixel information, a reference pixel corresponding to a reference pixel value in which the difference is smallest (S5). This processing is performed for each mapping information.

The mapping selection section 15 selects one of the plurality of mapping information based on the replacement block and the block before replacement, wherein the replacement block is calculated based on the reference pixel information selected by the reference pixel information selection section 14 The pixel value is replaced (S6).

For example, the encoding unit 19 outputs as encoded information: quantized data of the selected mapping information (for example, code 1), and reference pixel information (for example, the reference pixel P1 indicating Q1) corresponding to the selected mapping information. Information) Quantized data of pixel values (for example, P0) other than the quantized data object pixel, and quantized data of the maximum and minimum values (S7).

According to the above embodiment, it is possible to prevent the decoded image from being deteriorated as compared with the above-mentioned conventional technology.

For example, in the above embodiment, it may be configured to determine the color of each color (RGB or YUV) based on the overall number of bits and the difference ratio (range ratio, r1≤r2≤r3) of the maximum and minimum values of each color Bit allocation (b1≤b2≤b3), so that allocation is performed. In this case, the bit allocation information is obtained, and the bit allocation to each color is determined based on the bit allocation information. The bit allocation information, for example, is shown in the following table, which indicates the range ratio and bit set in advance for each code. Relationship between meta-allocations. The following table shows an example of a part of the bit allocation information. More specifically, for example, in the case of code 1, when the range ratio of pixel information (r1, r2, r3) is (2, 5, 4096), it is bit allocation information (2, 4, 4096). 5 at r2 is 4 or more of r2 corresponding to the bit allocation information, so based on the bit allocation information in the first row in the table below, the bit allocation of (b1, b2, b3) is set to (1, 2, 12) Wait. The following table is only an example of bit allocation information and is not limited to the following table. For example, each of the above codes or each code group (codes 1 to 4 and the like) may be set in advance. Thereby, the maximum range that can be represented by each bit number can be more balanced. Further, for example, a configuration may be adopted in which 1 is added to each range so that the above range (r1, etc.) does not become 0. In addition, in order to increase the weight of Y, it may be configured to be weighted to 3 times. Furthermore, in order to break the tie, G may be added to 1. Moreover, in the above description, the difference ratio between the maximum value and the minimum value is used as an example, and it is not limited to this.

[Table 1]

The present invention is not limited to the above-mentioned embodiment, and may be replaced with a structure that is substantially the same as the structure shown in the above-mentioned embodiment, a structure that achieves the same effect, or a structure that can achieve the same purpose. And, for example, in an 8K television, the image processing device may be applied when encoding / decoding image information.

The present invention is not limited to the above-mentioned embodiments, and various changes can be made within the scope shown in the patent application scope. Embodiments obtained by appropriately combining technical means disclosed in different embodiments are also included in the technical scope of the present invention. Furthermore, new technical features can be formed by combining the technical methods disclosed in the respective embodiments.

11‧‧‧Image processing device

12‧‧‧ Block Division

13‧‧‧Mapping Information Acquisition Department

14‧‧‧Reference pixel information selection section

15‧‧‧Map selection department

16‧‧‧ Quantum Department

17‧‧‧Storage Department

18‧‧‧ Computing Department

19‧‧‧ Coding Department

FIG. 1 is an explanatory diagram of a configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 2 is a schematic diagram showing an example of inputted image information.

FIG. 3 is a schematic diagram showing an example of the structure of a block that is divided into blocks.

FIG. 4A is a schematic diagram showing an example of mapping information.

FIG. 4B is a schematic diagram illustrating another example of the mapping information.

FIG. 4C is a schematic diagram showing another example of the mapping information.

FIG. 4D is a schematic diagram showing another example of the mapping information.

FIG. 4E is a schematic diagram showing another example of the mapping information.

FIG. 4F is a schematic diagram showing another example of the mapping information.

FIG. 4G is a schematic diagram showing another example of the mapping information.

FIG. 4H is a schematic diagram showing another example of the mapping information.

FIG. 5 is a schematic diagram showing an example of a flow of an embodiment of the present invention.

Claims (12)

An image processing device, comprising: Blocking Division, which divides image information into multiple blocks; The mapping information obtaining unit is configured to obtain mapping information, the mapping information is a plurality of reference pixel information indicating a reference pixel to be referred to by the target pixel, and position information and Information associated with the object pixel in the block; The reference pixel information selection unit selects one reference pixel information among the plurality of reference pixel information included in the mapping information based on the pixel value of the target pixel and the pixel value of the reference pixel. For example, the image processing device according to the first patent application range, wherein: The reference pixel information selection unit selects the one reference pixel information based on a difference between a pixel value calculated from the pixel value of the reference pixel and a pixel value of the target pixel. For example, the image processing device according to the first or second patent application scope, wherein: The number of reference pixels to which the target pixel refers is plural. For example, the image processing device of the third scope of the patent application, wherein: The pixel value calculated based on the pixel value of the reference pixel is a pixel value of the plurality of reference pixels based on the distance between the plurality of reference pixels and the target pixel. For example, the image processing device according to the first patent application range, wherein: The mapping information acquisition unit obtains a plurality of mapping information. The image processing apparatus further includes a mapping selection unit that selects one of the plurality of mapping information based on the replacement block and the block before the replacement. The replacement block is based on the reference pixel information. The pixel value calculated by referring to the pixel information selected by the selection unit is replaced. For example, the image processing device according to the scope of patent application No. 5, wherein: The plurality of mapping information includes bit width information, and the bit width information indicates a bit width for quantizing a pixel value of a pixel other than the target pixel; The image processing apparatus further includes a quantization unit that quantizes the block based on the selected mapping information. For example, the image processing device of the sixth scope of the application for patent, wherein: The bit width information is different according to the number of the target pixels included in the mapping information. For example, the image processing device in the seventh scope of the application for a patent, wherein: The quantization unit performs quantization based on a first value and a second value among pixel values of pixels in the block. For example, the image processing device in the eighth aspect of the patent application, wherein: The first value is the maximum value of the pixel values of the pixels in the block, and the second value is the minimum value of the pixel values of the pixels in the block. For example, the image processing device according to the first patent application range, wherein: The image processing apparatus further includes a bit allocation unit that determines a bit allocation of each color based on a range ratio of the first value and the second value of each color of each pixel, and assigns the bit allocation to each of the colors. Color allocation bits. An image processing method includes the following steps: Split image information into multiple blocks; Obtain mapping information, the mapping information is related to a plurality of reference pixel information indicating a reference pixel to be referred to by the target pixel, and position information indicating a positional relationship between the target pixel and the reference pixel is related to the target pixel in the block Information; and Based on the pixel value of the target pixel and the pixel value of the reference pixel, one of the reference pixel information is selected from the plurality of reference pixel information included in the mapping information. An image processing program is characterized in that a computer is used as a block section, a mapping information acquisition section, and a reference pixel information selection section to implement the functions of the following sections: The block division section divides the image information into a plurality of blocks; The mapping information obtaining unit is configured to obtain mapping information. The mapping information is a plurality of reference pixel information indicating a reference pixel to be referred to by the target pixel, and position information indicating a positional relationship between the target pixel and the reference pixel. Information associated with the target pixel within the block; and The reference pixel information selecting unit selects one reference pixel information among the plurality of reference pixel information included in the mapping information based on the pixel value of the target pixel and the pixel value of the reference pixel.