JP2021061487A - Image encoding device and image encoding method - Google Patents

Abstract

To provide an image encoding device and an image encoding method that improve an encoding performance of an image containing a defective pixel.SOLUTION: In a configuration of an encoding unit of the image encoding device, a division unit 201 divides an input image acquired by an input unit into a unit of pixel block. An encoding unit 202 identifies a position of a defective pixel in a selected pixel block using "position information on the defective pixel" and performs a process of replacing a pixel value of the pixel (defective pixel) at the identified position with another pixel value or excluding the pixel from the input image to perform encoding so that a length of code of the pixel block is a fixed length.SELECTED DRAWING: Figure 2

Description

The present invention relates to an image coding technique.

In imaging devices such as digital cameras, the amount of data in captured images is increasing due to higher resolution and higher gradation of captured images. On the other hand, since the band of the interface output from the image sensor (image sensor) to the image processing unit is narrow, a coding technique that guarantees transmission within the limited band is required.

In addition, noise may occur in the image captured by using the image sensor due to scratches on the lens, defects in the pixels of the image sensor, and the like. Pixels at noise points are hereinafter referred to as defective pixels. When defective pixels are present in the captured image to be coded, the change in pixel value between the defective pixels and the pixels other than the defective pixels is large, so that the coding performance of the captured image is deteriorated.

Conventionally, in order to suppress deterioration of coding performance due to defective pixels, edge determination is performed for each area, and in an area where edge pixels and flat pixels are mixed, edge pixels and flat pixels are used. There is a method of coding by changing the quantization width (Patent Document 1). In this method, an image is divided into small areas and encoded, edge determination is performed in each area, and an edge flag is added to each pixel in a region in which edge pixels and flat pixels are mixed. By setting the quantization width of the edge pixel to be large and the quantization width of the flat pixel to be small, the coding distortion is suppressed.

In addition, when frequency conversion is performed in block units to encode the conversion coefficient, in blocks with defective pixels, the defective pixels can be made inconspicuous by coding using a different quantization table. There is a method of correction (Patent Document 2).

Japanese Unexamined Patent Publication No. 2012-54882 Japanese Unexamined Patent Publication No. 11-103413

In the prior art, the deterioration of the coding performance due to the defective pixel is reduced by changing the coding parameter around the defective pixel, but the influence of the pixel value of the defective pixel is not eliminated. In general, defective pixels are corrected by replacing pixel values, so the pixel values of defective pixels often do not make sense. The present invention provides a technique for reducing a decrease in coding performance for an image even when the image including defective pixels is encoded.

The uniformity of the present invention is an input means for inputting an input image and a process of replacing the pixel value of a pixel designated as a defective pixel in the input image with another pixel value or excluding the pixel from the input image. It is characterized in that it is provided with a coding means for encoding the input image after the processing.

According to the configuration of the present invention, even in the case of coding an image including defective pixels, it is possible to reduce the deterioration of the coding performance for the image.

The block diagram which shows the functional structure example of an image coding apparatus. The block diagram which shows the structural example of the coding unit 103. Flowchart of fixed length coding of input image. The figure explaining the pixel block. The figure which shows the index n, the pixel value of a pixel OP, the difference value Q, fixed-length coded data, and 1 bit to be truncated. The figure which shows the relationship between a block range BR and a fixed length FL. The figure explaining the process in step S302. The figure which shows the relationship between an input image and a pixel block. The figure which shows the structural example of the code string. The figure which represented the change of the pixel value in a pixel block schematically. Flowchart of fixed length coding of input image. A block diagram showing a hardware configuration example of a computer device.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiment, not all of the plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the attached drawings, the same or similar configurations are designated by the same reference numbers, and duplicate explanations are omitted.

[First Embodiment]
In the present embodiment, the captured image output from the image sensor is divided into pixel blocks composed of a plurality of pixels, and the divided pixel blocks are coded with a fixed length. In this fixed-length coding, in order to improve the coding performance, the pixel value of the defective pixel is replaced with another pixel value by using the position information indicating the position of the defective pixel in the captured image, and then the pixel block is used. Is fixed-length coding. It should be noted that the position information of the defective pixel in the captured image will be described as being shareable as known information between the coding side and the decoding side.

First, a functional configuration example of the image coding apparatus according to the present embodiment will be described with reference to the block diagram of FIG. The image coding device according to the present embodiment is mounted on an image pickup device such as a digital camera, and the image coding device acquires an image captured image output from an image pickup device included in the image pickup device as an input image, and the image coding device is said to be the input image. The input image is coded with a fixed length in units of pixel blocks.

The input unit 101 acquires an captured image (captured by the image sensor) output from the image sensor as an input image. In the present embodiment, the input image acquired by the input unit 101 is a 14-bit RAW image having a Bayer array of R / G0 / G1 / B (the pixel value of each pixel of the RAW image is represented by 14 bits). is there. The image coding device encodes such an input image as a monochrome image for each color component. However, the image coding apparatus is not limited to encoding such a RAW image as a monochrome image for each color component, and encodes an image using another color representation as a monochrome image for each color component. You may. For example, an image in which each of RGB is represented by 8 bits and a luminance color difference component in which one pixel is represented by 8 bits each of YCbCr may be similarly encoded as a monochrome image for each color component. good.

The acquisition unit 102 acquires position information indicating the position of a defective pixel (a pixel designated as a defective pixel in the input image) in the input image. The acquisition method of the position information of the defective pixel is not limited to a specific acquisition method. For example, the acquisition unit 102 may acquire the position information of the defective pixel from an external device different from the image coding device, or may acquire the position information of the defective pixel from the memory of the image coding device or the image pickup device. good.

The configuration of the defective pixel position information is not limited to the specific configuration as long as the position of the defective pixel in the input image can be specified. For example, as a flag value corresponding to each pixel in the input image, a binary data group having a value "1" if the pixel is a defective pixel and a value "0" if the pixel is not a defective pixel is "defect". It may be "pixel position information".

The coding unit 103 uses the position information of the defective pixel acquired by the acquisition unit 102 to encode the input image acquired by the input unit 101 with a fixed length for each pixel block. A configuration example of the coding unit 103 will be described with reference to the block diagram of FIG.

The dividing unit 201 divides the input image acquired by the input unit 101 in units of pixel blocks. In the present embodiment, the vertical size (number of pixels in the vertical direction) of the input image is H, the size in the horizontal direction (number of pixels in the horizontal direction) is W, and the vertical size of the pixel block (number of pixels in the vertical direction). ) Is Bh, and the size in the horizontal direction (the number of pixels in the horizontal direction) is Bw. For convenience of explanation, H is an integral multiple of Bh and W is an integral multiple of Bw. That is, when the input image is divided into pixel blocks, incomplete pixel blocks (pixel blocks smaller in size than pixel blocks having a vertical size of Bh and a horizontal size of Bw) do not occur. To do. In the following, Bw = Bh = 4 as an example.

The relationship between the input image and the pixel block is shown in FIG. As shown in FIG. 8, the i-th pixel block in the horizontal direction (right direction) from the left end of the input image and the j-th pixel block in the vertical direction (downward direction) from the upper end of the input image are referred to as B (i, j). That is, as shown in FIG. 8, the pixel block in the upper left corner of the input image is described as B (0,0), and the pixel block adjacent to the right of the pixel block B (0,0) is described as B (1,0). The pixel block adjacent to the pixel block B (0,0) is referred to as B (0,1). Further, when the position of the upper left corner of the pixel block B (i, j) is set to the origin (0,0), the pixel at the position (m, n) in the pixel block B (i, j) is Bij (m, m, Notated as n). For example, the pixel (first pixel) in the upper left corner of the pixel block B (2, 3) is described as B23 (0,0).

The coding unit 202 encodes each pixel block divided by the dividing unit 201 so that the code length of the pixel block becomes a fixed length FL by using "position information of defective pixels" (fixed length coding). )I do.

The fixed length FL is a parameter that controls the maximum code length of the pixel block. In the present embodiment, the pixels constituting the pixel block include pixels that are held as they are without compression and pixels that are represented by a variable length code having a fixed length or less. The upper limit of the number of bits of the variable length code in the pixel block is defined as the fixed length FL, which will be described below. In the present embodiment, the fixed length FL is the same value for all pixel blocks, but the value is not limited to this, and may be changed for each pixel block. However, it is necessary to set the fixed length FL for each pixel block so as not to exceed the target code amount for the input image. The fixed value FL may be set in the image coding device by the user operating an interface provided in the image coding device or the imaging device, or may be set in the image coding device in advance. ..

The fixed-length coding of the input image by the coding unit 202 and the coding unit 203 will be described with reference to the flowchart of FIG. In the following, among the pixels constituting the pixel block, pixels that are not defective pixels will be referred to as effective pixels.

In step S301, the coding unit 202 selects one of the plurality of pixel blocks generated by the division by the dividing unit 201 as the selected pixel block from the pixel blocks that have not yet been fixed-length coded. Then, the coding unit 202 determines whether or not the selected pixel block contains defective pixels by using "position information of defective pixels". That is, when one or more of the positions of the defective pixels indicated by the "position information of the defective pixels" are included in the selected pixel block, the coding unit 202 contains one or more defective pixels in the selected pixel block. Judge that it is included. On the other hand, when none of the positions of the defective pixels indicated by the "position information of the defective pixels" is included in the selected pixel block, the coding unit 202 includes one defective pixel in the selected pixel block. Judge that there is no.

For example, when the "position information of the defective pixel" is the above-mentioned binary data group, the flag value corresponding to each pixel of the selected pixel block is referred to in the binary data group. Then, when one or more flag values having a value of "1" are included in the referenced flag values, it is determined that the selected pixel block contains one or more defective pixels. On the other hand, if all the referenced flag values are "0", it is determined that the selected pixel block does not include any defective pixels.

As a result of such a determination, if the selected pixel block contains one or more defective pixels, the process proceeds to step S302, and if the selected pixel block does not contain any defective pixels, the process proceeds. Proceeds to step S303.

In step S302, the coding unit 202 identifies the position of the defective pixel in the selected pixel block using "position information of the defective pixel", and sets the pixel value of the pixel (defective pixel) at the specified position to another pixel. Replace with a value. There are various methods for replacing the pixel value of a defective pixel with another pixel value.

For example, the coding unit 202 replaces the pixel value of the defective pixel in the selected pixel block with the pixel value of the effective pixel located in the vicinity of the defective pixel in the selected pixel block. For example, as shown in FIG. 7, it is assumed that the pixel indicated by “x” in the 4 × 4 selected pixel block is a defective pixel. At this time, the coding unit 202 searches for the pixel adjacent to the left of the defective pixel, the pixel adjacent to the right, the pixel adjacent to the top, and the pixel adjacent to the bottom in this order, and determines the pixel value of the defective pixel. The pixel value of the effective pixel first found in the search is used for replacement. If no effective pixel is found in this search, the priority order of the pixel position of the pixel value replacement source, such as diagonally upward to the right or diagonally upward to the left of the defective pixel, is determined in advance, and the search for the priority order is performed first. The pixel value of the defective pixel is replaced with the pixel value of the found effective pixel. If all the pixels in the selected pixel block are defective pixels, the pixel values of all the defective pixels in the selected pixel block are replaced with fixed values such as "0".

In this way, the coding unit 202 replaces the pixel values of all the defective pixels in the selected pixel block with other pixel values. Then, when the process of replacing the pixel values of all the defective pixels in the selected pixel block with other pixel values is completed (after the process), the process proceeds to step S303.

In step S303, the coding unit 202 specifies the maximum pixel value and the minimum pixel value among the pixel values of the pixels constituting the selected pixel block, and the pixel having the maximum pixel value is the maximum pixel Pmax. The pixel having the minimum pixel value is defined as the minimum pixel Pmin. Various processes can be applied to the process for specifying (searching) the maximum pixel Pmax and the minimum pixel Pmin from the selected pixel block. An example thereof will be described below.

As shown in FIG. 4A, the pixels in the selected pixel block B (i, j) are ranked from 0 to n in a predetermined scan order (for example, raster scan), and the pixels in the order are ordered from the first pixel. It is expressed as P0, P1, ... Pn. In the present embodiment, the number of pixels of the selected pixel block is 4 × 4 = 16, so n = 15. An example of each pixel value of the pixels P0 to Pn is shown in FIG. 4 (b). In the case of FIG. 4B, since the minimum pixel value among the pixel values of pixels P0 to Pn is “12” of pixel P0 and the maximum pixel value is “257” of pixel P7, pixel P0 is the minimum. It is specified as the pixel Pmin, and the pixel P7 is specified as the maximum pixel Pmax. Further, among the pixels P0 to Pn, the other pixels excluding the maximum pixel Pmax and the minimum pixel Pmin are expressed as OP1, OP2, OP3, ... OP14 in order from the beginning in the above order.

In step S304, the coding unit 202 encodes the minimum pixel Pmin and the maximum pixel Pmax. In the present embodiment, the pixel position and pixel value of the minimum pixel Pmin and the pixel position and pixel value of the maximum pixel Pmax are held as code data having a fixed length, respectively. As for the pixel position, since the size of the pixel block is 4 × 4, the pixel position can be expressed by 4 bits. Here, a natural binary representation is used, but another binary representation such as a police box binary may be used as long as it can be uniquely decoded. In the example of FIG. 4, since the pixel position of the minimum pixel Pmin is 0, it is expressed as "0000", and since the pixel position of the maximum pixel Pmax is 7, it is expressed as "0111". Pixel values are left uncompressed. That is, since the bit accuracy of the pixel value of each pixel of the input image is 14 bits, the pixel value is represented by 14 bits. However, the pixel values of the minimum pixel Pmin and the maximum pixel Pmax do not necessarily have to be uncompressed. For example, the pixel values of the minimum pixel Pmin and the maximum pixel Pmax may be expressed by bits smaller than the number of bits of the input pixel as long as the method can tolerate some deterioration. Then, the coding unit 202 stores the coding result of the minimum pixel Pmin and the coding result of the maximum pixel Pmax in the buffer 204.

In step S305, the coding unit 202 obtains the difference value DIFF which is the result of subtracting the pixel value of the minimum pixel Pmin from the pixel value of the maximum pixel Pmax. Then, the coding unit 202 obtains the bit range (block range of the selected pixel block) necessary for expressing the obtained difference value DIFF as the block range BR.

In the case of FIG. 4, since the pixel value of the minimum pixel Pmin is "12" and the pixel value of the maximum pixel Pmax is "257", the difference value DIFF is 257-12 = 245. That is, the pixel values of all the pixels in the selected pixel block are any of 246 gradations between the pixel value of the minimum pixel Pmin and the pixel value of the maximum pixel Pmax, and are from the minimum value (or maximum value). It can be expressed as a difference. Since 8 bits are required to express the maximum value (= DIFF = 245) of the difference from the minimum value (= 0) of the difference, the block range BR = 8.

In step S306, the coding unit 202 determines whether or not the block range BR is larger than 0. As a result of this determination, when the block range BR is larger than 0, the process proceeds to step S307 in order to encode other pixel OPs other than the minimum pixel Pmin and the maximum pixel Pmax in the selected pixel block. On the other hand, when the block range BR is 0, this means that all the pixels in the selected pixel block have the same pixel value, so it is necessary to encode other pixel OPs except the minimum pixel Pmin and the maximum pixel Pmax. Absent. However, in such a case, the coding process (steps S307 to S308) related to the pixel OP is skipped, and the process proceeds to step S309.

In step S307, the coding unit 202 encodes the pixel OP with a fixed length FL or less. Here, the pixel OP is each of the above OP1, OP2, OP3, ... OP14. When the block range BR of the selected pixel block is the fixed length FL or less, the coding unit 202 subtracts the pixel value of the minimum pixel Pmin from the pixel value of the pixel OPn (n = 1, 2, 3, ..., 14). The value Qn is output as the code data of the pixel OPn. On the other hand, when the block range BR of the selected pixel block is larger than the fixed length FL, the coding unit 202 first obtains a difference value Qn obtained by subtracting the pixel value of the minimum pixel Pmin from the pixel value of the pixel OPn. Then, the coding unit 202 generates the code data of the pixels OP1 to OP14 by truncating the lower bits so that Q1 to Q14 can be represented by the fixed length FL.

In the case of FIG. 4B, the block range BR = 8. Assuming that the fixed length FL = 7, by truncating the lower 1 bit of Q1 to Q14, the pixel value of the pixels OP1 to OP14 becomes a value that can be expressed by 7 bits. For example, by shifting Q1 to the right by 1 bit, the code data "26" (the actual code is "0011010" which expresses this in 7-bit natural binary) is generated. Similarly, the other Q2 to Q14 are shifted right by 1 bit to generate code data.

FIG. 5 shows the index n of the pixel OP, the pixel value of the pixel OP corresponding to the index n, the difference value Q obtained by subtracting the pixel value of the minimum pixel Pmin from the pixel value of the pixel OP, and the 8 bits constituting the difference value Q. It shows 7 bits as the code data (fixed length coded data) of the pixel OP and 1 bit (overbit coded data of the pixel OP) to be truncated.

Then, the coding unit 202 stores the coding result of the pixel OP in the buffer 204.

In step S308, the coding unit 202 determines whether or not the block range BR is equal to or less than the fixed length FL. As a result of this determination, if the block range BR is equal to or less than the fixed length FL, the process proceeds to step S309, and if the block range BR is larger than the fixed length FL, the process proceeds to step S310.

In step S310, the coding unit 203 encodes the bit truncated in step S307, that is, the lower (BR-FL) bit of the difference value Q as an overbit. In the overbit coding, the overbits of Q1 to Q14 may be encoded by run-length coding or the like in units of bit planes composed of bits at the same bit positions, or the overbits may be retained as they are. In the subsequent code amount control process, the data amount can be compared with the pool capacity described later, and the amount of data determined to be able to be added by the code amount control from the overbits is held in a form that can be added. I just need to be there.

Here, the relationship between the block range BR and the fixed length FL will be described with reference to FIG. 6A, 6B, and 6C show examples of three pixel blocks having different characteristics.

The pixel block shown in FIG. 6A is a pixel block in which the change in the pixel value in the pixel block is small and the block range BR is small (BR = 2). The pixel block shown in FIG. 6B is a pixel block in which the change in the pixel value in the pixel block is gentle and the block range BR is larger than that of the pixel block in FIG. 6A (BR = 7). The pixel block shown in FIG. 6 (c) is a pixel block in which the change in the pixel value in the pixel block is large and the block range BR is larger than that of the pixel block in FIG. 6 (b) (BR = 14).

Assuming that the fixed length FL = 7, the relationship between the block range BR of the pixel blocks of FIGS. 6 (a) to 6 (c) and the fixed length FL is as shown in FIGS. 6 (d) to 6 (f), respectively. Become a relationship.

In the case of the pixel block of FIG. 6A, since BR (= 2) <FL (= 7), the coding unit 202 encodes the pixel block with a fixed length of FL or less without information loss. In the case of the pixel block of FIG. 6B, since BR (= 7) = FL (= 7), the coding unit 202 encodes the pixel block with a fixed length of FL or less without information loss. In the case of the pixel block of FIG. 6C, since BR (= 14)> FL (= 7), the BP = (BR-FL) portion of the difference value Q exceeds the fixed length FL. Therefore, in such a case, the upper FL bit of the difference value Q is encoded by the coding unit 202, and the remaining lower BP bits are coded by the coding unit 203.

Then, the coding unit 203 stores the overbit coding result in the buffer 204.

In step S309, the coding unit 202 calculates the surplus data capacity (pool capacity) SD. The surplus data capacity SD refers to a capacity that does not need to be used when the block range BR is smaller than the fixed length FL. That is, (FL-BR) is the surplus data capacity SD for the block. In the example of FIG. 6, SD = 7-2 = 5 in the case of the pixel block of FIG. 6 (a), and SD = 7-7 = 0 in the case of the pixel block of FIG. 6 (b).

Since the pixel block targeted in step S309 satisfies the block range BR ≦ fixed length FL, it can be said that the pixel block can be losslessly coded at the fixed length FL or less. Further, the pixel block satisfying the block range BR <fixed length FL is a pixel block capable of lossless coding without utilizing the entire fixed length FL. Therefore, when the surplus data capacity SD is larger than 0, it can be assigned to the code data of the pixel block of the block range BR that exceeds the fixed length FL.

At the start of the process according to the flowchart of FIG. 3, SD is initialized to 0, and in step S309, the SD value is updated by adding (FL-BR) to the current SD value.

In the present embodiment, the surplus data capacity SD is defined as described above in order to compare the surplus data capacity SD and the overbit coded data in pixel block units. However, the definition of the surplus data capacity SD is not limited to the above definition as long as the surplus data capacity SD and the overbit coded data can be compared. For example, as the surplus data capacity SD for the actual pixel block, it means that there is a surplus of (FL-BR) bits for 14 pixels which are pixel OPs other than the minimum pixel and the maximum pixel, and therefore (FL-BR) ×. 14 may be used as the surplus data capacity SD.

In step S311, the coding unit 202 determines whether or not all the pixel blocks in the input image have been selected as the selected pixel blocks (whether or not the processing of steps S301 to S310 has been performed for all the pixel blocks in the input image). To do.

As a result of this determination, when all the pixel blocks in the input image are selected as the selected pixel blocks (the processes of steps S301 to S310 are performed for all the pixel blocks in the input image), the process proceeds to step S312. On the other hand, if a pixel block that has not yet been selected as a selected pixel block remains in the input image (a pixel block that has not yet been processed in steps S301 to S310 remains in the input image), the process is performed in step S301. Proceed to.

In step S312, the code amount control unit 205 performs the code amount control process of the input image. In the code amount control process, the code amount control unit 205 extracts partial data for the surplus data capacity SD from the overbit coding result (overbit coded data) stored in the buffer 204. Then, the code amount control unit 205 stores the partial data in the buffer 204 as coded data to be included in the code string described later. When the amount of overbit coded data is smaller than or the same as the surplus data capacity SD, all the overbit coded data is stored in the buffer 204 as coded data to be included in the code string described later. On the other hand, when the amount of overbit coded data is larger than the surplus data capacity SD, the buffer 204 includes the partial data of the surplus data capacity SD in the overbit coded data as the coded data to be included in the code string described later. Store in. Then, the code amount control unit 205 outputs each coding result stored in the buffer 204.

By performing the above coding process and the code amount control process in this way, it is possible to code each code amount control unit with a fixed length or less. FIG. 10 shows a diagram schematically showing the change of the pixel value in the pixel block. The horizontal axis represents the pixel position and the vertical axis represents the pixel value. In a pixel block having a pixel value as shown in FIG. 10A, the pixel value of the defective pixel (pixel position P (k)) is prominently larger than the pixel values of other pixels in the pixel block. have. Therefore, the value of the block range BR of this pixel block becomes large, and the coding performance of the pixel block deteriorates. By substituting the value of the defective pixel (pixel position P (k)) with the pixel value of the neighboring pixel and then encoding the pixel block, as shown in FIG. 10 (b), the block range BR The value becomes small, and the coding performance can be improved.

In the present embodiment, the pixel value of the defective pixel is replaced with the pixel value of the neighboring pixel, but the present invention is not limited to this. That is, it is sufficient if the minimum pixel value and the maximum pixel value can be searched in the pixel block with the defective pixel removed. Therefore, the pixel value of the defective pixel is set to the pixel value of the minimum pixel and the pixel value of the maximum pixel. It may be replaced with a pixel value other than. Therefore, the value to be replaced may be determined in advance to set the minimum value excluding the defective pixels in the pixel block.

The code string forming unit 104 forms a code string by adding necessary additional information to the coding result (coded data having a fixed length or less) output from the coding unit 103 (code amount control unit 205). A configuration example of the code string is shown in FIG.

At the beginning of the code string, information necessary for decoding the input image, for example, the number of pixels in the horizontal direction, the number of pixels in the vertical direction, the number of components, the number of bits of each component and the width of the pixel block, Additional information such as height is added as a header. In the present embodiment, the pixel block position can be specified at the time of decoding by adding the number of pixel blocks and the coding order of the pixel blocks in the code amount control area unit as the header. Following the header, fixed-length coded data for each code amount control unit is added.

Although it has been explained that the coding order of the pixel blocks is added as the header, the present invention is not limited to this. For example, when encoding one frame of a moving image, if the difference from the immediately preceding frame is small instead of the first frame, it may be decided to encode in the same order as the immediately preceding frame. In that case, the header may have a flag as to whether or not to encode in the same order as the immediately preceding frame.

The pixel values of defective pixels are replaced at the time of coding, but such pixels are generally subjected to interpolation processing from peripheral pixels. Therefore, it is not necessary for the decoding side to decode the pixel value of the original defective pixel. Therefore, it is not necessary to input the position information of the defective pixel on the decoding side, and the coded data to be decoded may be decoded as it is.

The output unit 105 outputs the code string formed by the code string forming unit 104. The output destination of the code string is not limited to a specific output destination, and may be a memory of an image coding device or an image pickup device, or an external device capable of communicating with the image pickup device.

In the present embodiment, the position information of the defective pixel has been described as the information created in advance, but the present invention is not limited to this, and the position information is updated by detecting the later defective pixel generated due to aged deterioration or the like. It doesn't matter.

Further, in the present embodiment, the method of dividing the input image into pixel blocks and then encoding the input image with a fixed length has been described, but the present invention is not limited to this. For example, a method such as performing predictive coding on the input image may be used. Specifically, even when the prediction error value is calculated with the pixel immediately before the pixel of interest as the prediction and entropy coding is performed, the prediction error is made around the defective pixel by replacing the defective pixel and coding. Can be suppressed from increasing and reducing the coding efficiency. Further, as another example, a coding method that performs frequency conversion using a DCT or the like may be used. Even in this case, it is possible to suppress the deterioration of the compression performance due to the generation of high-frequency components around the defective pixel. As described above, according to the present embodiment, even in the case of coding the input image including the defective pixel, it is possible to reduce the deterioration of the coding performance for the input image.

[Second Embodiment]
In each of the following embodiments including the present embodiment, the differences from the first embodiment will be described, and unless otherwise specified below, the same as the first embodiment. In the first embodiment, the configuration in which the pixel value of the defective pixel in the input image is replaced with another pixel value and then coded has been described, but in the present embodiment, the defective pixel in the input image is excluded and then coded. The configuration to be converted will be described.

The fixed-length coding of the input image by the coding unit 103 according to the present embodiment will be described with reference to the flowchart of FIG. In FIG. 11, the same processing steps as those in FIG. 3 are assigned the same step numbers, and the description of the processing steps will be omitted.

If it is determined in step S301 that the selected pixel block contains one or more defective pixels, the process proceeds to step S1101 and if it is determined that the selected pixel block does not contain any defective pixels. The process proceeds to step S303.

In step S1101, the coding unit 202 specifies the position of the defective pixel in the selected pixel block using "position information of the defective pixel", and excludes the pixel (defective pixel) at the specified position from the coding target. Perform the processing for. For example, the coding unit 202 updates the selected pixel block by omitting defective pixels from the selected pixel block. That is, the number of pixels constituting the selected pixel block is reduced by the number of defective pixels.

In the case of the present embodiment, it is necessary to acquire the position information of the defective pixel acquired at the time of coding also at the time of decoding to determine whether or not the defective pixel exists in the decoding target block as in the coding process. There is. Then, when the defective pixel exists, the coded data of the defective pixel position does not exist, so only the number of pixels excluding the number of defective pixels is decoded from the coded data. For the pixel at the defective pixel position, the pixel value of the neighboring pixel may be set as the decoding value, or a predetermined value may be set as the decoding value. For example, the pixel value of the defective pixel position is decoded by giving "0" as a decoding value.

As described above, according to the present embodiment, even in the case of coding the input image including the defective pixel, it is possible to reduce the deterioration of the coding performance for the input image. Further, by acquiring the position information of the defective pixel on the decoding side as well, the decoding can be performed without adding the information of the defective pixel to the coded data.

[Third Embodiment]
In the first and second embodiments, the image coding device has been described as being mounted on an image pickup device such as a digital camera. However, the device equipped with the image coding device is not limited to the imaging device. For example, an image coding device may be mounted on a device having an imaging function (smartphone, tablet terminal device, notebook computer, etc.). In this case, the captured image captured by the imaging function is used as an input image. Will be entered in. The image coding device may directly acquire the captured image from an imaging unit such as an image sensor, or may acquire the captured image once stored in the memory.

Further, the image coding device is not limited to being mounted on any device. For example, the image coding device may be a computer device to which an imaging device is connected, or may be a computer device that holds an input image by itself. Further, the image coding device may be a computer device capable of communicating with an external device (for example, a file server on a network) that holds an input image.

An example of a hardware configuration of a computer device applicable to an image coding device will be described with reference to the block diagram of FIG. The hardware configuration of the computer device applicable to the image coding device is not limited to the configuration shown in FIG. 12, and can be appropriately changed / modified.

The CPU 1201 executes various processes using computer programs and data stored in the RAM 1202 and the ROM 1203. As a result, the CPU 1201 controls the operation of the entire computer device, and executes or controls each of the above-described processes as performed by the image coding device.

The RAM 1202 is an area for storing computer programs and data loaded from the ROM 1203 and the external storage device 1206, and is for storing data (captured image, position information of defective pixels, etc.) received from the outside via the I / F 1207. Has an area of. The RAM 1202 also has a work area used by the CPU 1201 to execute various processes. As described above, the RAM 1202 can appropriately provide various areas. The ROM 1203 stores computer device setting data, a startup program, and the like.

The operation unit 1204 is a user interface such as a keyboard, a mouse, and a touch panel, and various information (position information of defective pixels, etc.) and instructions can be input to the computer device by the user's operation.

The display unit 1205 has a liquid crystal screen and a touch panel screen, and can display the processing result by the CPU 1201 with images, characters, and the like. The display unit 1205 may be a projection device such as a projector that projects an image or characters.

The external storage device 1206 is a large-capacity information storage device such as a hard disk drive device. The external storage device 1206 stores computer programs and data for causing the CPU 1201 to execute or control each of the above-mentioned processes as performed by the OS (operating system) and the image coding device.

The computer program stored in the external storage device 1206 includes a computer program for causing the CPU 1201 to execute the functions of the functional units other than the buffer 204 in FIGS. 1 and 2. Further, the data stored in the external storage device 1206 includes the data described as known information in the above description (position information of defective pixels, etc.).

The computer programs and data stored in the external storage device 1206 are appropriately loaded into the RAM 1202 according to the control by the CPU 1201, and are processed by the CPU 1201. The buffer 204 of FIG. 2 can be mounted by the above-mentioned RAM 1202 or the external storage device 1206.

The I / F 1207 is a network interface used by a computer device to perform data communication with an external device. The captured image and the position information of the defective pixel may be acquired from the external device via the I / F 1207 to the external storage device 1206 or the RAM 1202, or the code string may be transmitted to the external device via the I / F 1207.

The CPU 1201, RAM 1202, ROM 1203, operation unit 1204, display unit 1205, external storage device 1206, and I / F 1207 are all connected to the bus 1208.

The computer device having the configuration shown in FIG. 12 is also applicable to an image decoding device that decodes a code string generated by the image coding device. The image coding device and the image decoding device may be separate devices, or may be the same device (a device having a coding function and a decoding function).

It should be noted that the specific numerical values used in the above description are used for giving specific explanations, and are not intended to limit each of the above embodiments to these numerical values. In addition, some or all of the above-described embodiments may be combined as appropriate. In addition, a part or all of each of the above-described embodiments may be selectively used.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The invention is not limited to the above embodiments, and various modifications and modifications can be made without departing from the spirit and scope of the invention. Therefore, a claim is attached to make the scope of the invention public.

101: Input unit 102: Acquisition unit 103: Coding unit 104: Code string forming unit 105: Output unit 201: Dividing unit 202: Coding unit 203: Coding unit 204: Buffer 205: Code amount control unit

Claims (14)

Input means for inputting input images and
A coding means for encoding the input image after the processing of replacing the pixel value of the pixel designated as a defective pixel in the input image with another pixel value or excluding the pixel from the input image. An image coding apparatus comprising the above. The coding means is
The image coding apparatus according to claim 1, wherein the input image after the processing is coded by predictive coding. The coding means is
The image coding apparatus according to claim 1, wherein the input image after the processing is frequency-converted and coded. In addition
The input image is provided with a dividing means for dividing a pixel block composed of a plurality of pixels into units.
The coding means performs a process of replacing the pixel value of a pixel designated as a defective pixel in the pixel block with another pixel value or excluding the pixel from the pixel block, and the pixel block after the process is used. The image coding apparatus according to any one of claims 1 to 3, wherein the image coding apparatus is encoded. The coding means is
A first coding means for encoding each of the maximum pixel having the maximum pixel value and the minimum pixel having the minimum pixel value in the processed pixel block,
The number of bits capable of expressing the difference between the pixel value of the maximum pixel and the pixel value of the minimum pixel is obtained, and when the number of bits is greater than 0 and the fixed length or less, the maximum number of pixels in the processed pixel block is obtained. The difference value between the pixel value of the pixel and the pixel excluding the minimum pixel and the pixel value of the maximum pixel or the pixel value of the minimum pixel is used as the encoding result of the pixel, and the number of bits is larger than 0 and the fixed length. When it is large, a second coding means in which the result of truncating the lower bits so that the difference value can be expressed by the fixed length is used as the coding result of the pixel,
The image coding apparatus according to claim 4, further comprising a third coding means for encoding the lower bits. 5. The second coding means is characterized in that when the number of bits is 0, the processed pixel block does not encode pixels other than the maximum pixel and the minimum pixel. The image encoding device according to. In addition
The image coding apparatus according to claim 5 or 6, further comprising an output means for forming and outputting a code string including a coding result by the coding means. The output means is characterized in that the code string including the amount of data based on the difference between the fixed length and the number of bits of the coding result by the third coding means is formed and output. Item 7. The image coding apparatus according to Item 7. The coding means according to claim 4 to 8, wherein the coding means replaces the pixel value of a pixel designated as a defective pixel in the pixel block with the pixel value of a pixel in the vicinity of the pixel in the pixel block. The image coding apparatus according to any one item. The coding means is characterized in that the pixel value of a pixel designated as a defective pixel in the pixel block is replaced with a pixel value other than the maximum pixel value and the minimum pixel value in the pixel block. The image coding apparatus according to any one of claims 4 to 8. The image coding apparatus according to any one of claims 1 to 10, wherein the input image is an captured image output from an image pickup device. The image coding device according to any one of claims 1 to 11, wherein the image coding device is mounted on an imaging device. This is an image coding method performed by an image coding device.
The input means of the image coding device includes an input step of inputting an input image and
The coding means of the image coding apparatus performs a process of replacing the pixel value of a pixel designated as a defective pixel in the input image with another pixel value or excluding the pixel from the input image, and the process is performed. An image coding method comprising a coding step of coding a later input image. A computer program for causing a computer to function as each means of the image coding apparatus according to any one of claims 1 to 12.

Images