JP5263328B2 - Color image sensor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a color imaging element which reduces false colors and has high-resolution. <P>SOLUTION: In a color imaging element, an arrangement combination section 204 performs a rearrangement so as to provide an interval for one pixel between pixels indicated by an RR' random arrangement pattern S21, performs a rearrangement so as to provide an interval for one pixel between pixels of a BB' random arrangement pattern S22, rearranges the regular arrangement of a GG' regular arrangement pattern S23 into a checkered arrangement and combines the arrangements to generate a color filter pattern S10 that is a partial random arrangement pattern. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a solid-state image sensor including a digital camera and a color image sensor using the solid-state image sensor.

For example, in a photographing apparatus using a single-plate solid-state imaging device, only a single spectral sensitivity can be obtained even if photographing is performed as it is. For this reason, a method of performing color photographing by attaching different color filters to a large number of pixels and arranging them in a specific pattern is common. The captured image can obtain only one color for each pixel, and an image having a mosaic shape with respect to the color is generated. However, by performing interpolation using color information obtained from surrounding pixels, it is possible to generate an image in which all the colors are aligned in all the pixels. Such interpolation processing is called color separation processing.
However, since each color is sampled discretely, if the captured image contains high-frequency components that exceed the Nyquist frequency, aliasing (folding of the high-frequency components back into the low-frequency region) occurs, and a different color is estimated. There was a problem that.

  In order to solve the above problems, inventions have been made to reduce the regularity of the color filter array and suppress the occurrence of false colors. In the pseudo-random Bayer array proposed by FillFactory (acquired by Cypress in 2004), G is arranged in a checkered pattern, and R and B are arranged pseudo-randomly in the remaining pixel positions. A three-color G checkered pseudorandom array is shown.

  Japanese Patent Laid-Open No. 2000-316169, “Hideaki Yoshida.“ Color Imaging Device and Color Imaging Device ”” shows a 6-color random array under the condition that it is adjacent to 6 color filters at 4 sides or 4 corners of the target pixel position. Has been.

  No. 0,804,037, (A2) “Mutze, Ulrich, Dr.“ Process and system for generating a full color image or multispectral image from the image data of a CCD image sensor with a mosaic color filter. ”” 5 An array with a 3x3 repeating pattern in color and an array with a pseudo-random pattern are shown.

In FillFactory's pseudo-random Bayer array, although the R and B arrays are pseudo-random, the repetition period is short, so the false color generation position in the spatial frequency domain is slightly different from Bayer, which substantially reduces false color. The latter two are more random than the former, so the effect of reducing false colors can be expected. However, because all colors are arranged randomly without distinction, G exists in checkered. However, there is a problem that it is disadvantageous as compared with the Bayer array and the pseudo-random Bayer array that can use the correlation processing shown in Japanese Patent No. 2931520.
In order to reduce false colors, it is necessary that some colors in the array are randomly arranged,
In order to improve the reproducibility of resolution, it is necessary that some colors of the array are regularly arranged.
In addition, in order to interpolate colors that do not exist in the target pixel from surrounding pixels, all colors must be present in the vicinity even if some colors are randomly arranged.
A method for generating an array that satisfies all the above conditions has not been known so far.

JP 2000-316169 EP Public Notice No. 0,804,037 (A2)

Conventionally, inventions have been made in which color filters (filters) used in a single-plate color image pickup device are randomly arranged. However, even if the arrangement can obtain a false color reduction effect, the resolution is lowered instead. Met.
SUMMARY OF THE INVENTION An object of the present invention is to provide a high-resolution color image sensor that reduces false colors.

In order to solve the above-described problems of the prior art and achieve the above-described object, the color image sensor of the first invention is a color image sensor having a pixel group in which a plurality of pixels each composed of a photoelectric conversion element are arranged. A color filter that combines a random array pattern and a regular array pattern, and the random array pattern has a desired frequency of color within an area of a prescribed size including the pixel position for an arbitrary pixel position. to fit in the error range, which are arranged some of the color of the pixel at random, the regular array pattern is obtained by arranged the remaining pixel colors regularly.

The term random in the present invention does not mean complete randomness.
When a completely random color arrangement is generated, the frequency of local colors is not always stable.
In the present invention, the color filter arrangement is a color filter arrangement in which the frequency of color existence within a specified size region including the pixel position of interest falls within a desired error range and the regularity is sufficiently lost. This is called a random arrangement, and a color filter arrangement in which colors are regularly arranged is called a regular arrangement.
Furthermore, a sequence formed by synthesizing a random sequence and a regular sequence is called a partially random sequence.
A region having a specified size is called a local region.
As for the local area, a single rectangular area may be designated simply, or a plurality of areas may be designated regardless of rectangular or non-rectangular.
When a plurality of areas are designated, an allowable error range having a different existence frequency may be defined for each area.

Also in the partially random arrangement obtained by the present invention, the color presence frequency in the local region including the target pixel position falls within the desired error range.
However, the local region in the partially random array and the local region in the random array are different, and the color existence frequency expected in each region is also different.
The partially random sequence is a regular combination of a random sequence and a regular sequence.
For this reason, when the frequency of the expected local color and the local region is specified for a part of the random array, the local region and the expected frequency of the color for the random array can be specified by considering the synthesis procedure. .
That is, the conditions required for the random array can be determined by defining the size of the local area and the color existence frequency sufficient to perform the color separation process on the partially random array.
A feature of some random arrays is that some of the colors in the array are randomly arranged, so that the effect of reducing false colors can be obtained, and some colors in the array are regularly arranged. The improvement of the reproducibility of resolution can be expected.

In the present invention, it is preferable that the generation method of a color filter array in which color filters are randomly arranged has a color presence frequency in an area of a specified size including the target pixel position for an arbitrary target pixel position. The target pixel position is randomly selected for the color filter array in which the colors are regularly arranged so as to fall within the desired error range, the color C1 at the target pixel position is replaced with another color C2, and the target pixel position is included. The frequency of existence of the color existing in the area of the specified size is checked, and if the existence frequency does not fall within the desired error range, the process of returning the color at the target pixel position to C1 is repeatedly performed. .
By using such a random color filter array generation method, color filters are randomly arranged so that the frequency of color presence within a specified size region including the target pixel position falls within a desired error range. A random color filter array can be generated.

In the present invention, it is preferable that the color filter array generation method in which color filters are randomly arranged has a white noise image having a continuous value or a discrete value represented by a sufficiently large number of quantization bits as a pixel value. On the other hand, the pixel value of the image obtained by applying the high-pass filter that can sufficiently suppress the low frequency component is quantized, and one color corresponds to one pixel value.
By using such a random color filter array generation method, color filters are randomly arranged so that the frequency of color presence within a specified size region including the target pixel position falls within a desired error range. A random color filter array can be generated.

  According to the present invention, it is possible to provide a color image sensor with reduced false colors and high resolution.

FIG. 1 is an overall configuration diagram of an image processing apparatus 1 according to the first embodiment of the present invention. FIG. 2 is a diagram showing color filter patterns generated by the image processing apparatus 1 shown in FIG. FIG. 3 is a diagram for explaining the RR ′ random array pattern generated by the RR ′ random array generator illustrated in FIG. 1. FIG. 4 is a diagram for explaining the BB ′ random array pattern generated by the BB ′ random array generator shown in FIG. 1. FIG. 5 is a diagram for explaining a GG ′ regular array pattern generated by the GG ′ regular array generation unit shown in FIG. 1. FIG. 6 is a diagram for explaining a regular arrangement pattern. FIG. 7 is a diagram for explaining a procedure in which the RR ′ random array generation unit generates a regular array pattern. FIG. 8 is a flowchart for explaining the procedure for generating the RR ′ random array pattern based on the regular array pattern generated by the RR ′ random array generator in the procedure shown in FIG. FIG. 9 is a flowchart showing in detail step ST407 of FIG. 8 by the RR ′ random array generation unit. FIG. 10 is a flowchart for explaining the processing of the GG ′ regular array generation unit. FIG. 11 is a flowchart for explaining the processing of the array synthesis unit. FIG. 12 is a diagram for explaining a white noise image generation method according to the second embodiment of the present invention. FIG. 13 is a diagram illustrating an example of convolution filter coefficients when m = 2. FIG. 14 is a flowchart for explaining the processing of the blue noise image generation unit. FIG. 15 is a flowchart for explaining an operation example of the RR ′ random array generation unit according to the second embodiment of this invention. FIG. 16 is a diagram showing color filter patterns generated by the image processing apparatus 300 shown in FIG. FIG. 17 is a block diagram illustrating a configuration of the image processing apparatus 300. FIG. 18 is a diagram for explaining the RB random array pattern generated by the random array generator shown in FIG. FIG. 19 is a diagram for explaining the G rule array pattern generated by the regular array generation unit shown in FIG.

Hereinafter, a color imaging device according to an embodiment of the present invention will be described.
<First Embodiment>
FIG. 1 is an overall configuration diagram of an image processing apparatus 1 according to the first embodiment of the present invention.
As illustrated in FIG. 1, the image processing apparatus 1 includes, for example, an RR ′ random array generation unit 201, a BB ′ random array generation unit 202, a GG ′ regular array generation unit 203, and an array synthesis unit 204.
The image processing apparatus 1 generates a color filter pattern S10 as shown in FIG.
In the color filter pattern S10, G is arranged every other line horizontally and vertically, G ′ is arranged every other line horizontally and vertically, and arranged on a line different from G.
Further, R and R ′ are randomly arranged at pixel positions every other line horizontally and vertically at pixel positions where G and G ′ do not exist.
B and B ′ are randomly arranged at pixel positions where R, R ′, G, and G ′ do not exist.
Note that R, R′B, B′G, and G ′ indicate positions on pixels (photoelectric conversion elements) or filter patterns of different colors, respectively.
R ′ is associated with a color closer to R than B, B ′, G, and G ′.
B ′ is associated with a color closer to B than R, R ′, G, and G ′.
G ′ is associated with a color closer to G than R, R′B, and B ′.
In this embodiment, R and R ′ indicate red, B and B ′ indicate blue, and G and G ′ indicate green.

In the image processing apparatus 1 illustrated in FIG. 1, the RR ′ random array generation unit 201 generates an RR ′ random array pattern S21 in which R and R ′ are included in the array as illustrated in FIG.
As shown in FIG. 4, the BB ′ random array generation unit 202 generates a BB ′ random array pattern S22 in which B and B ′ are included in the array.
The GG ′ regular array generation unit 203 generates a GG ′ regular array pattern S23 in which only G and G ′ are included in the array, as shown in FIG.

  The array synthesis unit 204 rearranges the pixels indicated by the RR ′ random array pattern S21 so that an interval of one pixel is opened, and opens an interval of one pixel between the pixels of the BB ′ random array pattern S22. In this way, the regular arrangement of the GG ′ regular arrangement pattern S23 is arranged into a checkered arrangement, and these are combined to generate a color filter pattern S10 that is a partially random arrangement pattern.

The details of each component of the image processing apparatus 1 shown in FIG. 1 will be described below.
[RR ′ random sequence generator 201]
The RR ′ random array generation unit 201 first generates a regular array pattern S30 of n × n pixels in which R and R ′ are regularly arrayed as shown in FIG.
FIG. 7 is a diagram for explaining a procedure in which the RR ′ random array generation unit 201 generates the regular array pattern S30.
In step ST301, the RR ′ random array generation unit 201 initializes a variable i indicating a pixel position for executing the process to 1, and initializes a variable j indicating a pixel position for executing the process to 1 in step ST302.
Next, the RR ′ random array generation unit 201 determines whether or not the remainder obtained by dividing i + j by 2 in step ST303 is 0. If the remainder is 0, the pixel position (i, j) is determined in step ST304. If the remainder is not 0, the color at the pixel position (i, j) is R ′ in step ST305.
ColRR ′ (i, j) used in FIG. 7 represents the color of the pixel position (i, j).
Hereinafter, ColZ (x, y) represents the color of the pixel position (x, y), and an appropriate character string is appropriately attached to the portion of the subscript Z to distinguish the arrangement.
ColRR ′ (x, y) indicates the color of the array composed of R and R ′, and ColBB ′ (x, y) indicates the color of the array composed of B and B ′. ColGG ′ (x, y) indicates the color of the array composed of G and G ′, and ColComp (x, y) indicates the color of the array obtained by synthesizing the three sequences.

  Next, the RR ′ random array generation unit 201 determines whether j = n in step ST306. If j = n is not satisfied, j is updated to j + 1 in step ST308, and the process returns to step ST303. If j = n, the process proceeds to step ST307.

  Next, the RR ′ random sequence generation unit 201 determines whether i = n in step ST307. If i = n is not satisfied, i is updated to i + 1 in step ST309, and the process returns to step ST302. If i = n, the process proceeds to step ST310, and a regular array pattern S30 of n × n pixels in which R and R ′ are regularly arranged is generated.

FIG. 8 is a flowchart for explaining a procedure for generating the RR ′ random array pattern S21 based on the regular array pattern S30 generated by the RR ′ random array generator 201 according to the procedure shown in FIG.
The RR ′ random array generation unit 201 acquires the regular array pattern S30 generated by the procedure of FIG. 7 in step ST401.
The RR ′ random array generation unit 201 initializes a variable k indicating the number of loops to 1 in step ST402.
Next, the RR ′ random array generation unit 201 sets a variable x indicating a pixel position to be processed in step ST403 as a positive integer random number less than n, and variable y indicating a pixel position to be processed in step ST404. Let be a random integer with a positive integer value less than n.
At this time, there are various random number generation methods, but it is desirable that the distribution of the generated random numbers is not biased. For example, M. Matsumoto and T. Nishimura, “Mersenne Twister: A 623-dimensionally equidistributed uniform pseudorandom number generator ”, ACM Trans. On Modeling and Computer Simulation Vol. 8, No. 1, January pp.3-30, random numbers by Mersenne Twister method may be used.

Next, the RR ′ random sequence generation unit 201 temporarily stores ColRR ′ (x, y) in the variable C in step ST405.
Next, the RR ′ random array generation unit 201 randomly sets ColRR ′ (x, y) to R or R ′ in step ST406. To select R or R 'at random, for example, select R when the remainder of a sufficiently large positive integer value divided by 2 is 0, and select R' when the remainder is 1 do it.

Next, in step ST407, the RR ′ random array generation unit 201 checks whether the existence frequency of R and R ′ existing in the local region including the target pixel position (x, y) is within a desired error range. .
If it is not within the error range, the RR ′ random array generation unit 201 proceeds to step ST410, sets ColRR ′ (x, y) to C, which is the original color, and updates k to k + 1 in step ST411. Then, the process returns to step ST403. On the other hand, if it is within the error range, the RR ′ random array generation unit 201 proceeds to step ST408.

  In step ST408, the RR ′ random array generation unit 201 updates k to k + 1 in step ST411 and returns to step ST403 if k is equal to or smaller than a constant MAXLOOP indicating the maximum number of loops. If k is larger than the constant MAXLOOP indicating the maximum number of loops, the process proceeds to step ST409, where the RR ′ RR ′ random array pattern S21 of n × n pixels is output, and the process is terminated.

FIG. 9 is a flowchart showing in detail step ST407 of FIG. 8 by the RR ′ random array generation unit 201 below.
In the present embodiment, the local area for checking the existence frequency is a single rectangular area of m × m pixels including the target pixel position (x, y), and the existence frequency of R and R ′ existing in this rectangular area Is greater than the constant const1 and smaller than the constant const2.

In step ST501, the RR ′ random array generation unit 201 initializes a variable i indicating the position of the local region to i = −m + 1, and sets a variable j indicating the position of the local region to j = −m + 1 in step ST502. initialize.
Further, the RR ′ random array generation unit 201 initializes a variable a indicating the number of Rs included in the local region to 0 in step ST503, and the number of R ′ included in the local region in step ST504. The variable b indicating is initialized to 0.
Next, the RR ′ random array generation unit 201 initializes a variable s indicating a pixel position for executing processing in the local region to 0 in step ST505, and indicates a pixel position for executing processing in the local region in step ST506. The variable t is initialized to 0.

Next, in step ST507, the RR ′ random sequence generation unit 201 checks whether ColRR ′ (x + i + s, y + j + t) is R. The variables x and y used here are those used in FIG.
If ColRR ′ (x + i + s, y + j + t) is R, RR ′ random array generation unit 201 updates a to a + 1 in step ST508, and proceeds to step ST510.
On the other hand, if ColRR ′ (x + i + s, y + j + t) is not R, RR ′ random array generation unit 201 updates b to b + 1 in step ST509, and proceeds to step ST510.

The RR ′ random sequence generation unit 201 determines whether t = m−1 in step ST510. If t = m−1, t is updated to t + 1 in step ST512, and the process of step ST507 is performed. Returning, if t = m−1, the process proceeds to step ST511.
Next, RR ′ random sequence generation section 201 determines whether or not s = m−1 in step ST511. If s = m−1 is not satisfied, s ′ is updated to s + 1 in step ST513, and step ST506. Returning to the process, if s = m−1, the process proceeds to step ST514.

Next, in step ST514, the RR ′ random array generation unit 201 checks whether const1 <a / (a + b) <const2 and const1 <b / (a + b) <const2, and does not satisfy the above condition. In this case, it is determined that the presence frequencies of R and R ′ in the local region including the target pixel position (x, y) are not within the desired error range.
If the above condition is satisfied, the RR ′ random sequence generation unit 201 proceeds to step ST515.
Next, the RR ′ random sequence generation unit 201 determines whether j = 0 in step ST515. If j = 0, j is updated to j + 1 in step ST517, and the process returns to step ST503. If j = 0, the process proceeds to step ST516.

  Next, RR ′ random sequence generation section 201 determines whether i = 0 in step ST516. If i = 0, i is updated to i + 1 in step ST518, and the process returns to step ST502. If i = 0, it is determined that the existence frequency of R and R ′ in the local region including the target pixel position (x, y) is within a desired error range.

[BB ′ random sequence generation unit 202]
The processing of the BB ′ random sequence generation unit 202 is the same as the processing for R and R ′ by the RR ′ random sequence generation unit 201 (the processing of FIGS. 7, 8, and 9). Against.

[GG ′ regular array generator 203]
The GG ′ regular array generation unit 203 generates a color filter pattern S10 that is a partially random array pattern indicating an array of n × 2n pixels in which G and G ′ are regularly arrayed.
FIG. 10 is a flowchart for explaining the processing of the GG ′ regular array generation unit 203.
In step ST601, the GG ′ regular array generation unit 203 initializes a variable i indicating a pixel position for executing the process to 1, and initializes a variable j indicating a pixel position for executing the process to 1 in step ST602.
Next, the GG ′ regular array generation unit 203 determines whether or not the remainder obtained by dividing i by 2 in step ST603 is 0. If the remainder is 0, ColGG ′ (i, j) is determined in step ST604. If G is not zero and ColGG ′ (i, j) is G ′ in step ST605.

Next, GG ′ regular array generation section 203 determines whether or not j = n in step ST606. If not j = n, j is updated to j + 1 in step ST608, and the process of step ST603 is performed. Returning, if j = n, the process proceeds to step ST607.
Next, GG ′ regular array generation section 203 determines whether i = 2n in step ST607, and if i = 2n, i is updated to i + 1 in step ST609, and the process of step ST602 is performed. If i = 2n, the process proceeds to step ST610, where a GG ′ regular array pattern S23, which is a GG ′ regular array of n × 2n pixels, is output, and the process ends.

[Sequence synthesizing unit 204]
The array synthesis unit 204 includes an RR ′ random array pattern S21 from the RR ′ random array generation unit 201, a BB ′ random array pattern S22 from the BB ′ random array generation unit 202, and a GG ′ regular array generation unit 203. Based on the GG ′ regular array pattern S23, the color filter pattern S10 shown in FIG. 2 having a partially random array of 2n × 2n pixels composed of six colors of R, R ′, B, B ′, G, and G ′ is generated. To do.

FIG. 11 is a flowchart for explaining the processing of the array synthesis unit 204.
The array synthesis unit 204 acquires the RR ′ random array of n × n pixels generated by the RR ′ random array generation unit 201 in step ST701, and the n × n generated by the BB ′ random array generation unit 202 in step ST702. Obtain a BB ′ random array of pixels.
In addition, the array synthesis unit 204 acquires the GG ′ regular array of n × 2n pixels generated by the GG ′ regular array generation unit 203 in step ST703.

Next, array synthesizing section 204 initializes variable i indicating the pixel position for executing the process to 1 in step ST704, and initializes variable j indicating the pixel position for executing the process to 1 in step ST705.
Next, array composition section 204 determines whether or not the remainder obtained by dividing i + j by 2 in step ST706 is 0. If the remainder is 0, the process proceeds to step ST707, and if the remainder is not 0, step ST708 is performed. Proceed to
In step ST707, array synthesizing section 204 determines whether or not the remainder obtained by dividing j by 2 is 0. If the remainder is 0, ColComp (i, j) is changed to ColBB ′ (i / 2, j in step ST709. / 2), and the process proceeds to step ST713. If the remainder is not 0, ColComp (i, j) is set to ColRR ′ ((i + 1) / 2, (j + 1) / 2) in step ST710, and the process proceeds to step ST713. move on.
In step ST708, the array composition unit 204 determines whether or not the remainder obtained by dividing j by 2 is 0. If the remainder is 0, ColComp (i, j) is changed to ColGG ′ (i, j / 2) in step ST711. The process proceeds to step ST713. If the remainder is not 0, ColComp (i, j) is set to ColGG ′ (i, (j + 1) / 2) in step ST712, and the process proceeds to step ST713.

Next, array synthesis section 204 determines whether or not j = 2n in step ST713,
If j = 2n, j is updated to j + 1 in step ST716, and the process returns to step ST706. If j = 2n, the process proceeds to step ST714.
Next, array synthesizing section 204 determines whether i = 2n in step ST714. If i = 2n, i is updated to i + 1 in step ST717, and the process returns to step ST705, where i = If it is 2n, the process proceeds to step ST715, where a color filter pattern S10 having a partially random arrangement of 2n × 2n pixels is output and the process is terminated.

As described above, according to the image processing apparatus 1, it is possible to generate the color filter pattern S10 having the color filter array used for the single-plate color imaging device in which some colors are randomly arranged.
In the image pickup apparatus using the color filter pattern S10 as the array of the image pickup elements, since some colors of the array are randomly arranged, false colors are reduced, and some colors of the array are regularly arranged. Therefore, the reproducibility of resolution can be improved. Further, in the image processing apparatus 1, at the same time as being randomly arranged, the presence frequency of a color within an area of a prescribed size including the target pixel position is within a desired error range for any target pixel position. Therefore, all the colors always exist in the vicinity of the target pixel, and the color filter pattern S10 can be generated by performing the color separation process.

Second Embodiment
In the present embodiment, a case will be described in which a random array is generated from a white noise image in the RR ′ random array generation unit 201 and the BB ′ random array generation unit 202 illustrated in FIG. 1.
In this case, a blue noise image is generated by applying a high-pass filter to the white noise image generated by the white noise image generation unit by the blue noise image generation unit.
The RR ′ random array generation unit 201 binarizes the blue noise image by an error diffusion method, and generates R R ′ random array pattern S21 by associating R and R ′ with the two values.

First, the white noise image generation unit will be described.
The white noise image generation unit generates an n × n pixel white noise image having a real value in the interval [0, 1] as a pixel value.
FIG. 12 is a diagram for explaining a white noise image generation method according to the present embodiment.
In step ST801, the white noise image generation unit initializes a variable i indicating a pixel position where the process is executed to 1, and initializes a variable j indicating a pixel position where the process is executed to 1 in step ST802.
Next, in step ST803, the white noise image generation unit sets the pixel value at the pixel position (i, j) as a real-valued random number in the interval [0, 1].
PRR ′ (i, j) used in FIG. 12 represents a pixel value at the pixel position (i, j) of the white noise image.

Next, the white noise image generation unit determines whether j = n in step ST804. If j = n, j is updated to j + 1 in step ST806, and the process returns to step ST803, j If = n, the process proceeds to step ST805.
Next, the white noise image generation unit determines whether i = n in step ST805. If i = n, i is updated to i + 1 in step ST807, and the process returns to step ST802. If = n, the process proceeds to step ST808, and a white noise image of n × n pixels is output and the process is terminated.

Next, the blue noise image generation unit will be described.
The blue noise image generator applies a high-pass filter to the white noise image,
It is a means for generating an n × n pixel blue noise image having a real value in the interval [0, 1] as a pixel value.
In the present embodiment, a high-pass filter is realized using a convolution filter having a size of 2m + 1 × 2m + 1 pixels.

FIG. 13 shows an example of convolution filter coefficients when m = 2.
As another embodiment, a configuration may be adopted in which a high-pass filter is applied on a frequency space using Fourier transform.

FIG. 14 is a flowchart for explaining the processing of the blue noise image generation unit.
The blue noise image generation unit acquires an n × n pixel white noise image generated by the white noise image generation unit in step ST901.
Next, in step ST902, the blue noise image generation unit initializes a variable i indicating a pixel position where the process is executed to 1, and initializes a variable j indicating a pixel position where the process is executed to 1 in step ST903.
Next, the blue noise image generation unit initializes the pixel value at the pixel position (i, j) to 0 in step ST904.

P′RR ′ (i, j) used in FIG. 14 represents the pixel value at the pixel position (i, j) of the blue noise image.
Next, the blue noise image generation unit initializes a variable s indicating a pixel position in the convolution filter to −m in step ST905, and sets a variable t indicating a pixel position in the convolution filter to −m in step ST906. Initialize to.
Next, in step ST907, the blue noise image generation unit multiplies PRR ′ (i + s, j + t) by a convolution filter coefficient w (s + m, t + m) to obtain P′RR ′ ( The sum of i, j) is defined as a new P′RR ′ (i, j).
Next, the blue noise image generation unit determines whether t = m in step ST908. If t = m, t is updated to t + 1 in step ST913, and the process returns to step ST907, and t If = m, the process proceeds to step ST909.

Next, the blue noise image generation unit determines whether s = m in step ST909, and if s = m, updates s to s + 1 in step ST914, returns to the process in step ST906, and returns s If = m, the process proceeds to step ST910.
Next, the blue noise image generation unit determines whether j = n in step ST910. If j = n, j is updated to j + 1 in step ST915, and the process returns to step ST905. If = n, the process proceeds to step ST911.
Next, the blue noise image generation unit determines whether i = n in step ST911. If i = n is not satisfied, i is updated to i + 1 in step ST916, and the process returns to step ST903. If = n, the process proceeds to step ST912, a blue noise image of n × n pixels is output, and the process ends.

Hereinafter, the processing of the RR ′ random array generation unit 201 in the present embodiment will be described.
The RR ′ random array generation unit 201 applies binarization by an error diffusion method to the blue noise image generated by the blue noise image generation unit, associates R and R ′ with the two values, and RR ′ random An array pattern S21 is generated.
In this embodiment, the Floyd-Steinberg type error diffusion method is used, but other error diffusion methods may be used.
In addition, the reason why the error diffusion method is used in this embodiment is that the low-frequency signal component is well preserved when quantization is performed, and if the same effect can be obtained by other quantization methods, the error diffusion method is used. It may be used.

FIG. 15 is a flowchart for explaining an operation example of the RR ′ random array generation unit 201 of the present embodiment.
The RR ′ random array generation unit 201 in the present embodiment acquires the n × n pixel blue noise image generated by the blue noise image generation unit in step ST1001.
Next, in step ST1002, the RR ′ random array generation unit 201 initializes a variable i indicating a pixel position for executing the process to 1, and initializes a variable j indicating a pixel position for executing the process to 1 in step ST1003. .
Next, the RR ′ random sequence generation unit 201 determines whether P′RR ′ (i, j)> 0.5 in step ST1004. If P′RR ′ (i, j)> 0.5, step ST1005 In step ST1006, e is set to 1-P′RR ′ (i, j), ColRR ′ (i, j) is set to R in step ST1006, and the process proceeds to step ST1009.
If P′RR ′ (i, j)> 0.5, RR ′ random sequence generation unit 201 sets e to P′RR ′ (i, j) in step ST1007, and ColRR ′ (i, j) in step ST1008. Set to R ′ and proceed to step ST1009.
Next, the RR ′ random array generation unit 201 updates P′RR ′ (i + 1, j) to P′RR ′ (i + 1, j) + e × 7/16 in step ST1009, and performs step ST1010. P′RR ′ (i−1, j + 1) is updated to P′RR ′ (i−1, j + 1) + e × 3/16 in step ST1011, and P′RR ′ (i, j + 1) is updated to P′RR ′ (i, j + 1) + e × 5/16, and in step ST1012, P′RR ′ (i + 1, j + 1) is changed to P′RR ′ (i + 1, j + 1) + e × 1/16, and the process proceeds to step ST1013.

Next, the RR ′ random sequence generation unit 201 determines whether j = n in step ST1013. If j = n is not satisfied, j is updated to j + 1 in step ST1016, and the process returns to step ST1004. If j = n, the process proceeds to step ST1014.
Next, the RR ′ random sequence generation unit 201 determines whether i = n in step ST1014. If i = n is not satisfied, i is updated to i + 1 in step ST1017, and the process returns to step ST1003. If i = n, the process proceeds to step ST1015 to output an RR ′ random array of n × n pixels, and the process ends.

<Third Embodiment>
In the present embodiment, the color filter pattern S310 shown in FIG. 16 is generated using the image processing apparatus 300 shown in FIG.
As illustrated in FIG. 17, the image processing apparatus 300 includes, for example, a random array generation unit 301, a regular array generation unit 303, and an array synthesis unit 304.
In the image processing apparatus 300 shown in FIG. 17, the random array generation unit 301 generates an RB random array pattern S301 in which R and B are included in the array as shown in FIG.
As shown in FIG. 19, the regular array generation unit 303 generates a G regular array pattern S302 in which only G is included in the array.

The sequence synthesis unit 304 rearranges the RB random sequence pattern S301 generated by the random sequence generation unit 301 into a checkered sequence, rearranges the G regular sequence pattern S302 generated by the regular sequence generation unit 303 into a checkered sequence, By combining the two, a color filter pattern S310 shown in FIG. 16, which is a partially random array, is generated.
In the present embodiment, the case where one random array generation unit 301 and one regular array generation unit 303 are used is illustrated, but a plurality of these may be used.

The present invention is not limited to the embodiment described above.
That is, the present invention can be applied even if various changes and substitutions are made with respect to the components of the above-described embodiment within the technical scope of the present invention or an equivalent range thereof.

  DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus, 201 ... RR 'random arrangement | sequence production | generation part, 202 ... BB' random arrangement | positioning production | generation part, 203 ... GG 'regular arrangement | positioning production | generation part, 204 ... Array composition part, 300 ... Image processing apparatus, 301 ... Random arrangement Generation unit, 303 ... regular array generation unit, 304 ... array synthesis unit

Claims (5)

A color imaging element having a pixel group in which a plurality of pixels each including a photoelectric conversion element are arranged,
It has a color filter that combines a random array pattern and a regular array pattern,
In the random arrangement pattern, pixels of some colors are randomly arranged for any pixel position so that the frequency of color existence within a specified size area including the pixel position falls within a desired error range. are those that were,
The said regular arrangement pattern is a color image sensor which arrange | positions the pixel of the remaining color regularly. In the random sequence pattern,
To color filter patterns ordered regularly, colors to be assigned to the target pixel position location which is randomly selected is replaced with other colors,
The color assigned to the target pixel position is not replaced when the existence frequency of the color allocated in a region having a prescribed size including the target pixel position does not fall within a desired error range. color imaging device according. In the random sequence pattern ,
A pixel value of an image obtained by performing high-pass filter processing on a white noise image having a continuous value or a discrete value represented by a predetermined number of quantization bits as a pixel value is quantized to one pixel value. One of the color image sensor according to claim 1, the color Ru correspond. Of the six colors including the first red, the first blue, and the first green, the second red closest to the first red, the second blue closest to the first blue, and the When the second green closest to the first green is defined,
The random array pattern is first random array pattern in which a said second red and the first red randomly second having an array of said first blue and second blue randomly And a random arrangement pattern of
In the regular arrangement pattern, the first green color and the second green color are regularly arranged ,
In the color filter, wherein a first random array pattern, the second random array pattern, color imaging device according to claim 1, said ordered array pattern is synthesized. Wherein the random sequence pattern includes a red, blue are arranged randomly,
The ordered array pattern, green are regularly arranged,
Wherein the color filter includes: the random sequence pattern, color imaging device according to claim 1, said ordered array pattern is synthesized.

Images