KR101882739B1 - Images Upsampling and Pattern Assignment Apparatus and Method thereof - Google Patents

Abstract

According to the present invention, an apparatus for changing image resolution using a pattern comprises: a database storing RRGB, RGGB and RGBB patterns for a plurality of training images and a target image, and actual measurement values for R, G, and B for each pixel of the training images and the target image; a training filter design module to receive the RRGB, RGGB and RGBB patterns for the plurality of training images from the database to separate the RRGB, RGGB and RGBB patterns for the training image into images of R, G, and B channels, downsample the separated images by channel to form low resolution images by channel to form a reconstruction image, and receive the actual measurement values for R, G, and B for each pixel of the training images and the target image from the database to design a training filter; and a performance verification module to select a color filter array to verify performance among a plurality of color filter arrays having different patterns to receive a target image photographed by an image sensor having the selected color filter array to separate the target image into images of R, G, and B channels, downsample the separated images by channel to form low resolution images by channel, and input the target image into the designed training filter to acquire the reconstruction image to compare the target image and the reconstruction image to verify performance of the color filter array.

Description

(Image Upsampling and Pattern Assignment Apparatus and Method thereof)

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and a method for converting image resolution using a pattern.

For images taken with a digital camera, filters should be displayed on sensors such as CCD or CMOS [1-3]. These filters are pre-assigned along the sensor's location [4.5].

The Bayer pattern color filter array (CFA) is a widely used CFA, with the RGGB pattern generally being adopted, but it can be an alternative to the CFA filter.

To obtain a color image from a CFA image, there must be three color components, red, green, and blue, at each pixel location.

In the mosaic phase, the CFA image loses two color components and leaves only one color information. Therefore, this information should be reconstructed by interpolation.

The interpolation method is nearest-neighbor, bilinear or bi-cubic [6-10].

In general, this method can cause serious aliasing effects in the color plane [11-13]. Thus, demosaicing in an individual plane is not effective.

Alternatively, some methods use predefined filters, where demosaicing is performed in the frequency domain [14,15].

All filters are pre-engineered with pre-drill and less image aliasing than spatial methods.

[1] B. E. Bayer, "Color imaging array", U.S. Pat. Patent 3 971 065, (1976). [2] J. Hamilton and J. Adams, "Adaptive color plan interpolation in single sensor color electronic camera", US Patent 5,629,734, (1997). [3] J. Wu, A. Paul, Y. Xing, Y. Fang, J. Jeong, L. Jiao and G. Shi, "Morphological dilation image 107 coding with context weights prediction" . 25, no. 10, (2010), pp.717.728. [4] W. Wu, Z. Liu and X. He, "Learning-based super resolution using kernel partial least squares", Image Vision Comput., Vol. 29, (2011), pp. 394.406. [5] J. Wu, J. Huang, G. Jeon, J. Jeong and L.C. Jiao, " An adaptive autoregressive de-interlacing method ", Optical Engineering, vol. 50, no. 5, (2011), pp. 057001. [6] W. Wu, Z. Liu, W. Gueaieb and X. He, "Single-image super-resolution based on Markov random field and contour transform", J. Electron. Imaging, vol. 20, (2011), pp. 023005. [7] J. Wu, T. Li, T.-J. Hsieh, Y.-L. Chang and B. Huang, " Digital signal processor-based 3D wavelet 116 error-resilient lossless compression of high-resolution spectrometer data ", Journal of Applied Remote Sensing, vol. 5, (2011), pp. 051504. [8] K. He, J. Sun and X. Tang, "Guided image filtering", in Proc. of the 11th European Conf. on Computer Vision (ECCV) 6311, (2010), pp. 1-14.

The present invention relates to a method and an apparatus for efficiently analyzing the performance of each color filter array by comparing and analyzing an image obtained by reconstructing an image captured through a plurality of color filter arrays and an original image, Apparatus and method.

A database storing RRGB, RGGB, and RGBB patterns for a plurality of training images and target images, and storing actual values of each of R, G, and B for each training image and each pixel of the target image; RRGB, RGGB, and RGBB patterns for a plurality of training images are received from the database, and R, G, and B patterns of the training images are separated into R, G, and B channels for each channel. G, and B for each pixel of each training image and a target image in the database to form a training image by forming low resolution images for each channel by downsampling the image, Training filter design module to design; And a color filter array having a different pattern from among a plurality of color filter arrays. The target image photographed by the image sensor having the selected color filter array is input to the database, B channel images and down-sampling each separated channel image to form low resolution images for each channel. The target image is input to the designed training filter to obtain a restored image, and the restored image is compared with the target image And a performance verification module for verifying the performance of the color filter array.

(A) a plurality of training images and RRGB, RGGB, and RGBB patterns for a target image are stored in a database, and each of R, G, and B is measured for each pixel of each training image and a target image Storing a value; (B) The training filter design module receives RRGB, RGGB, and RGBB patterns for a plurality of training images in the database, and acquires R, G, and B channel images for RRGB, RGGB, G, and B for each pixel of each training image and the target image in the database by forming low resolution images for each channel by downsampling the separated images for each channel, Designing a training filter by receiving an actual value; And (C) the performance verification module selects a color filter array to verify performance among a plurality of color filter arrays having different patterns, and outputs the target image captured by the image sensor having the selected color filter array to the database Separates the images of the R, G, and B channels for each channel, downsamples the separated images for each channel, forms low resolution images for each channel, inputs the target images to the designed training filters to obtain restored images And comparing the target image and the restored image to verify the performance of the color filter array.

As described above, according to the present invention, the performance of each color filter array can be efficiently analyzed by comparing and analyzing an image obtained by reconstructing an image captured through a plurality of color filter arrays and an original image.

In addition, the present invention can provide validation results for a plurality of color filter arrays to provide information that is effective for color filter array selection.

Figure 1 shows an example of a diagonal pattern CFA.
Figure 2 shows how RRGB, RGGB and RGBB Bayer patterns CFA are formed.
3 is a block diagram of a video resolution conversion apparatus using a pattern according to an embodiment of the present invention.
4 is a diagram illustrating a design process of a training filter according to an embodiment of the present invention.
5 is a flowchart of a color filter array performance verification method according to an embodiment of the present invention.
6 (a) is a red channel frequency response of RRGB case for # 104 image, (b) is a green channel frequency response of RRGB case for # 104 image, (c) (D) represents the red channel frequency response of the RRGB case for the # 105 image.
Figure 7 (a) shows the red channel frequency response for the # 104 image, (b) shows the green channel frequency response for the RGGB case for the # 104 image, (D) represents the red channel frequency response of the RGGB case for the # 105 image.
8B is a green channel frequency response of the case of RGBB to the image of # 104, and FIG. 8C is a case of RGBB of the image of # 104 (D) represents the red channel frequency response of the RGBB case for the # 105 image.
9 (a) shows a simulation result image for CFA image of # 23 LC image. Figures 9 (b) and 9 (c) show the green and blue components of Figure 9 (a). 9 (d) shows the restored red component. Figure 9 (e) shows the demosaiced image very visually close to the original image shown in Figure 9 (f). Figure 9 (f) shows the original image.
10 (a) shows a simulation result image of the CFA image of the # 24 LC image. Figures 10 (b) and 10 (c) show the green and blue components of Figure 10 (a). 10 (d) shows the restored red component. Fig. 10 (e) shows a demosaiced image which is very visually close to the original image shown in Fig. 10 (f). Figure 10 (f) shows the original image.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments will be described in detail below with reference to the accompanying drawings.

The following examples are provided to aid in a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, this is merely an example and the present invention is not limited thereto.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. The following terms are defined in consideration of the functions of the present invention, and may be changed according to the intention or custom of the user, the operator, and the like. Therefore, the definition should be based on the contents throughout this specification. The terms used in the detailed description are intended only to describe embodiments of the invention and should in no way be limiting. Unless specifically stated otherwise, the singular form of a term includes plural forms of meaning. In this description, the expressions "comprising" or "comprising" are intended to indicate certain features, numbers, steps, operations, elements, parts or combinations thereof, Should not be construed to preclude the presence or possibility of other features, numbers, steps, operations, elements, portions or combinations thereof.

It is also to be understood that the terms first, second, etc. may be used to describe various components, but the components are not limited by the terms, and the terms may be used to distinguish one component from another .

Figure 1 shows an example of a diagonal pattern CFA. In the diagonal pattern CFA, the three color components are assigned in diagonal form.

The image in the first column is a diagonal pattern CFA and the image in the second column is a separate image for each of R, G, B. The image in the third column is the restored image for each R, G, B. The image in the last column is an image obtained by combining restored images for R, G, and B, respectively.

Generally unwanted color artifacts are generated between the second and third column stages. Therefore, it is important to reproduce r, g, b information from a given R, G, B well.

Figure 2 shows how RRGB, RGGB and RGBB Bayer patterns CFA are formed.

The three images located below in FIG. 2 are the RGBB, RRGB, and RGGB patterns of FIG. Here, the four pixels can be treated as a pair.

The ratio of R: G: B is 2: 1: 1 in the RRGB color filter array, the ratio of R: G: B is 1: 2: 1 in the RGGB color filter array, Is 1: 1: 2.

The RRGB color filter array, the RGGB color filter array, and the RGBB color filter array of FIG. 2 are superior to conventional Bayer color pattern filters in performance against color artifacts, suppression performance against aliasing phenomena Is excellent.

There are many demosaicing methods to improve subjective and objective performance. The three best known methods are nearest-neighbor interpolation, bilinear interpolation, and bi-cubic interpolation method.

Each method has different characteristics. For example, nearest neighbor interpolation simply replicates adjacent information of the same color component, but this method causes inadequate hue.

To use the bilinear method, we obtain the average value of the adjacent pixels of each color component, but this method leads to severe color artifacts. These artifacts are particularly indicated in the detail area (eg edge).

The bi-cubic method is an extended version of the bilinear method, which uses more pixels to calculate the missing components. At each pixel, different weights are assigned and generally higher weights are assigned to pixels near the center.

The kernel K is a form of a convolution matrix and the kernel K can be represented by a 3 * 3 matrix and can be used to detect, blur, sharpen, or emboss an image edge.

For example, equation (1) represents an edge detection kernel. Here, the relationship between a and b is a = -b, the relationship between c and d is d = -4c, and the relationship between e and f is f = -8e.

(1)

Equation (2) shows a sharpening kernel in which the sum of all coefficients reaches one. Here, the relationship between g and h is h = -4g + 1.

(2)

Equation (3) shows the blurring kernel. There are two types of blurring kernels, normalized and Gaussian.

The normalization kernel takes the following form.

(3)

The relationship between i and j is j = 9i. In the case of Gaussian, an example of a Gaussian blurring kernel is shown in the following equation (4).

(4)

Here, the relationship between k, l, m and n is n = 4m = 8l = 16k.

The Gaussian function is represented as a variable with spread σ as follows.

(5)

The above equation is represented by Equation 6 below with two variables.

(6)

On the other hand, the filtering process is represented by a convolution using matrix multiplication and simply "* ".

In the present invention, the least squares method, which is one of the standard techniques widely used to obtain necessary conditions at a minimum cost, is used. This method is particularly useful for data fitting problems such as redundancy and interpolation, and smoothing and approximation problems arise.

In the present invention, a training filter used for the mosaic is obtained by using a plurality of training images given with a focus on this characteristic.

3 is a block diagram of a video resolution conversion apparatus using a pattern according to an embodiment of the present invention.

Referring to FIG. 3, an image resolution conversion apparatus using a pattern according to an exemplary embodiment of the present invention includes a database 10, a training filter design module 100, and a performance verification module 200.

Data for a plurality of training images are stored in the database 10, and the data for the plurality of training images stored are passed through an RRGB color filter array, an RGGB color filter array, and an RGBB color filter array for each training image RRGB, RGGB and RGBB patterns are stored.

For each pixel of each training image, measured values for each of R, G, and B are stored.

Data for a plurality of target images is stored in the database 10, and data for a plurality of target images stored in the database 10 passes through the RRGB color filter array, the RGGB color filter array, and the RGBB color filter array for each target image One RRGB, RGGB and RGBB patterns are stored.

Then, actual values of R, G, and B are stored for each pixel of each target image.

Next, the training filter design module 100 includes an image input unit 110, a downsampling unit 120, a signal processing unit 130, and a filter design unit 140.

The image input unit 110 receives RRGB, RGGB, and RGBB patterns for a plurality of training images in the database 10.

The downsampling unit 120 separates the R, G, and B channels of the RRGB, RGGB, and RGBB patterns of each training image into the images of each channel and downsamples the separated images for each channel, Images.

At this time, the signal processing unit 130 performs a specified signal process on the downsampled image output from the downsampling unit 120.

For example, the signal processing performed by the signal processing unit 130 performs processing for detecting, blurring, sharpening, or embossing image edges.

Then, the filter designing unit 140 receives the low-resolution images for each of the R, G, B channels of the RRGB, RGGB, and RGBB patterns of each training image and downsampled for each channel, forms a reconstructed image, 10), training filters are designed by receiving actual values of R, G, and B for each pixel of each training image.

At this time, the filter designing unit 140 designs a training filter through a process of minimizing a square error between an actual pixel value of the training image and a restored pixel value of the restored image.

The designed training filter is obtained as follows.

The actual pixel values of the N number of training images _{(x i) (i = 1,2} , · · ·, N) Y i when _{La, (x 1, Y 1)} , (x 2, Y 2), · · · , is represented by (x _n, y _n), wherein when as the reconstructed pixel values of the reconstructed image y _i, n of the given function, as follows: for each training image x _i using the least square method X _ij (j = 1, 2, ..., n).

(7)

Here, j = 1, 2, ..., n, and β _j is a decision coefficient corresponding to each of the n specified functions X _ij for the training image x _i , i = 1, 2, N, x _i is the training image, and y _i is the reconstructed image.

Equation (7) is expressed in a matrix form as shown in Equation (8).

(8)

Here, X is a training image matrix ,? Is a decision coefficient matrix, and y is a reconstructed image matrix.

Then, the decision coefficient β is obtained by the following method in the sense that the cost is minimized.

(9)

Here, S is a square error and represents an error obtained by subtracting the pixel value of the reconstructed image from the pixel value of the training image.

The decision coefficient matrix ? Using the training image matrix X and the transposed matrix X ^T of the training image matrix and the reconstructed image matrix y , the final equation is as shown in Equation 10 below.

(10)

The performance verification module includes an image input unit 210, a downsampling unit 220, a restored image acquisition unit 230, and a performance verification unit 240.

The image input unit 210 selects a color filter array to verify the performance among a plurality of color filter arrays having different patterns and displays a target image photographed by the image sensor having the selected color filter array in a database Receive input.

Of course, in one embodiment, if the color filter array verification method is implemented as a hardware or software module of a computing system, the computing system may receive and process a subject image captured with an image sensor having a selected color filter array.

In this way, the image input unit 210 can select a target image already stored in the database and input the image, and can also receive an image photographed directly from the image sensor.

The downsampling unit 220 separates the input image into the R, G, and B channels, and downsamples the separated images to form low resolution images.

The restored image obtaining unit 230 obtains a restored image by inputting a target image into a designed training filter.

The training filter may be color filtered through a color filter to interpolate the photographed image to restore the image. That is, when the target image is input, the designed training filter interpolates the input target image to acquire the reconstructed image.

The performance verifying unit 240 verifies the performance of the color filter array by comparing the target image and the restored image when the restored image is obtained through the training filter.

4 is a diagram illustrating a design process of a training filter according to an embodiment of the present invention.

Referring to FIG. 4, in the training filter designing process according to an exemplary embodiment of the present invention, the training filter design module receives RRGB, RGGB, and RGBB patterns for a plurality of training images in a database (S100).

The training filter design module separates the R, G, and B channels of each training image into R, G, and B channels for each channel and downsamples the separated images for each channel to generate low resolution images for each channel (S110).

At this time, the training filter designing module performs signal processing for the downsampled image (S120).

For example, the signal processing carried out performs processing of detecting, blurring, sharpening, or embossing image edges.

Then, the training filter design module inputs the low-resolution images for each channel, which are separated by R, G, and B channels and downsampled for the RRGB, RGGB, and RGBB patterns of each training image, to design a training filter (S130).

5 is a flowchart of a color filter array performance verification method according to an embodiment of the present invention.

Referring to FIG. 5, in the color filter array performance verification method according to an exemplary embodiment of the present invention, a performance verification module selects a color filter array to verify performance among a plurality of color filter arrays having different patterns , The target image photographed by the image sensor having the selected color filter array is received from the database (S200).

Of course, in one embodiment, if the color filter array verification method is implemented as a hardware or software module of a computing system, the computing system may receive and process a subject image captured with an image sensor having a selected color filter array.

In this way, the performance verification module can select a target image already stored in the database and input the captured image directly from the image sensor.

When the target image is input, the performance verification module separates the input image into R, G, and B images, and downsamples the separated images to form low-resolution images (S210).

The performance verification module inputs the target image to the designed training filter (step S220).

The training filter may be color filtered through a color filter to interpolate the photographed image to restore the image. That is, when the target image is input, the designed training filter interpolates the input target image to obtain a restored image (step S230).

When the reconstructed image is obtained through the training filter, the performance verification module verifies the performance of the color filter array by comparing the target image and the reconstructed image (step S240).

In one embodiment, a color filter array verification method according to an embodiment of the present invention compares at least one of CPSNR, S-CIELAB, and Peak Signal to Noise Ratio (PSNR) of an original image and a restored image The performance of the color filter array can be verified.

Here we have studied three cases of the modified Bayer CFA of CPSNR and S-CIELAB. We used 20 LC images in the simulation. Especially # 101-120 images were selected. Table 1 shows CPSNR and S-CIELAB performance for three types of CFAs.

(Table 1)

Three color components were thoroughly tested in Table 2-4. Table 2 compares the red channel PSNR performance. Because the RRGB pattern has two more reds, the RRGB pattern performs better than RGGB or RGBB. In the same way, the green channel PSNR is best in the RGGB pattern (Table 3) and the blue channel PSNR is best in the RGBB pattern (Table 4).

(Table 2)

(Table 3)

(Table 4)

Figures 6 to 8 show the frequency response of the designed filter.

6 (a) is a red channel frequency response of RRGB case for # 104 image, (b) is a green channel frequency response of RRGB case for # 104 image, (c) (D) represents the red channel frequency response of the RRGB case for the # 105 image.

Figure 7 (a) shows the red channel frequency response for the # 104 image, (b) shows the green channel frequency response for the RGGB case for the # 104 image, (D) represents the red channel frequency response of the RGGB case for the # 105 image.

8B is a green channel frequency response of the case of RGBB to the image of # 104, and FIG. 8C is a case of RGBB of the image of # 104 (D) represents the red channel frequency response of the RGBB case for the # 105 image.

9 (a) shows a simulation result image for CFA image of # 23 LC image. Figures 9 (b) and 9 (c) show the green and blue components of Figure 9 (a). 9 (d) shows the restored red component. Therefore, this information is not actual information. Figure 9 (e) shows the demosaiced image very visually close to the original image shown in Figure 9 (f). Figure 9 (f) shows the original image.

A similar experiment was performed on the # 24 LC image.

10 (a) shows a simulation result image of the CFA image of the # 24 LC image. Figures 10 (b) and 10 (c) show the green and blue components of Figure 10 (a). 10 (d) shows the restored red component. Fig. 10 (e) shows a demosaiced image which is very visually close to the original image shown in Fig. 10 (f). Figure 10 (f) shows the original image.

The present invention shows a modified Bayer pattern color filter arrangement demosaicing method.

The demosaicking method is assumed to be a super-resolution approach in color images, which is employed in digital cameras to restore full color 3-channel images.

In a pair of Bayer pattern CFA, there are two green pixels, one red pixel and one blue pixel. This is called the RGGB format. However, the following alternatives can be found. Its format is RRGB or RGBB.

In the present invention, the effects of three CFA scenarios were investigated. Experimental results show that the obtained filter provides satisfactory performance.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

10: Database 100: Training Filter Design Module
110: video input unit 120: down sampling unit
130: Signal processing section 140: Filter design section
200: performance verification module 210:
220: Downsampling unit 230:
240:

Claims (10)

A database storing RRGB, RGGB, and RGBB patterns for a plurality of training images and target images, and storing actual values of R, G, and B for each pixel of each training image and a target image;
RRGB, RGGB, and RGBB patterns for a plurality of training images are received from the database, and R, G, and B patterns of the training images are separated into R, G, and B channels for each channel. G, and B for each pixel of each training image and a target image in the database to form a training image by forming low resolution images for each channel by downsampling the image, Training filter design module to design; And
A color filter array to be tested is selected from among a plurality of color filter arrays having different patterns, and a target image photographed by an image sensor having the selected color filter array is inputted from the database and R, G, B Each channel is divided into channels and down-sampled for each channel to form low-resolution images for each channel. The target image is input to the designed training filter to obtain a reconstructed image, and the target image and the reconstructed image are compared And a performance verification module for verifying the performance of the color filter array. The method according to claim 1,
The training filter design module
An image input unit receiving RRGB, RGGB, and RGBB patterns for a plurality of training images in the database;
The RRGB, RGGB, and RGBB patterns of the training images input from the image input unit are divided into R, G, and B channels for each channel, and down-sampled images for each channel are separated into low resolution images A down-sampling unit for forming a plurality of sub- And
The downsampling unit separates R, G, and B channels of R, G, and B patterns of each training image and forms low resolution images for each channel down-sampled for each R, G and B channels to form a reconstructed image. And a filter designing unit for designating a training filter by receiving measured values of each of R, G and B for each pixel of the target image. The method according to claim 2,
The training filter design module
And a signal processing unit for performing edge detection, blurring, sharpening, and embossing on the downsampled image output from the downsampling unit. The method according to claim 2,
Wherein the filter designing unit designates a training filter through a process of minimizing a square error between a real pixel value of the training image and a restored pixel value of the restored image. The method according to claim 1,
The performance verification module
An image input unit for selecting a color filter array to be tested for performance among a plurality of color filter arrays having different patterns and receiving a target image photographed by an image sensor having the selected color filter array in the database;
The R, G, and B channels of the R, G, B and B patterns of the plurality of target images input from the image input unit are divided into the R, G, and B channels for each channel, and the separated images for each channel are down- A down-sampling unit for forming a plurality of sub-
A restored image acquiring unit for acquiring a reconstructed image by inputting the target image into a designed training filter; And
And a performance verifying unit comparing the target image and the restored image to verify the performance of the color filter array when the restored image is obtained through the training filter. (A) storing RRGB, RGGB and RGBB patterns for a plurality of training images and target images in a database, and storing measured values for each of R, G, and B for each pixel of each training image and a target image ;
(B) The training filter design module receives RRGB, RGGB, and RGBB patterns for a plurality of training images in the database, and acquires R, G, and B channel images for RRGB, RGGB, G, and B for each pixel of each training image and the target image in the database by forming low resolution images for each channel by downsampling the separated images for each channel, Designing a training filter by receiving an actual value; And
(C) The performance verification module selects a color filter array to verify the performance among a plurality of color filter arrays having different patterns, and stores the target image captured by the image sensor having the selected color filter array in a database It separates the images into R, G, and B channels by channel, downsamples the separated images for each channel, forms low resolution images for each channel, inputs the target images to the designed training filters, And comparing the image with the restored image to verify the performance of the color filter array. The method of claim 6,
The step (B)
(B-1) receiving the RRGB, RGGB, and RGBB patterns for a plurality of training images in the training filter design module;
(B-2) The training filter design module separates the R, G, and B channels of the RRGB, RGGB, and RGBB patterns of each of the input training images into channels, Forming low resolution images for each channel by sampling; And
(B-3) The training filter design module separates R, G, and B channels of the RRGB, RGGB, and RGBB patterns of each training image into low resolution images for each channel, And designing a training filter by receiving actual values of R, G, and B for each pixel of each training image and a target image in the database, and designing a training filter. The method of claim 7,
Further comprising performing edge detection, blurring, sharpening, and embossing on the downsampled image after the step (B-2). The method of claim 7,
Wherein the step (B-3) comprises the steps of minimizing a square error between an actual pixel value of the training image and a restored pixel value of the restored image. The method of claim 6,
The step (C)
(C-1) The performance verification module selects a color filter array to verify performance among a plurality of color filter arrays having different patterns, and outputs the target image captured by the image sensor having the selected color filter array Receiving input from the database;
(C-2) The performance verification module separates the R, G, and B channels into R, G, and B channels for each RRGB, RGGB, and RGBB patterns of the plurality of target images input, Forming low resolution images for each channel;
(C-3) the performance verification module inputs the target image to a designed training filter to obtain a restored image; And
(C-4) The performance verification module checks the performance of the color filter array by comparing the target image and the restored image when the restored image is obtained through the training filter.

Images