CN111681191A - Color image demosaicing method and system based on FPGA and storage medium - Google Patents
Color image demosaicing method and system based on FPGA and storage medium Download PDFInfo
- Publication number
- CN111681191A CN111681191A CN202010391687.1A CN202010391687A CN111681191A CN 111681191 A CN111681191 A CN 111681191A CN 202010391687 A CN202010391687 A CN 202010391687A CN 111681191 A CN111681191 A CN 111681191A
- Authority
- CN
- China
- Prior art keywords
- channel
- color
- pixel point
- component
- reconstructing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000012545 processing Methods 0.000 claims abstract description 9
- 238000004364 calculation method Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 9
- 101100248200 Arabidopsis thaliana RGGB gene Proteins 0.000 claims description 3
- 238000004422 calculation algorithm Methods 0.000 abstract description 6
- 238000004891 communication Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 6
- 230000009471 action Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration by the use of local operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
Abstract
The color image demosaicing method, the color image demosaicing system and the storage medium based on the FPGA can solve the technical problem that the reconstructed image quality is poor due to the fact that the correlation among different color channels cannot be effectively utilized for traditional algorithms such as bilinear interpolation. Comprising the following steps implemented by a computer device: s100, determining the arrangement mode of the CFA array; s200, determining whether a color channel of a pixel point in a bayer image is an R channel, a G channel or a B channel; s300, acquiring pixel point information in a neighborhood around a pixel point to be reconstructed; s400, reconstructing a GB color component of the R channel; s500, reconstructing an RB color component of the G channel; and S600, reconstructing the RG color component of the B channel. The invention fully utilizes the correlation and gradient information among different color channels and can reconstruct and obtain a high-quality full-color image. Meanwhile, two pixel points can be processed in one clock period, and the processing speed of the algorithm is obviously improved.
Description
Technical Field
The invention relates to the technical field of digital images, in particular to a color image demosaicing method and system based on an FPGA.
Background
For a color image, each pixel point needs to acquire data of three channels of RGB. And image sensors such as CCD and CMOS are used in cooperation with a color filter array CFA, and each pixel point can only obtain color data of one channel in RGB channels, namely an original image in raw format. And reconstructing a full color image aiming at incomplete sampling output by the CFA color channel, namely reconstructing a complete RGB three-color channel of each pixel, namely a demosaicing processing process of the color image. For the traditional algorithms such as bilinear interpolation, the quality of the reconstructed image is not ideal because the correlation between different color channels cannot be effectively utilized.
Disclosure of Invention
The color image demosaicing method and system based on the FPGA can solve the technical problem that reconstructed images are poor in quality due to the fact that correlation among different color channels cannot be effectively utilized for traditional algorithms such as bilinear interpolation.
In order to achieve the purpose, the invention adopts the following technical scheme:
a color image demosaicing method based on an FPGA comprises the following steps:
implementing, by a computer device, the steps of:
s100, determining the arrangement mode of the CFA array;
s200, determining whether a color channel of a pixel point in a bayer image is an R channel, a G channel or a B channel;
s300, acquiring pixel point information in a neighborhood around a pixel point to be reconstructed;
s400, reconstructing a GB color component of the R channel;
s500, reconstructing an RB color component of the G channel;
and S600, reconstructing the RG color component of the B channel.
Further, the S100 determines an arrangement of the CFA array;
the method specifically comprises the following steps:
it is first determined which CFA array the CCD or CMOS image sensor employs: RGGB, GRBG, GBRG, BGGR.
Further, the S200 determines whether a color channel of a pixel point in the bayer image is an R channel, a G channel, or a B channel;
the method specifically comprises the following steps:
and determining whether the color channel corresponding to each pixel point in the input bayer image is an R channel, a G channel or a B channel by counting rows and columns of the input image and combining the corresponding CFA array.
Further, the S300 obtains pixel information in a neighborhood around the pixel to be reconstructed;
the method specifically comprises the following steps:
for an R channel in a bayer image, data of a G channel and data of a B channel of the bayer image need to be reconstructed;
for a G channel in a bayer image, data of an R channel and data of a B channel of the bayer image need to be reconstructed;
for a B channel in a bayer image, data of an R channel and data of a G channel need to be reconstructed;
the method comprises the following steps of processing the pixel points to be reconstructed by utilizing the information of 24 pixel points in total, namely 5 rows and 5 columns around the pixel points to be reconstructed, wherein the method comprises the following steps:
simultaneously acquiring data of a pixel point to be reconstructed and 24 pixel points in the neighborhood of the pixel point in a clock period;
a RAM or FIFO buffer with the size of 5 lines of pixels is arranged in the FPGA, and after the set time delay, the data of 25 pixels are obtained in a pipeline mode in each clock period.
Further, the S400 reconstructs a GB color component of the R channel;
the method specifically comprises the following steps:
reconstructing data of a G component and a B component of an R channel in an original bayer image;
after 24 pixel data of a pixel point to be reconstructed and the neighborhood thereof are obtained, selecting an interpolation parameter, and then carrying out interpolation calculation;
the interpolation parameters and the cached pixel points are in one-to-one correspondence;
setting Xij as an interpolation parameter corresponding to the pixel point Pij, wherein i and j represent the ith row and the jth column in the neighborhood;
for the parameters for reconstructing the G component, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=0,x23=2,x24=0,x25=0;
X31=-1,x32=2,x33=4,x34=2,x35=-1;
X41=0,x42=0,x43=2,x44=0,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0;
for the calculation of the B component, the following parameters are chosen:
x11=0,x12=0,x13=-3/2,x14=0,x15=0;
X21=0,x22=2,x23=0,x24=2,x25=0;
X31=-3/2,x32=0,x33=6,x34=0,x35=-3/2;
X41=0,x42=2,x43=0,x44=2,x45=0;
X51=0,x52=0,x53=-3/2,x54=0,x55=0。
further, S500, reconstructing an RB color component of the G channel;
the method specifically comprises the following steps:
for the calculation of the R component, if the G channel is in R row and B column, the following parameters are selected:
x11=0,x12=0,x13=1/2,x14=0,x15=0;
X21=0,x22=-1,x23=0,x24=-1,x25=0;
X31=-1,x32=4,x33=5,x34=4,x35=-1;
X41=0,x42=-1,x43=0,x44=-1,x45=0;
X51=0,x52=0,x53=1/2,x54=0,x55=0;
if the G channel is in B row and R column, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=-1,x23=4,x24=-1,x25=0;
X31=1/2,x32=0,x33=5,x34=0,x35=1/2;
X41=0,x42=-1,x43=4,x44=-1,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0;
for the calculation of the B component, if the G channel is in R row and B column, the following parameters are selected:
x11=0,x12=0,x13=1/2,x14=0,x15=0;
X21=0,x22=-1,x23=0,x24=-1,x25=0;
X31=-1,x32=4,x33=5,x34=4,x35=-1;
X41=0,x42=-1,x43=0,x44=-1,x45=0;
X51=0,x52=0,x53=1/2,x54=0,x55=0;
if the G channel is in B row and R column, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=-1,x23=4,x24=-1,x25=0;
X31=1/2,x32=0,x33=5,x34=0,x35=1/2;
X41=0,x42=-1,x43=4,x44=-1,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0。
further, the S600 reconstructs an RG color component of the B channel;
the method specifically comprises the following steps:
for the calculation of the R component, the following parameters were chosen:
x11=0,x12=0,x13=-3/2,x14=0,x15=0;
X21=0,x22=2,x23=0,x24=2,x25=0;
X31=-3/2,x32=0,x33=6,x34=0,x35=-3/2;
X41=0,x42=2,x43=0,x44=2,x45=0;
X51=0,x52=0,x53=-3/2,x54=0,x55=0;
for the calculation of the G component, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=0,x23=2,x24=0,x25=0;
X31=-1,x32=2,x33=4,x34=2,x35=-1;
X41=0,x42=0,x43=2,x44=0,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0。
on the other hand, the invention also discloses a color image demosaicing system based on the FPGA,
the method comprises the following units:
a CFA array arrangement determining unit for determining the arrangement of the CFA array;
the RGB determining unit is used for determining whether the color channel of the pixel point in the bayer image is an R channel, a G channel or a B channel;
the neighborhood pixel information acquisition unit is used for acquiring pixel point information in a neighborhood around a pixel point to be reconstructed;
and the color component reconstruction unit is used for reconstructing a GB color component of the R channel, reconstructing an RB color component of the G channel and reconstructing an RG color component of the B channel.
Meanwhile, the invention also discloses a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and when the computer program is executed by a processor, the color image demosaicing method based on the FPGA is realized.
According to the technical scheme, the color image demosaicing method based on the FPGA fully utilizes correlation and gradient information among different color channels, and can reconstruct and obtain a high-quality full-color image. Meanwhile, two pixel points can be processed in one clock period, and the processing speed of the algorithm is obviously improved.
Drawings
FIG. 1 is a flow chart of a method of the present invention;
FIG. 2 is a schematic diagram of a pixel buffer according to the present invention;
FIG. 3 is a schematic diagram of the structure of interpolation parameters according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention.
As shown in fig. 1, the color image demosaicing method based on FPGA according to this embodiment includes the following steps implemented by a computer device:
s100, determining the arrangement mode of the CFA array;
s200, determining whether a color channel of a pixel point in a bayer image is an R channel, a G channel or a B channel;
s300, acquiring pixel point information in a neighborhood around a pixel point to be reconstructed;
s400, reconstructing a GB color component of the R channel;
s500, reconstructing an RB color component of the G channel;
and S600, reconstructing the RG color component of the B channel.
The following detailed description is made with reference to the accompanying drawings:
1. determining the arrangement mode of CFA array
The CFA array has four arrangement modes of RGGB, GRBG, GBRG and BGGR, and firstly, which CFA array is adopted by the CCD or CMOS image sensor is determined.
2. Determining color channels of pixel points in bayer image
By counting rows and columns of the input image and combining the corresponding CFA array, whether the color channel corresponding to each pixel point in the input bayer image is an R channel, a G channel or a B channel can be determined.
3. Obtaining neighborhood pixel information
For an R channel in a bayer image, data of a G channel and data of a B channel of the bayer image need to be reconstructed; for a G channel in a bayer image, data of an R channel and data of a B channel of the bayer image need to be reconstructed; for the B channel in the bayer image, data of its R channel and G channel needs to be reconstructed. In order to obtain a better reconstruction effect, the information of the pixel points in the neighborhood around the pixel point to be reconstructed needs to be fully utilized. The invention utilizes the information of 24 pixels in total in 5 rows and 5 columns around the pixel to be reconstructed to process the pixel to be reconstructed. In order to exert the advantage of parallel processing of the FPGA, the data of the pixel to be reconstructed and the 24 pixels in the neighborhood thereof need to be acquired simultaneously in one clock cycle. Since the image data input to the FPGA is input line by line, it is impossible to acquire 5 lines of pixel data at the same time. Even though buffering is performed by DDR, if DDR is read randomly, efficiency of DDR reading is extremely low, and it is difficult to meet a real-time processing requirement in a video. Therefore, a RAM or FIFO buffer with a pixel size of 5 lines needs to be arranged inside the FPGA, and after a certain time delay, data of 25 pixels can be obtained in a pipeline manner in each clock cycle, as shown in fig. 2, where p33 is a pixel to be reconstructed.
4. Reconstructing GB color component of R channel
For the R channel in the original bayer image, data of its G component and B component needs to be reconstructed. After 24 pixel data of the pixel point to be reconstructed and the neighborhood thereof are obtained, a proper interpolation parameter needs to be selected, and then interpolation calculation is carried out. The structure of the interpolation parameter is shown in fig. 3, and the interpolation parameter corresponds to the pixel points cached in fig. 2 one by one.
For parameters for reconstructing the G component, through simulation and experiment, the following parameters are selected:
setting Xij as an interpolation parameter corresponding to the pixel point Pij, wherein i and j represent the ith row and the jth column in the neighborhood;
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=0,x23=2,x24=0,x25=0;
X31=-1,x32=2,x33=4,x34=2,x35=-1;
X41=0,x42=0,x43=2,x44=0,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0;
for the calculation of the B component, the following parameters were selected through simulation and experiment:
x11=0,x12=0,x13=-3/2,x14=0,x15=0;
X21=0,x22=2,x23=0,x24=2,x25=0;
X31=-3/2,x32=0,x33=6,x34=0,x35=-3/2;
X41=0,x42=2,x43=0,x44=2,x45=0;
X51=0,x52=0,x53=-3/2,x54=0,x55=0;
5. reconstructing RB color components of G channel
Because the G component has two positions in the original image: the R row, B column and B row, R column, because the pixel arrangement of the surrounding neighborhood is different, when reconstructing RB component in G channel, the selected parameters are also different.
For the calculation of the R component, if the G channel is in R row and B column, the following parameters are selected:
x11=0,x12=0,x13=1/2,x14=0,x15=0;
X21=0,x22=-1,x23=0,x24=-1,x25=0;
X31=-1,x32=4,x33=5,x34=4,x35=-1;
X41=0,x42=-1,x43=0,x44=-1,x45=0;
X51=0,x52=0,x53=1/2,x54=0,x55=0;
if the G channel is in B row and R column, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=-1,x23=4,x24=-1,x25=0;
X31=1/2,x32=0,x33=5,x34=0,x35=1/2;
X41=0,x42=-1,x43=4,x44=-1,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0;
for the calculation of the B component, if the G channel is in R row and B column, the following parameters are selected:
x11=0,x12=0,x13=1/2,x14=0,x15=0;
X21=0,x22=-1,x23=0,x24=-1,x25=0;
X31=-1,x32=4,x33=5,x34=4,x35=-1;
X41=0,x42=-1,x43=0,x44=-1,x45=0;
X51=0,x52=0,x53=1/2,x54=0,x55=0;
if the G channel is in B row and R column, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=-1,x23=4,x24=-1,x25=0;
X31=1/2,x32=0,x33=5,x34=0,x35=1/2;
X41=0,x42=-1,x43=4,x44=-1,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0;
6. reconstructing RG components of B channels
As described above, for the calculation of the R component, the following parameters are selected:
x11=0,x12=0,x13=-3/2,x14=0,x15=0;
X21=0,x22=2,x23=0,x24=2,x25=0;
X31=-3/2,x32=0,x33=6,x34=0,x35=-3/2;
X41=0,x42=2,x43=0,x44=2,x45=0;
X51=0,x52=0,x53=-3/2,x54=0,x55=0;
for the calculation of the G component, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=0,x23=2,x24=0,x25=0;
X31=-1,x32=2,x33=4,x34=2,x35=-1;
X41=0,x42=0,x43=2,x44=0,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0。
in summary, in the color image demosaicing method based on the FPGA of the embodiment of the present invention, the arrangement manner of the CFA array is determined first, and then interpolation parameters adopted by each channel are reconstructed for different pixel point positions, and the parameters can fully utilize correlation and gradient information between each color channel. And reconstructing a pixel point, wherein pixel information of 5 rows and 5 columns in the surrounding neighborhood is needed, so that a 5-row buffer needs to be designed in the FPGA, so that data of 25 pixel points in total of 5 rows and 5 columns in the surrounding neighborhood of the pixel point to be processed can be simultaneously acquired in one clock cycle. And finally, setting a counter, determining the position of each pixel point in the whole frame of image, and calculating by using a corresponding interpolation parameter.
The color image demosaicing method based on the FPGA fully utilizes the correlation and gradient information among different color channels, and can reconstruct and obtain a high-quality full-color image. Meanwhile, two pixel points can be processed in one clock period, and the processing speed of the algorithm is obviously improved.
On the other hand, the embodiment of the invention discloses a color image demosaicing system based on FPGA,
the method comprises the following units:
a CFA array arrangement determining unit for determining the arrangement of the CFA array;
the RGB determining unit is used for determining whether the color channel of the pixel point in the bayer image is an R channel, a G channel or a B channel;
the neighborhood pixel information acquisition unit is used for acquiring pixel point information in a neighborhood around a pixel point to be reconstructed;
and the color component reconstruction unit is used for reconstructing a GB color component of the R channel, reconstructing an RB color component of the G channel and reconstructing an RG color component of the B channel.
It is understood that the system provided by the embodiment of the present invention corresponds to the method provided by the embodiment of the present invention, and the explanation, the example and the beneficial effects of the related contents can refer to the corresponding parts in the method.
The embodiment of the application also provides an electronic device, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus,
a memory for storing a computer program;
the processor is used for realizing the color image demosaicing method based on the FPGA when executing the program stored in the memory, and the method comprises the following steps:
s100, determining the arrangement mode of the CFA array;
s200, determining whether a color channel of a pixel point in a bayer image is an R channel, a G channel or a B channel;
s300, acquiring pixel point information in a neighborhood around a pixel point to be reconstructed;
s400, reconstructing a GB color component of the R channel;
s500, reconstructing an RB color component of the G channel;
and S600, reconstructing the RG color component of the B channel.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other Programmable logic devices, discrete Gate or transistor logic devices, or discrete hardware components.
In yet another embodiment provided by the present application, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any of the above-mentioned FPGA-based color image demosaicing methods.
In yet another embodiment provided by the present application, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the above embodiments of the FPGA-based color image demosaicing method.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (9)
1. A color image demosaicing method based on FPGA is characterized in that: implementing, by a computer device, the steps of:
s100, determining the arrangement mode of the CFA array;
s200, determining whether a color channel of a pixel point in a bayer image is an R channel, a G channel or a B channel;
s300, acquiring pixel point information in a neighborhood around a pixel point to be reconstructed;
s400, reconstructing a GB color component of the R channel;
s500, reconstructing an RB color component of the G channel;
and S600, reconstructing the RG color component of the B channel.
2. The FPGA-based color image demosaicing method of claim 1, wherein: the S100 determines the arrangement mode of the CFA array;
the method specifically comprises the following steps:
it is first determined which CFA array the CCD or CMOS image sensor employs: RGGB, GRBG, GBRG, BGGR.
3. The FPGA-based color image demosaicing method of claim 2, wherein: the S200 determines whether the color channel of the pixel point in the bayer image is an R channel, a G channel or a B channel;
the method specifically comprises the following steps:
and determining whether the color channel corresponding to each pixel point in the input bayer image is an R channel, a G channel or a B channel by counting rows and columns of the input image and combining the corresponding CFA array.
4. The FPGA-based color image demosaicing method of claim 3, wherein: the S300 acquires pixel point information in the neighborhood around the pixel point to be reconstructed;
the method specifically comprises the following steps:
for an R channel in a bayer image, data of a G channel and data of a B channel of the bayer image need to be reconstructed;
for a G channel in a bayer image, data of an R channel and data of a B channel of the bayer image need to be reconstructed;
for a B channel in a bayer image, data of an R channel and data of a G channel need to be reconstructed;
the method comprises the following steps of processing the pixel points to be reconstructed by utilizing the information of 24 pixel points in total, namely 5 rows and 5 columns around the pixel points to be reconstructed, wherein the method comprises the following steps:
simultaneously acquiring data of a pixel point to be reconstructed and 24 pixel points in the neighborhood of the pixel point in a clock period;
a RAM or FIFO buffer with the size of 5 lines of pixels is arranged in the FPGA, and after the set time delay, the data of 25 pixels are obtained in a pipeline mode in each clock period.
5. The FPGA-based color image demosaicing method of claim 4, wherein: the S400 reconstructs GB color components of the R channel;
the method specifically comprises the following steps:
reconstructing data of a G component and a B component of an R channel in an original bayer image;
after 24 pixel data of a pixel point to be reconstructed and the neighborhood thereof are obtained, selecting an interpolation parameter, and then carrying out interpolation calculation;
the interpolation parameters and the cached pixel points are in one-to-one correspondence;
setting Xij as an interpolation parameter corresponding to the pixel point Pij, wherein i and j represent the ith row and the jth column in the neighborhood;
for the parameters for reconstructing the G component, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=0,x23=2,x24=0,x25=0;
X31=-1,x32=2,x33=4,x34=2,x35=-1;
X41=0,x42=0,x43=2,x44=0,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0;
for the calculation of the B component, the following parameters are chosen:
x11=0,x12=0,x13=-3/2,x14=0,x15=0;
X21=0,x22=2,x23=0,x24=2,x25=0;
X31=-3/2,x32=0,x33=6,x34=0,x35=-3/2;
X41=0,x42=2,x43=0,x44=2,x45=0;
X51=0,x52=0,x53=-3/2,x54=0,x55=0。
6. the FPGA-based color image demosaicing method of claim 5, wherein: s500, reconstructing an RB color component of the G channel;
the method specifically comprises the following steps:
for the calculation of the R component, if the G channel is in R row and B column, the following parameters are selected:
x11=0,x12=0,x13=1/2,x14=0,x15=0;
X21=0,x22=-1,x23=0,x24=-1,x25=0;
X31=-1,x32=4,x33=5,x34=4,x35=-1;
X41=0,x42=-1,x43=0,x44=-1,x45=0;
X51=0,x52=0,x53=1/2,x54=0,x55=0;
if the G channel is in B row and R column, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=-1,x23=4,x24=-1,x25=0;
X31=1/2,x32=0,x33=5,x34=0,x35=1/2;
X41=0,x42=-1,x43=4,x44=-1,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0;
for the calculation of the B component, if the G channel is in R row and B column, the following parameters are selected:
x11=0,x12=0,x13=1/2,x14=0,x15=0;
X21=0,x22=-1,x23=0,x24=-1,x25=0;
X31=-1,x32=4,x33=5,x34=4,x35=-1;
X41=0,x42=-1,x43=0,x44=-1,x45=0;
X51=0,x52=0,x53=1/2,x54=0,x55=0;
if the G channel is in B row and R column, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=-1,x23=4,x24=-1,x25=0;
X31=1/2,x32=0,x33=5,x34=0,x35=1/2;
X41=0,x42=-1,x43=4,x44=-1,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0。
7. the FPGA-based color image demosaicing method of claim 6, wherein: s600, reconstructing an RG color component of a B channel;
the method specifically comprises the following steps:
for the calculation of the R component, the following parameters were chosen:
x11=0,x12=0,x13=-3/2,x14=0,x15=0;
X21=0,x22=2,x23=0,x24=2,x25=0;
X31=-3/2,x32=0,x33=6,x34=0,x35=-3/2;
X41=0,x42=2,x43=0,x44=2,x45=0;
X51=0,x52=0,x53=-3/2,x54=0,x55=0;
for the calculation of the G component, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=0,x23=2,x24=0,x25=0;
X31=-1,x32=2,x33=4,x34=2,x35=-1;
X41=0,x42=0,x43=2,x44=0,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0。
8. the color image demosaicing system based on the FPGA is characterized in that:
the method comprises the following units:
a CFA array arrangement determining unit for determining the arrangement of the CFA array;
the RGB determining unit is used for determining whether the color channel of the pixel point in the bayer image is an R channel, a G channel or a B channel;
the neighborhood pixel information acquisition unit is used for acquiring pixel point information in a neighborhood around a pixel point to be reconstructed;
and the color component reconstruction unit is used for reconstructing a GB color component of the R channel, reconstructing an RB color component of the G channel and reconstructing an RG color component of the B channel.
9. A computer-readable storage medium characterized by: the computer-readable storage medium has stored therein a computer program which, when executed by a processor, implements the FPGA-based color image demosaicing method of any of claims 1-7.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010391687.1A CN111681191B (en) | 2020-05-11 | 2020-05-11 | Color image demosaicing method, system and storage medium based on FPGA |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010391687.1A CN111681191B (en) | 2020-05-11 | 2020-05-11 | Color image demosaicing method, system and storage medium based on FPGA |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111681191A true CN111681191A (en) | 2020-09-18 |
CN111681191B CN111681191B (en) | 2024-02-27 |
Family
ID=72433374
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010391687.1A Active CN111681191B (en) | 2020-05-11 | 2020-05-11 | Color image demosaicing method, system and storage medium based on FPGA |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111681191B (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116342394A (en) * | 2023-05-30 | 2023-06-27 | 之江实验室 | Real-time image demosaicing method, device and medium based on FPGA |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060104505A1 (en) * | 2004-11-15 | 2006-05-18 | Chih-Lung Chen | Demosaicking method and apparatus for color filter array interpolation in digital image acquisition systems |
US20070177033A1 (en) * | 2006-01-30 | 2007-08-02 | Microsoft Corporation | Bayesian demosaicing using a two-color image |
CN102663703A (en) * | 2012-04-20 | 2012-09-12 | 西安电子科技大学 | Treelet-based Bayer type CFA image denoising method |
CN109636726A (en) * | 2018-12-14 | 2019-04-16 | 中国科学院长春光学精密机械与物理研究所 | Regular hexagon pixel color filter coefficients method and device |
CN110009590A (en) * | 2019-04-12 | 2019-07-12 | 北京理工大学 | A kind of high-quality colour image demosaicing methods based on convolutional neural networks |
-
2020
- 2020-05-11 CN CN202010391687.1A patent/CN111681191B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060104505A1 (en) * | 2004-11-15 | 2006-05-18 | Chih-Lung Chen | Demosaicking method and apparatus for color filter array interpolation in digital image acquisition systems |
US20070177033A1 (en) * | 2006-01-30 | 2007-08-02 | Microsoft Corporation | Bayesian demosaicing using a two-color image |
CN102663703A (en) * | 2012-04-20 | 2012-09-12 | 西安电子科技大学 | Treelet-based Bayer type CFA image denoising method |
CN109636726A (en) * | 2018-12-14 | 2019-04-16 | 中国科学院长春光学精密机械与物理研究所 | Regular hexagon pixel color filter coefficients method and device |
CN110009590A (en) * | 2019-04-12 | 2019-07-12 | 北京理工大学 | A kind of high-quality colour image demosaicing methods based on convolutional neural networks |
Non-Patent Citations (2)
Title |
---|
孙辉;柏旭光;孙丽娜;李志强;: "Bayer图像色彩还原线性插值方法" * |
尹勇;胡磊;: "一种改进的Bayer图像彩色恢复差值算法" * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116342394A (en) * | 2023-05-30 | 2023-06-27 | 之江实验室 | Real-time image demosaicing method, device and medium based on FPGA |
CN116342394B (en) * | 2023-05-30 | 2023-08-18 | 之江实验室 | Real-time image demosaicing method, device and medium based on FPGA |
Also Published As
Publication number | Publication date |
---|---|
CN111681191B (en) | 2024-02-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111080528B (en) | Image super-resolution and model training method and device, electronic equipment and medium | |
CN109658337B (en) | FPGA implementation method for real-time electronic despinning of images | |
US20210327033A1 (en) | Video processing method and apparatus, and computer storage medium | |
CN107680043B (en) | Single image super-resolution output method based on graph model | |
WO2022267494A1 (en) | Image data generation method and apparatus | |
Licciardo et al. | Stream processor for real-time inverse Tone Mapping of Full-HD images | |
WO2023160426A1 (en) | Video frame interpolation method and apparatus, training method and apparatus, and electronic device | |
CN111681191B (en) | Color image demosaicing method, system and storage medium based on FPGA | |
Cadenas et al. | Parallel pipelined array architectures for real-time histogram computation in consumer devices | |
CN102611842A (en) | Image processing devices and image processing methods | |
KR101140953B1 (en) | Method and apparatus for correcting distorted image | |
CN108024074B (en) | Miniaturized infrared imaging method based on SOPC | |
Toczek et al. | Scene-based non-uniformity correction: from algorithm to implementation on a smart camera | |
WO2024041027A1 (en) | Panoramic-image processing method, and computer device and storage medium | |
CN116342394B (en) | Real-time image demosaicing method, device and medium based on FPGA | |
WO2023082685A1 (en) | Video enhancement method and apparatus, and computer device and storage medium | |
CN106558021A (en) | Video enhancement method based on super-resolution technique | |
Blasinski et al. | FPGA architecture for real-time barrel distortion correction of colour images | |
WO2021233232A1 (en) | Image processing method and image processing model training method and device | |
CN104776919B (en) | Infrared focal plane array ribbon Nonuniformity Correction system and method based on FPGA | |
US11270412B2 (en) | Image signal processor, method, and system for environmental mapping | |
TW200818908A (en) | Image processing apparatus and method | |
CN110895790B (en) | Scene image super-resolution method based on posterior degradation information estimation | |
Fuentes et al. | FPGA implementation of the bilinear interpolation algorithm for image demosaicking | |
Azgin et al. | A high performance alternating projections image demosaicing hardware |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
CB02 | Change of applicant information |
Country or region after: China Address after: Room 707-710, 7th floor, building B3, innovation industrial park, No. 800, Wangjiang West Road, hi tech Zone, Hefei City, Anhui Province Applicant after: Anhui Baicheng Huitong Technology Co.,Ltd. Address before: Room 707-710, 7th floor, building B3, innovation industrial park, No. 800, Wangjiang West Road, hi tech Zone, Hefei City, Anhui Province Applicant before: ANHUI BAI CHENG HUI TONG TECHNOLOGY CO.,LTD. Country or region before: China |
|
CB02 | Change of applicant information | ||
GR01 | Patent grant | ||
GR01 | Patent grant |