CN111681191A - Color image demosaicing method and system based on FPGA and storage medium - Google Patents

Color image demosaicing method and system based on FPGA and storage medium Download PDF

Info

Publication number
CN111681191A
CN111681191A CN202010391687.1A CN202010391687A CN111681191A CN 111681191 A CN111681191 A CN 111681191A CN 202010391687 A CN202010391687 A CN 202010391687A CN 111681191 A CN111681191 A CN 111681191A
Authority
CN
China
Prior art keywords
channel
color
pixel point
component
reconstructing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010391687.1A
Other languages
Chinese (zh)
Other versions
CN111681191B (en
Inventor
汪俊锋
朱文佳
张晓慧
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Anhui Bai Cheng Hui Tong Technology Co ltd
Original Assignee
Anhui Bai Cheng Hui Tong Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Anhui Bai Cheng Hui Tong Technology Co ltd filed Critical Anhui Bai Cheng Hui Tong Technology Co ltd
Priority to CN202010391687.1A priority Critical patent/CN111681191B/en
Publication of CN111681191A publication Critical patent/CN111681191A/en
Application granted granted Critical
Publication of CN111681191B publication Critical patent/CN111681191B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/20Image enhancement or restoration by the use of local operators
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/90Determination of colour characteristics

Abstract

The color image demosaicing method, the color image demosaicing system and the storage medium based on the FPGA can solve the technical problem that the reconstructed image quality is poor due to the fact that the correlation among different color channels cannot be effectively utilized for traditional algorithms such as bilinear interpolation. Comprising the following steps implemented by a computer device: s100, determining the arrangement mode of the CFA array; s200, determining whether a color channel of a pixel point in a bayer image is an R channel, a G channel or a B channel; s300, acquiring pixel point information in a neighborhood around a pixel point to be reconstructed; s400, reconstructing a GB color component of the R channel; s500, reconstructing an RB color component of the G channel; and S600, reconstructing the RG color component of the B channel. The invention fully utilizes the correlation and gradient information among different color channels and can reconstruct and obtain a high-quality full-color image. Meanwhile, two pixel points can be processed in one clock period, and the processing speed of the algorithm is obviously improved.

Description

Color image demosaicing method and system based on FPGA and storage medium
Technical Field
The invention relates to the technical field of digital images, in particular to a color image demosaicing method and system based on an FPGA.
Background
For a color image, each pixel point needs to acquire data of three channels of RGB. And image sensors such as CCD and CMOS are used in cooperation with a color filter array CFA, and each pixel point can only obtain color data of one channel in RGB channels, namely an original image in raw format. And reconstructing a full color image aiming at incomplete sampling output by the CFA color channel, namely reconstructing a complete RGB three-color channel of each pixel, namely a demosaicing processing process of the color image. For the traditional algorithms such as bilinear interpolation, the quality of the reconstructed image is not ideal because the correlation between different color channels cannot be effectively utilized.
Disclosure of Invention
The color image demosaicing method and system based on the FPGA can solve the technical problem that reconstructed images are poor in quality due to the fact that correlation among different color channels cannot be effectively utilized for traditional algorithms such as bilinear interpolation.
In order to achieve the purpose, the invention adopts the following technical scheme:
a color image demosaicing method based on an FPGA comprises the following steps:
implementing, by a computer device, the steps of:
s100, determining the arrangement mode of the CFA array;
s200, determining whether a color channel of a pixel point in a bayer image is an R channel, a G channel or a B channel;
s300, acquiring pixel point information in a neighborhood around a pixel point to be reconstructed;
s400, reconstructing a GB color component of the R channel;
s500, reconstructing an RB color component of the G channel;
and S600, reconstructing the RG color component of the B channel.
Further, the S100 determines an arrangement of the CFA array;
the method specifically comprises the following steps:
it is first determined which CFA array the CCD or CMOS image sensor employs: RGGB, GRBG, GBRG, BGGR.
Further, the S200 determines whether a color channel of a pixel point in the bayer image is an R channel, a G channel, or a B channel;
the method specifically comprises the following steps:
and determining whether the color channel corresponding to each pixel point in the input bayer image is an R channel, a G channel or a B channel by counting rows and columns of the input image and combining the corresponding CFA array.
Further, the S300 obtains pixel information in a neighborhood around the pixel to be reconstructed;
the method specifically comprises the following steps:
for an R channel in a bayer image, data of a G channel and data of a B channel of the bayer image need to be reconstructed;
for a G channel in a bayer image, data of an R channel and data of a B channel of the bayer image need to be reconstructed;
for a B channel in a bayer image, data of an R channel and data of a G channel need to be reconstructed;
the method comprises the following steps of processing the pixel points to be reconstructed by utilizing the information of 24 pixel points in total, namely 5 rows and 5 columns around the pixel points to be reconstructed, wherein the method comprises the following steps:
simultaneously acquiring data of a pixel point to be reconstructed and 24 pixel points in the neighborhood of the pixel point in a clock period;
a RAM or FIFO buffer with the size of 5 lines of pixels is arranged in the FPGA, and after the set time delay, the data of 25 pixels are obtained in a pipeline mode in each clock period.
Further, the S400 reconstructs a GB color component of the R channel;
the method specifically comprises the following steps:
reconstructing data of a G component and a B component of an R channel in an original bayer image;
after 24 pixel data of a pixel point to be reconstructed and the neighborhood thereof are obtained, selecting an interpolation parameter, and then carrying out interpolation calculation;
the interpolation parameters and the cached pixel points are in one-to-one correspondence;
setting Xij as an interpolation parameter corresponding to the pixel point Pij, wherein i and j represent the ith row and the jth column in the neighborhood;
for the parameters for reconstructing the G component, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=0,x23=2,x24=0,x25=0;
X31=-1,x32=2,x33=4,x34=2,x35=-1;
X41=0,x42=0,x43=2,x44=0,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0;
for the calculation of the B component, the following parameters are chosen:
x11=0,x12=0,x13=-3/2,x14=0,x15=0;
X21=0,x22=2,x23=0,x24=2,x25=0;
X31=-3/2,x32=0,x33=6,x34=0,x35=-3/2;
X41=0,x42=2,x43=0,x44=2,x45=0;
X51=0,x52=0,x53=-3/2,x54=0,x55=0。
further, S500, reconstructing an RB color component of the G channel;
the method specifically comprises the following steps:
for the calculation of the R component, if the G channel is in R row and B column, the following parameters are selected:
x11=0,x12=0,x13=1/2,x14=0,x15=0;
X21=0,x22=-1,x23=0,x24=-1,x25=0;
X31=-1,x32=4,x33=5,x34=4,x35=-1;
X41=0,x42=-1,x43=0,x44=-1,x45=0;
X51=0,x52=0,x53=1/2,x54=0,x55=0;
if the G channel is in B row and R column, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=-1,x23=4,x24=-1,x25=0;
X31=1/2,x32=0,x33=5,x34=0,x35=1/2;
X41=0,x42=-1,x43=4,x44=-1,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0;
for the calculation of the B component, if the G channel is in R row and B column, the following parameters are selected:
x11=0,x12=0,x13=1/2,x14=0,x15=0;
X21=0,x22=-1,x23=0,x24=-1,x25=0;
X31=-1,x32=4,x33=5,x34=4,x35=-1;
X41=0,x42=-1,x43=0,x44=-1,x45=0;
X51=0,x52=0,x53=1/2,x54=0,x55=0;
if the G channel is in B row and R column, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=-1,x23=4,x24=-1,x25=0;
X31=1/2,x32=0,x33=5,x34=0,x35=1/2;
X41=0,x42=-1,x43=4,x44=-1,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0。
further, the S600 reconstructs an RG color component of the B channel;
the method specifically comprises the following steps:
for the calculation of the R component, the following parameters were chosen:
x11=0,x12=0,x13=-3/2,x14=0,x15=0;
X21=0,x22=2,x23=0,x24=2,x25=0;
X31=-3/2,x32=0,x33=6,x34=0,x35=-3/2;
X41=0,x42=2,x43=0,x44=2,x45=0;
X51=0,x52=0,x53=-3/2,x54=0,x55=0;
for the calculation of the G component, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=0,x23=2,x24=0,x25=0;
X31=-1,x32=2,x33=4,x34=2,x35=-1;
X41=0,x42=0,x43=2,x44=0,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0。
on the other hand, the invention also discloses a color image demosaicing system based on the FPGA,
the method comprises the following units:
a CFA array arrangement determining unit for determining the arrangement of the CFA array;
the RGB determining unit is used for determining whether the color channel of the pixel point in the bayer image is an R channel, a G channel or a B channel;
the neighborhood pixel information acquisition unit is used for acquiring pixel point information in a neighborhood around a pixel point to be reconstructed;
and the color component reconstruction unit is used for reconstructing a GB color component of the R channel, reconstructing an RB color component of the G channel and reconstructing an RG color component of the B channel.
Meanwhile, the invention also discloses a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and when the computer program is executed by a processor, the color image demosaicing method based on the FPGA is realized.
According to the technical scheme, the color image demosaicing method based on the FPGA fully utilizes correlation and gradient information among different color channels, and can reconstruct and obtain a high-quality full-color image. Meanwhile, two pixel points can be processed in one clock period, and the processing speed of the algorithm is obviously improved.
Drawings
FIG. 1 is a flow chart of a method of the present invention;
FIG. 2 is a schematic diagram of a pixel buffer according to the present invention;
FIG. 3 is a schematic diagram of the structure of interpolation parameters according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention.
As shown in fig. 1, the color image demosaicing method based on FPGA according to this embodiment includes the following steps implemented by a computer device:
s100, determining the arrangement mode of the CFA array;
s200, determining whether a color channel of a pixel point in a bayer image is an R channel, a G channel or a B channel;
s300, acquiring pixel point information in a neighborhood around a pixel point to be reconstructed;
s400, reconstructing a GB color component of the R channel;
s500, reconstructing an RB color component of the G channel;
and S600, reconstructing the RG color component of the B channel.
The following detailed description is made with reference to the accompanying drawings:
1. determining the arrangement mode of CFA array
The CFA array has four arrangement modes of RGGB, GRBG, GBRG and BGGR, and firstly, which CFA array is adopted by the CCD or CMOS image sensor is determined.
2. Determining color channels of pixel points in bayer image
By counting rows and columns of the input image and combining the corresponding CFA array, whether the color channel corresponding to each pixel point in the input bayer image is an R channel, a G channel or a B channel can be determined.
3. Obtaining neighborhood pixel information
For an R channel in a bayer image, data of a G channel and data of a B channel of the bayer image need to be reconstructed; for a G channel in a bayer image, data of an R channel and data of a B channel of the bayer image need to be reconstructed; for the B channel in the bayer image, data of its R channel and G channel needs to be reconstructed. In order to obtain a better reconstruction effect, the information of the pixel points in the neighborhood around the pixel point to be reconstructed needs to be fully utilized. The invention utilizes the information of 24 pixels in total in 5 rows and 5 columns around the pixel to be reconstructed to process the pixel to be reconstructed. In order to exert the advantage of parallel processing of the FPGA, the data of the pixel to be reconstructed and the 24 pixels in the neighborhood thereof need to be acquired simultaneously in one clock cycle. Since the image data input to the FPGA is input line by line, it is impossible to acquire 5 lines of pixel data at the same time. Even though buffering is performed by DDR, if DDR is read randomly, efficiency of DDR reading is extremely low, and it is difficult to meet a real-time processing requirement in a video. Therefore, a RAM or FIFO buffer with a pixel size of 5 lines needs to be arranged inside the FPGA, and after a certain time delay, data of 25 pixels can be obtained in a pipeline manner in each clock cycle, as shown in fig. 2, where p33 is a pixel to be reconstructed.
4. Reconstructing GB color component of R channel
For the R channel in the original bayer image, data of its G component and B component needs to be reconstructed. After 24 pixel data of the pixel point to be reconstructed and the neighborhood thereof are obtained, a proper interpolation parameter needs to be selected, and then interpolation calculation is carried out. The structure of the interpolation parameter is shown in fig. 3, and the interpolation parameter corresponds to the pixel points cached in fig. 2 one by one.
For parameters for reconstructing the G component, through simulation and experiment, the following parameters are selected:
setting Xij as an interpolation parameter corresponding to the pixel point Pij, wherein i and j represent the ith row and the jth column in the neighborhood;
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=0,x23=2,x24=0,x25=0;
X31=-1,x32=2,x33=4,x34=2,x35=-1;
X41=0,x42=0,x43=2,x44=0,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0;
for the calculation of the B component, the following parameters were selected through simulation and experiment:
x11=0,x12=0,x13=-3/2,x14=0,x15=0;
X21=0,x22=2,x23=0,x24=2,x25=0;
X31=-3/2,x32=0,x33=6,x34=0,x35=-3/2;
X41=0,x42=2,x43=0,x44=2,x45=0;
X51=0,x52=0,x53=-3/2,x54=0,x55=0;
5. reconstructing RB color components of G channel
Because the G component has two positions in the original image: the R row, B column and B row, R column, because the pixel arrangement of the surrounding neighborhood is different, when reconstructing RB component in G channel, the selected parameters are also different.
For the calculation of the R component, if the G channel is in R row and B column, the following parameters are selected:
x11=0,x12=0,x13=1/2,x14=0,x15=0;
X21=0,x22=-1,x23=0,x24=-1,x25=0;
X31=-1,x32=4,x33=5,x34=4,x35=-1;
X41=0,x42=-1,x43=0,x44=-1,x45=0;
X51=0,x52=0,x53=1/2,x54=0,x55=0;
if the G channel is in B row and R column, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=-1,x23=4,x24=-1,x25=0;
X31=1/2,x32=0,x33=5,x34=0,x35=1/2;
X41=0,x42=-1,x43=4,x44=-1,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0;
for the calculation of the B component, if the G channel is in R row and B column, the following parameters are selected:
x11=0,x12=0,x13=1/2,x14=0,x15=0;
X21=0,x22=-1,x23=0,x24=-1,x25=0;
X31=-1,x32=4,x33=5,x34=4,x35=-1;
X41=0,x42=-1,x43=0,x44=-1,x45=0;
X51=0,x52=0,x53=1/2,x54=0,x55=0;
if the G channel is in B row and R column, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=-1,x23=4,x24=-1,x25=0;
X31=1/2,x32=0,x33=5,x34=0,x35=1/2;
X41=0,x42=-1,x43=4,x44=-1,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0;
6. reconstructing RG components of B channels
As described above, for the calculation of the R component, the following parameters are selected:
x11=0,x12=0,x13=-3/2,x14=0,x15=0;
X21=0,x22=2,x23=0,x24=2,x25=0;
X31=-3/2,x32=0,x33=6,x34=0,x35=-3/2;
X41=0,x42=2,x43=0,x44=2,x45=0;
X51=0,x52=0,x53=-3/2,x54=0,x55=0;
for the calculation of the G component, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=0,x23=2,x24=0,x25=0;
X31=-1,x32=2,x33=4,x34=2,x35=-1;
X41=0,x42=0,x43=2,x44=0,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0。
in summary, in the color image demosaicing method based on the FPGA of the embodiment of the present invention, the arrangement manner of the CFA array is determined first, and then interpolation parameters adopted by each channel are reconstructed for different pixel point positions, and the parameters can fully utilize correlation and gradient information between each color channel. And reconstructing a pixel point, wherein pixel information of 5 rows and 5 columns in the surrounding neighborhood is needed, so that a 5-row buffer needs to be designed in the FPGA, so that data of 25 pixel points in total of 5 rows and 5 columns in the surrounding neighborhood of the pixel point to be processed can be simultaneously acquired in one clock cycle. And finally, setting a counter, determining the position of each pixel point in the whole frame of image, and calculating by using a corresponding interpolation parameter.
The color image demosaicing method based on the FPGA fully utilizes the correlation and gradient information among different color channels, and can reconstruct and obtain a high-quality full-color image. Meanwhile, two pixel points can be processed in one clock period, and the processing speed of the algorithm is obviously improved.
On the other hand, the embodiment of the invention discloses a color image demosaicing system based on FPGA,
the method comprises the following units:
a CFA array arrangement determining unit for determining the arrangement of the CFA array;
the RGB determining unit is used for determining whether the color channel of the pixel point in the bayer image is an R channel, a G channel or a B channel;
the neighborhood pixel information acquisition unit is used for acquiring pixel point information in a neighborhood around a pixel point to be reconstructed;
and the color component reconstruction unit is used for reconstructing a GB color component of the R channel, reconstructing an RB color component of the G channel and reconstructing an RG color component of the B channel.
It is understood that the system provided by the embodiment of the present invention corresponds to the method provided by the embodiment of the present invention, and the explanation, the example and the beneficial effects of the related contents can refer to the corresponding parts in the method.
The embodiment of the application also provides an electronic device, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus,
a memory for storing a computer program;
the processor is used for realizing the color image demosaicing method based on the FPGA when executing the program stored in the memory, and the method comprises the following steps:
s100, determining the arrangement mode of the CFA array;
s200, determining whether a color channel of a pixel point in a bayer image is an R channel, a G channel or a B channel;
s300, acquiring pixel point information in a neighborhood around a pixel point to be reconstructed;
s400, reconstructing a GB color component of the R channel;
s500, reconstructing an RB color component of the G channel;
and S600, reconstructing the RG color component of the B channel.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other Programmable logic devices, discrete Gate or transistor logic devices, or discrete hardware components.
In yet another embodiment provided by the present application, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any of the above-mentioned FPGA-based color image demosaicing methods.
In yet another embodiment provided by the present application, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the above embodiments of the FPGA-based color image demosaicing method.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (9)

1. A color image demosaicing method based on FPGA is characterized in that: implementing, by a computer device, the steps of:
s100, determining the arrangement mode of the CFA array;
s200, determining whether a color channel of a pixel point in a bayer image is an R channel, a G channel or a B channel;
s300, acquiring pixel point information in a neighborhood around a pixel point to be reconstructed;
s400, reconstructing a GB color component of the R channel;
s500, reconstructing an RB color component of the G channel;
and S600, reconstructing the RG color component of the B channel.
2. The FPGA-based color image demosaicing method of claim 1, wherein: the S100 determines the arrangement mode of the CFA array;
the method specifically comprises the following steps:
it is first determined which CFA array the CCD or CMOS image sensor employs: RGGB, GRBG, GBRG, BGGR.
3. The FPGA-based color image demosaicing method of claim 2, wherein: the S200 determines whether the color channel of the pixel point in the bayer image is an R channel, a G channel or a B channel;
the method specifically comprises the following steps:
and determining whether the color channel corresponding to each pixel point in the input bayer image is an R channel, a G channel or a B channel by counting rows and columns of the input image and combining the corresponding CFA array.
4. The FPGA-based color image demosaicing method of claim 3, wherein: the S300 acquires pixel point information in the neighborhood around the pixel point to be reconstructed;
the method specifically comprises the following steps:
for an R channel in a bayer image, data of a G channel and data of a B channel of the bayer image need to be reconstructed;
for a G channel in a bayer image, data of an R channel and data of a B channel of the bayer image need to be reconstructed;
for a B channel in a bayer image, data of an R channel and data of a G channel need to be reconstructed;
the method comprises the following steps of processing the pixel points to be reconstructed by utilizing the information of 24 pixel points in total, namely 5 rows and 5 columns around the pixel points to be reconstructed, wherein the method comprises the following steps:
simultaneously acquiring data of a pixel point to be reconstructed and 24 pixel points in the neighborhood of the pixel point in a clock period;
a RAM or FIFO buffer with the size of 5 lines of pixels is arranged in the FPGA, and after the set time delay, the data of 25 pixels are obtained in a pipeline mode in each clock period.
5. The FPGA-based color image demosaicing method of claim 4, wherein: the S400 reconstructs GB color components of the R channel;
the method specifically comprises the following steps:
reconstructing data of a G component and a B component of an R channel in an original bayer image;
after 24 pixel data of a pixel point to be reconstructed and the neighborhood thereof are obtained, selecting an interpolation parameter, and then carrying out interpolation calculation;
the interpolation parameters and the cached pixel points are in one-to-one correspondence;
setting Xij as an interpolation parameter corresponding to the pixel point Pij, wherein i and j represent the ith row and the jth column in the neighborhood;
for the parameters for reconstructing the G component, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=0,x23=2,x24=0,x25=0;
X31=-1,x32=2,x33=4,x34=2,x35=-1;
X41=0,x42=0,x43=2,x44=0,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0;
for the calculation of the B component, the following parameters are chosen:
x11=0,x12=0,x13=-3/2,x14=0,x15=0;
X21=0,x22=2,x23=0,x24=2,x25=0;
X31=-3/2,x32=0,x33=6,x34=0,x35=-3/2;
X41=0,x42=2,x43=0,x44=2,x45=0;
X51=0,x52=0,x53=-3/2,x54=0,x55=0。
6. the FPGA-based color image demosaicing method of claim 5, wherein: s500, reconstructing an RB color component of the G channel;
the method specifically comprises the following steps:
for the calculation of the R component, if the G channel is in R row and B column, the following parameters are selected:
x11=0,x12=0,x13=1/2,x14=0,x15=0;
X21=0,x22=-1,x23=0,x24=-1,x25=0;
X31=-1,x32=4,x33=5,x34=4,x35=-1;
X41=0,x42=-1,x43=0,x44=-1,x45=0;
X51=0,x52=0,x53=1/2,x54=0,x55=0;
if the G channel is in B row and R column, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=-1,x23=4,x24=-1,x25=0;
X31=1/2,x32=0,x33=5,x34=0,x35=1/2;
X41=0,x42=-1,x43=4,x44=-1,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0;
for the calculation of the B component, if the G channel is in R row and B column, the following parameters are selected:
x11=0,x12=0,x13=1/2,x14=0,x15=0;
X21=0,x22=-1,x23=0,x24=-1,x25=0;
X31=-1,x32=4,x33=5,x34=4,x35=-1;
X41=0,x42=-1,x43=0,x44=-1,x45=0;
X51=0,x52=0,x53=1/2,x54=0,x55=0;
if the G channel is in B row and R column, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=-1,x23=4,x24=-1,x25=0;
X31=1/2,x32=0,x33=5,x34=0,x35=1/2;
X41=0,x42=-1,x43=4,x44=-1,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0。
7. the FPGA-based color image demosaicing method of claim 6, wherein: s600, reconstructing an RG color component of a B channel;
the method specifically comprises the following steps:
for the calculation of the R component, the following parameters were chosen:
x11=0,x12=0,x13=-3/2,x14=0,x15=0;
X21=0,x22=2,x23=0,x24=2,x25=0;
X31=-3/2,x32=0,x33=6,x34=0,x35=-3/2;
X41=0,x42=2,x43=0,x44=2,x45=0;
X51=0,x52=0,x53=-3/2,x54=0,x55=0;
for the calculation of the G component, the following parameters are selected:
x11=0,x12=0,x13=-1,x14=0,x15=0;
X21=0,x22=0,x23=2,x24=0,x25=0;
X31=-1,x32=2,x33=4,x34=2,x35=-1;
X41=0,x42=0,x43=2,x44=0,x45=0;
X51=0,x52=0,x53=-1,x54=0,x55=0。
8. the color image demosaicing system based on the FPGA is characterized in that:
the method comprises the following units:
a CFA array arrangement determining unit for determining the arrangement of the CFA array;
the RGB determining unit is used for determining whether the color channel of the pixel point in the bayer image is an R channel, a G channel or a B channel;
the neighborhood pixel information acquisition unit is used for acquiring pixel point information in a neighborhood around a pixel point to be reconstructed;
and the color component reconstruction unit is used for reconstructing a GB color component of the R channel, reconstructing an RB color component of the G channel and reconstructing an RG color component of the B channel.
9. A computer-readable storage medium characterized by: the computer-readable storage medium has stored therein a computer program which, when executed by a processor, implements the FPGA-based color image demosaicing method of any of claims 1-7.
CN202010391687.1A 2020-05-11 2020-05-11 Color image demosaicing method, system and storage medium based on FPGA Active CN111681191B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010391687.1A CN111681191B (en) 2020-05-11 2020-05-11 Color image demosaicing method, system and storage medium based on FPGA

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010391687.1A CN111681191B (en) 2020-05-11 2020-05-11 Color image demosaicing method, system and storage medium based on FPGA

Publications (2)

Publication Number Publication Date
CN111681191A true CN111681191A (en) 2020-09-18
CN111681191B CN111681191B (en) 2024-02-27

Family

ID=72433374

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010391687.1A Active CN111681191B (en) 2020-05-11 2020-05-11 Color image demosaicing method, system and storage medium based on FPGA

Country Status (1)

Country Link
CN (1) CN111681191B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116342394A (en) * 2023-05-30 2023-06-27 之江实验室 Real-time image demosaicing method, device and medium based on FPGA

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060104505A1 (en) * 2004-11-15 2006-05-18 Chih-Lung Chen Demosaicking method and apparatus for color filter array interpolation in digital image acquisition systems
US20070177033A1 (en) * 2006-01-30 2007-08-02 Microsoft Corporation Bayesian demosaicing using a two-color image
CN102663703A (en) * 2012-04-20 2012-09-12 西安电子科技大学 Treelet-based Bayer type CFA image denoising method
CN109636726A (en) * 2018-12-14 2019-04-16 中国科学院长春光学精密机械与物理研究所 Regular hexagon pixel color filter coefficients method and device
CN110009590A (en) * 2019-04-12 2019-07-12 北京理工大学 A kind of high-quality colour image demosaicing methods based on convolutional neural networks

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060104505A1 (en) * 2004-11-15 2006-05-18 Chih-Lung Chen Demosaicking method and apparatus for color filter array interpolation in digital image acquisition systems
US20070177033A1 (en) * 2006-01-30 2007-08-02 Microsoft Corporation Bayesian demosaicing using a two-color image
CN102663703A (en) * 2012-04-20 2012-09-12 西安电子科技大学 Treelet-based Bayer type CFA image denoising method
CN109636726A (en) * 2018-12-14 2019-04-16 中国科学院长春光学精密机械与物理研究所 Regular hexagon pixel color filter coefficients method and device
CN110009590A (en) * 2019-04-12 2019-07-12 北京理工大学 A kind of high-quality colour image demosaicing methods based on convolutional neural networks

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
孙辉;柏旭光;孙丽娜;李志强;: "Bayer图像色彩还原线性插值方法" *
尹勇;胡磊;: "一种改进的Bayer图像彩色恢复差值算法" *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116342394A (en) * 2023-05-30 2023-06-27 之江实验室 Real-time image demosaicing method, device and medium based on FPGA
CN116342394B (en) * 2023-05-30 2023-08-18 之江实验室 Real-time image demosaicing method, device and medium based on FPGA

Also Published As

Publication number Publication date
CN111681191B (en) 2024-02-27

Similar Documents

Publication Publication Date Title
CN111080528B (en) Image super-resolution and model training method and device, electronic equipment and medium
CN109658337B (en) FPGA implementation method for real-time electronic despinning of images
US20210327033A1 (en) Video processing method and apparatus, and computer storage medium
CN107680043B (en) Single image super-resolution output method based on graph model
WO2022267494A1 (en) Image data generation method and apparatus
Licciardo et al. Stream processor for real-time inverse Tone Mapping of Full-HD images
WO2023160426A1 (en) Video frame interpolation method and apparatus, training method and apparatus, and electronic device
CN111681191B (en) Color image demosaicing method, system and storage medium based on FPGA
Cadenas et al. Parallel pipelined array architectures for real-time histogram computation in consumer devices
CN102611842A (en) Image processing devices and image processing methods
KR101140953B1 (en) Method and apparatus for correcting distorted image
CN108024074B (en) Miniaturized infrared imaging method based on SOPC
Toczek et al. Scene-based non-uniformity correction: from algorithm to implementation on a smart camera
WO2024041027A1 (en) Panoramic-image processing method, and computer device and storage medium
CN116342394B (en) Real-time image demosaicing method, device and medium based on FPGA
WO2023082685A1 (en) Video enhancement method and apparatus, and computer device and storage medium
CN106558021A (en) Video enhancement method based on super-resolution technique
Blasinski et al. FPGA architecture for real-time barrel distortion correction of colour images
WO2021233232A1 (en) Image processing method and image processing model training method and device
CN104776919B (en) Infrared focal plane array ribbon Nonuniformity Correction system and method based on FPGA
US11270412B2 (en) Image signal processor, method, and system for environmental mapping
TW200818908A (en) Image processing apparatus and method
CN110895790B (en) Scene image super-resolution method based on posterior degradation information estimation
Fuentes et al. FPGA implementation of the bilinear interpolation algorithm for image demosaicking
Azgin et al. A high performance alternating projections image demosaicing hardware

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Country or region after: China

Address after: Room 707-710, 7th floor, building B3, innovation industrial park, No. 800, Wangjiang West Road, hi tech Zone, Hefei City, Anhui Province

Applicant after: Anhui Baicheng Huitong Technology Co.,Ltd.

Address before: Room 707-710, 7th floor, building B3, innovation industrial park, No. 800, Wangjiang West Road, hi tech Zone, Hefei City, Anhui Province

Applicant before: ANHUI BAI CHENG HUI TONG TECHNOLOGY CO.,LTD.

Country or region before: China

CB02 Change of applicant information
GR01 Patent grant
GR01 Patent grant