CN108810506A - A kind of Penetrating Fog enhancing image processing method and system based on FPGA - Google Patents
A kind of Penetrating Fog enhancing image processing method and system based on FPGA Download PDFInfo
- Publication number
- CN108810506A CN108810506A CN201810608569.4A CN201810608569A CN108810506A CN 108810506 A CN108810506 A CN 108810506A CN 201810608569 A CN201810608569 A CN 201810608569A CN 108810506 A CN108810506 A CN 108810506A
- Authority
- CN
- China
- Prior art keywords
- image
- fpga
- penetrating fog
- processing
- image data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000002708 enhancing effect Effects 0.000 title claims abstract description 54
- 230000000149 penetrating effect Effects 0.000 title claims abstract description 49
- 238000003672 processing method Methods 0.000 title claims abstract description 10
- 238000012545 processing Methods 0.000 claims abstract description 75
- 238000012937 correction Methods 0.000 claims abstract description 18
- 238000000034 method Methods 0.000 claims description 26
- 230000008569 process Effects 0.000 claims description 11
- 238000012512 characterization method Methods 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 3
- 238000013139 quantization Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 9
- 238000012806 monitoring device Methods 0.000 abstract description 2
- 239000013589 supplement Substances 0.000 abstract description 2
- 238000004422 calculation algorithm Methods 0.000 description 40
- 230000006870 function Effects 0.000 description 14
- 238000001914 filtration Methods 0.000 description 9
- 238000013461 design Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 6
- 238000012544 monitoring process Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 238000005457 optimization Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000007423 decrease Effects 0.000 description 3
- 238000009795 derivation Methods 0.000 description 3
- 230000002093 peripheral effect Effects 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 241001269238 Data Species 0.000 description 2
- 230000000052 comparative effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 239000000428 dust Substances 0.000 description 2
- 238000003708 edge detection Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000009897 systematic effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000000844 transformation Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 235000019504 cigarettes Nutrition 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000003595 mist Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000003739 neck Anatomy 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/64—Circuits for processing colour signals
- H04N9/646—Circuits for processing colour signals for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/44—Receiver circuitry for the reception of television signals according to analogue transmission standards
- H04N5/445—Receiver circuitry for the reception of television signals according to analogue transmission standards for displaying additional information
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/64—Circuits for processing colour signals
- H04N9/73—Colour balance circuits, e.g. white balance circuits or colour temperature control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/77—Circuits for processing the brightness signal and the chrominance signal relative to each other, e.g. adjusting the phase of the brightness signal relative to the colour signal, correcting differential gain or differential phase
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The present invention provides a kind of, and the Penetrating Fog based on FPGA enhances image processing method and system, wherein, image processing system includes fpga chip and ARM chips, fpga chip is connect with ARM chips, it is provided with image data acquiring interface on fpga chip, and color cast correction processing, Penetrating Fog enhancing processing, low contrast enhancing processing and/or details enhancing are carried out to the image data of acquisition and handled；It is also set up on fpga chip for by the output interface of treated image data output.The present invention has carried out multiple processing to the image of acquisition, picture quality is set to be greatly improved, the image data acquired in the case of bad weather is unaffected, finally obtained fog free images effect is relatively good, to various monitoring devices in the prior art can play enhancing, supplement effect.
Description
Technical field
The invention belongs to technical field of image processing, more particularly to a kind of Penetrating Fog based on FPGA enhances image processing method
And system.
Background technology
Video Supervision Technique passes through long-run development, comes into the application stage of comparative maturity now.But in certain necks
Domain cannot still meet specific observation requirements.For example, in the environment such as cigarette, mist, haze, aqueous vapor, rain, snow, dust, dark, underwater
Under, traditional monitoring equipment is difficult to play a role or even helpless.The epoch that extensive style increases in video monitoring market, this
Problem can temporarily be ignored.But as manufacturer and user are more and more deeper to the understanding of scientific equipment efficiency, it is desirable that also just corresponding
When the river rises the boat goes up.
Under normal circumstances, existing video monitoring system pass through image transmission and conversion, such as imaging, amplitude, scanning,
Transmission and display lamp, often cause the decline of picture quality.In photography since illumination condition is insufficient or excessively, can make image
Darker or lighter；Distortion, relative motion, atmospheric turbulance of optical system etc. can all make image fuzzy, can also draw in transmission process
Enter various types of noises.In short, the image of input is in visual effect and identification convenience etc., there may be problems.
To solve the above-mentioned problems, it is proposed that Publication No. " CN105023256A ", it is entitled " a kind of image defogging method and system "
Chinese patent, the image defogging method of the patent needs to obtain the global atmosphere light and transmissivity in each channel of image,
The fog free images that each channel is recovered according to the global atmosphere light and transmissivity in each channel, to obtain fogless figure
Picture.But the method for the patent is more complicated, also not good enough to the effect of the fogless processing of image, obtained picture quality is not high.
Invention content
The purpose of the present invention is to provide a kind of, and the Penetrating Fog based on FPGA enhances image processing method and system, for solving
The bad problem of image defogging treatment effect in the prior art.
To achieve the goals above, the present invention provides a kind of, and the Penetrating Fog based on FPGA enhances image processing system, including
Fpga chip and ARM chips, the fpga chip are connect with the ARM chips, and image data is provided on the fpga chip
Acquisition interface, and to the image data of acquisition carry out color cast correction processing, Penetrating Fog enhancing processing, low contrast enhancing processing and/
Or details enhancing processing；It is also set up on the fpga chip for by the output interface of treated image data output.
The problem of must not flexibly being controlled FPGA for solution, is provided with for being communicated with host computer on the ARM chips
The network interface of connection, the ARM chips be used for issued according to host computer instruction control FPGA to the image data of acquisition at
Reason.
Further, further include gradient statistical module, the gradient statistical module is used to carry out the image data of acquisition
Gradient statistical disposition, and the image data after gradient counts is sent to host computer by the ARM chips.
The first driving interface and the first debugging interface are provided on the fpga chip.
The second driving interface and the second debugging interface are provided on the ARM chips.
Image is monitored and is observed for convenience, further includes Electronic magnification module and character adding module, acquisition
For image data after details enhancing processing, the overlap-add procedure of zoom processing and character adding module by Electronic magnification module is defeated
Go out.
The present invention also provides a kind of, and the Penetrating Fog based on FPGA enhances image processing method, includes the following steps：
Image data is acquired, color cast correction processing, Penetrating Fog enhancing processing, low contrast enhancing are carried out to described image data
Processing and/or details enhancing processing, and image data exports by treated.
Further, gradient statistical disposition has been carried out to the image data of acquisition.
Further, the process of the color cast correction processing is：Obtain the number in tri- channels R, G, B of the image of acquisition
According to the foundation characterization partially green intensity map of image chooses the pixel of the minimum containing green components in image as image entirety colour cast journey
The measurement of degree refers to quantization means of the corresponding intensity map values as image color cast according to set proportion in the intensity map.
Image is monitored and is observed for convenience, the image data of acquisition is after details enhancing processing, by electronics
Zoom and character adding processing output.
The beneficial effects of the invention are as follows：
The present invention includes fpga chip and ARM chips, and the fpga chip is connect with the ARM chips, the FPGA cores
On piece is provided with image data acquiring interface, and handles the processing of the image data of acquisition progress color cast correction, Penetrating Fog enhancing, is low
Contrast enhancement processing and/or details enhancing processing；It is also set up on the fpga chip for by treated, image data to be defeated
The output interface gone out.The present invention has carried out multiple processing to the image of acquisition, so that picture quality is greatly improved, in day
The image data acquired in the case that gas is severe is unaffected, and finally obtained fog free images effect is relatively good, to the prior art
In various monitoring devices can play enhancing, supplement effect.
Description of the drawings
Fig. 1, which is the electronics Penetrating Fog based on FPGA of the present invention, enhances the structural schematic diagram of image processing system；
Fig. 2 is the image optimization algorithm flow schematic diagram of the present invention；
Fig. 3-1 is three sections of linear transformation schematic diagrames of Penetrating Fog algorithm；
Fig. 3-2 is the schematic diagram that luminance dynamic range is calculated using statistics with histogram method of Penetrating Fog algorithm；
Fig. 4-1 is the newer schematic diagram of row hisgram that low contrast enhances the algorithm center rightmost side；
Fig. 4-2 is in low contrast enhancing algorithm and the newer schematic diagram of histogram；
Fig. 5 is detail enhancement algorithms schematic diagram；
Fig. 6 is the schematic diagram that the bilinearity difference of Electronic magnification algorithm is handled.
Specific implementation mode
The specific implementation mode of the present invention is further described below in conjunction with the accompanying drawings：
Penetrating Fog technology includes that (for example translucent cover is slight dirty to dust, aqueous vapor, tiny barrier for practical application
Dirty and rainwater etc.) penetrate.Under these rugged environments, picture quality can decline to a great extent or even can not acquire monitoring objective
Image, it is therefore desirable to which post-processing and optimization are carried out to image using Penetrating Fog technology.Image optimization processing mainly uses various
Algorithm re-calibrates fuzzy image, reproduces image scene.It is with the development of chip technology, the image on computer is excellent
Change system makes single operation program, is cured on FPGA, and here it is so-called digital Penetrating Fog technologies, due to simple number
Penetrating Fog will not change color, therefore, also be colored Penetrating Fog.
The electronics Penetrating Fog enhancing image processing system based on FPGA of the present embodiment includes fpga chip and ARM cores
Piece, fpga chip are connect with ARM chips, image data acquiring interface are provided on fpga chip, and to the image data of acquisition
Carry out color cast correction processing, Penetrating Fog enhancing processing, low contrast enhancing processing and/or details enhancing processing；On fpga chip also
It is arranged for by the output interface of treated image data output.
The network interface for being communicated to connect with host computer is provided on ARM chips, what ARM chips were used to be issued according to host computer
Instruction control FPGA handles the image data of acquisition.The first driving interface and the first tune are provided on above-mentioned fpga chip
It tries mouth, the second driving interface and the second debugging interface is provided on ARM chips.
The Penetrating Fog based on FPGA of the present embodiment enhances image processing system, further includes gradient statistical module, the gradient
Statistical module is used to carry out gradient statistical disposition to the image data of acquisition, and the image data after gradient counts is passed through
The ARM chips are sent to host computer；In order to show the present embodiment image processing system diversity function, further include electronics
Zoom module and character adding module, the image data of acquisition is after details enhancing processing, by the zoom of Electronic magnification module
The output of the overlap-add procedure of processing and character adding module.
Using the above-mentioned electronics Penetrating Fog enhancing image processing system based on FPGA to the method for image procossing, as shown in Fig. 2,
Mainly color cast correction processing, Penetrating Fog enhancing processing, low contrast enhancing processing and/or details is carried out to the image data of acquisition to increase
Strength reason etc. repeatedly processing, so as to get picture quality it is very high, do not influence the analysis of the image to acquisition in inclement weather.
Specifically, as shown in Figure 1, image processing system includes hardware components and software section, hardware components mainly wrap
Include interface, processing core, peripheral hardware, power module.Wherein, interface includes 1 road HD-SDI inputs, 2 road HD-SDI outputs, 2 road PAL
Output, 1 road RS422 communications, power supply and the first debugging interface, the first debugging interface include RS422 debugging interfaces and JTAG debugging
Interface, ARM chips share JTAG debugging interfaces with fpga chip, so, the second debugging interface of ARM chips is that JTAG debugging connects
Mouthful.Processing core considers resource and power consumption, selects the Cycione V of altera corp using in the fpga chip for embedding ARM
SX Series FPGAs.Peripheral hardware includes mainly external the second driving interface, that is, DDR3 driving interfaces of ARM, 1 tunnel TF card, 1 road Nor
The first Flash, FPGA external driving interface, the i.e. conversion chip of DDR3 driving interfaces and several interfaces.Power module connects
5V inputs are received, voltage needed for each chip on plate, such as 3.3V, 2.5V, 1.8V, 1.1V are converted to.
Software section includes ARM softwares and fpga logic, ARM loading Linux operating systems, and realizes that serial ports, DDR3 drive
The basic drivers such as mobile interface, flash boot, application layer are mainly responsible for communication command parsing, Comprehensive Control and character mask figure
It draws.
Fpga logic realizes main image processing function, including the enhancing of gradient statistics, color cast correction, Penetrating Fog, low comparison
Enhancing, details enhancing, Electronic magnification, character adding, image scaling are spent, interface driver, operating mode configuration etc. are additionally included
Interface frame related content.
Image processing system inputs 1080p coloured image numbers by the external 1080p25/30 high definition cameras of HD-SDI interfaces
According to, FPGA is entered, data are handled after entering FPGA by color cast correction, Penetrating Fog enhancing, low contrast enhancing, details enhancing,
It realizes main Processing Algorithm, completes the improvement of picture quality, improve observability；Subsequently through Electronic magnification and character adding
(the 2 same content of road SDI the output phases) is exported by HD-SDI interfaces；Simultaneously to meet pal mode, pass through image after Electronic magnification
Scaling by 1920*1080 image downs be 720*576, then overlaying character from PAL interfaces export.Herein, enhance by details
Image data that treated is handled by Electronic magnification using image scaling, is because Electronic magnification processing can be maintained at defeated
Go out resolution ratio it is constant in the case of enlarged drawing, corresponding visual field can become smaller, if treated that data are direct by details enhancing
Image scaling processing is carried out, visual field does not change.
To realize that the intelligibility evaluation of image, the image data for inputting FPGA are carried out at the same time region gradient statistical disposition, and
Statistical value is fed back into the parts ARM, host computer is transmitted to by serial ports.
ARM on the one hand by serial ports receive host computer transmission data and order, including the related data of overlaying character and
Operating mode control command, and the information such as self-test state, clarity statistical value are fed back to host computer；On the other hand according to host computer
Operating mode Comprehensive Control is realized in order, and is drawn according to obtained system information more fresh character mask figure, by character mask figure
It is supplied to the character adding module of FPGA.
Image processing system configures related peripherals and interface circuit using the FPGA for embedding ARM as processing core, realizes image
Real-time processing.ARM softwares are mainly responsible for the content of 3 parts：
(1) serial communication Comprehensive Control：The scheme control and systematic parameter that host computer is sent, root are received by 422 serial ports
Instruction is sent out to FPGA portion according to mode control commands content, controls the switch of FPGA operating modes；Systematic parameter is carried simultaneously
Character mask figure drafting module is supplied, as drafting input information；In addition the functions such as System self-test, comprehensive task scheduling are completed.
(2) character mask figure is drawn：Library is drawn based on SKIA or other characters, into the drafting of line character mask figure, and is delayed
It deposits to DDR3 external ARM, use is subsequently superimposed for FPGA portion.
(3) interface driver and system boot：System electrification configuration, including the parts ARM and FPGA portion are completed, ARM is completed
Partial interface driver, including DDR3, serial ports, AXI buses etc..
Fpga logic includes mainly 3 partial contents：
(1) interface driver part：1) HD-SDI decodings and HD-SDI codings are completed, the part is by calling altera to provide
IP complete；2) DDR3 buffer controls, it is main to complete input and output image buffer storage, on the basis of official provides IP, carry out mostly logical
Road Read-write Catrol encapsulation；3) SAF7129 drives, and is related to SAF7129I2C configurations, the conversion of PAL system data protocol；
(2) operating mode control section：1) the AXI bus communications with ARM are completed, the control command that ARM is transmitted is received, returns
Return gradient statistical information and other work state informations；2) control command based on ARM, each algorithm in placement algorithm function module
The operating mode of module.
(3) algorithm function module：Completion includes gradient statistics, color cast correction, Penetrating Fog enhancing, low contrast enhancing, details
The processing of all algorithm function modules, each intermodule interface uniformly make including enhancing, Electronic magnification, image scaling, character adding
With Dval, DataRGB [23:0], CLK_pixel realizes that full streamlined processing, each pixel clock export 1 processes pixel knot
Fruit.
Each image optimization algorithm is described in detail separately below：
1) color cast correction algorithm
Input：Original video stream sequence, video image resolution ratio are 1920 × 1080 pixels, locating depth 24bit.
Output：Sequence of video images after white balance, video image resolution ratio are 1920 × 1080 pixels, and locating depth is
24bit。
Implementation process designs：
Foggy image is since the penetration capacity of different-waveband light in natural light is different, and there are colour casts to ask when being often imaged
Topic, after the enhancing of image Penetrating Fog, colour cast can also be amplified enhancing and become abnormal apparent, therefore this algorithm is directed to sample video
The partially green problem of the tone of appearance proposes a kind of method of color cast correction processing before Penetrating Fog enhancing.
R, G, B triple channel pixel value of known input picture I (x, y) are respectively r (x, y), g (x, y) and b (x, y), then table
The partially green intensity map G (x, y) of image is levied, can be calculated by the following formula：
G (x, y)=max (0, g (x, y)-max (r (x, y), b (x, y)))
Wherein, max () operator indicates to calculate the maximum value of two numbers, and the value range of G (x, y) is [0,255]；G(x,
Y) physical significance characterization is how much green contained by each pixel of image.In view of the characteristic of colour cast image is exhausted big portion
Partial image all includes green components, in image procossing, can calculate the pixel comprising green components minimum in full figure and make
For the measurement of image entirety colour cast degree, it is contemplated that the error that single-point is chosen, choose in real process in intensity map G (x, y) from
It is small to the strong map values G chosen greatly corresponding to a certain proportion of point40The quantization means of (x, y) as image color cast.After color cast correction
The channels G pixel value g'(x, y) can be calculated by the following formula：
G ' (x, y)=max (0, g (x, y)-G40(x,y))
The color cast correction algorithm of the present embodiment is small, adaptable etc. with calculation amount relative to traditional white balance algorithm
Feature, and algorithm comparative example value RThresSelection and insensitive, R during realizationThresValue range chooses [0.3,0.4]
It is good.
2) Penetrating Fog algorithm
Input：Sequence of video images after color cast correction, video image resolution ratio are 1920 × 1080 pixels, and locating depth is
24bit。
Output：Sequence of video images after Penetrating Fog, video image resolution ratio are 1920 × 1080 pixels, locating depth 24bit.
Implementation process designs：
Penetrating Fog algorithm realizes that Penetrating Fog enhances function using the algorithm that color histogram stretches.The straight of foggy image is calculated first
Side's figure distribution, it is contemplated that foggy image is whole since the histogram distribution quantity in low-light level part is seldom, causes vision
The low problem of upper contrast.This algorithm considers the statistical distribution characteristic of image pixel histogram, calculates the height that image pixel stretches
Then low-light level thresholding increases contrast by the method that R, G, B three channel histogram stretch, realizes Penetrating Fog effect.
R, G, B triple channel pixel value of known input picture I (x, y) are respectively r (x, y), g (x, y) and b (x, y), then grey
Image Y (x, y) is spent to calculate using the psychology formula of gray count：
Y (x, y)=0.30*r (x, y)+0.59*g (x, y)+0.11*b (x, y)
When image enhancement processing, in order to protrude interested target or gray scale interval, piecewise linear transform may be used,
Entire gray scale interval is divided into several gray scale intervals, and stretching will enhance the corresponding gray scale interval of target, opposite to inhibit to lose interest in
Gray level, to achieve the purpose that enhancing.Common piecewise linear transform is three sections of linear transformations, as shown in figure 3-1, number
Learning expression formula is：
M is image maximum brightness in formula, by adjusting the position of broken line inflection point and the slope of segmented linear, i.e. control parameter
A, b, c, the value of d is, it can be achieved that extension or compression to any gray scale interval.
Penetrating Fog algorithm passes through low-light level ratio thresholding in setting image using the histogram of statistics gray level image Y (x, y)
Rlow and high brightness ratio thresholding Rhigh calculates the dynamic range [LowThres, HighThres] of brightness of image, it is final really
The parameter of fixed three sections of linear transformations, principle is as shown in figure 3-2.In real process, Rlow takes 0.005, Rhigh that 0.01, a is taken to take
LowThres, b take HighThres, c that LowThres/3, d is taken to take (HighThres+255)/2.
3) low contrast enhances algorithm
Input：Sequence of video images after Penetrating Fog, video image resolution ratio are 1920 × 1080 pixels, locating depth 24bit.
Output：The enhanced sequence of video images of low contrast, video image resolution ratio are 1920 × 1080 pixels, locating depth
For 24bit.
Implementation process designs：Image defogging algorithm often makes the reduction of image overall brightness, and original brightness is relatively low
Region due to further decreasing for brightness and so that original image detail produces missing to a certain extent, this is just needed
Luminance compensation is carried out using the module of low contrast enhancing.
The core of low contrast algorithm design is the estimation to low contrast regions, which uses one for gray level image
The mode of the medium filtering of dimensioning, the average brightness of extraction pixel region is as the pixel while ignoring image detail
The reference weights of enhancing, i.e. brightness derivation graph；Then a non-linear transform function is selected (to choose index letter in practical programs
Number), by brightness derivation graph respectively to image R, G, B triple channel carry out brightness stretching in the way of realize contrast enhance.It is known
The RGB triple channel pixel values of input picture I (x, y) are respectively r (x, y), g (x, y) and b (x, y), the gray level image Y of this module
(x, y) is calculated by the following formula：
Y (x, y)=max (r (x, y), g (x, y), b (x, y))
Gray level image medium filtering uses Fast Median Filtering algorithm, firstly, for each row image, all safeguards one for it
A histogram (for 8 bit images, which has 256 elements), in entire processing procedure, these histogram datas are all
It must be maintained.The information of 2r+1 vertically adjacent pixels is had accumulated per row hisgram, and (wherein r is medium filtering half
Diameter), when initial, this 2r+1 pixel is respectively centered on each pixel of the first row.The histogram of core passes through tired
2r+1 adjacent row hisgram data acquisitions of product.During entire filtering, these histogram datas are in two steps
Keep newest with the constant time.As shown in Fig. 4-1 and 4-2, the case where considering to move right a pixel from some pixel.It is right
In current line, the row hisgram of the core rightmost side firstly the need of update, and at this time the data in the row hisgram of the row or more than
It is calculated centered on that pixel of a line corresponding position.Therefore it needs to subtract the corresponding histogram of a most upper pixel and then add
The histogram information of a pixel below.Row hisgram data are exactly reduced a line by the effect done so.This step is apparent
It is the operation of 0 (1), only once addition and a subtraction, and it is unrelated in radius r.Second step updates core histogram, is 2r+1
The sum of a row hisgram.This is the row hisgram data by subtracting the leftmost side, then adds that handled by the first step
What the row hisgram data of row obtained.As previously mentioned, addition, subtraction and calculate histogram intermediate value it is time-consuming be all some according to
Rely in the calculating of image locating depth, and it is unrelated with filter radius.
Using brightness derivation graph, if image R, G, B triple channel gray value is Yr/g/b(x, y), the threeway after gray scale stretching
Road gray value is Y'r/g/b(x, y), the nonlinear function that gray scale stretching uses for：
Wherein, Meduim (x, y) is gray scale point, Yr/g/bThe pixel value of (x, y) corresponding medium filtering image, YminFor ash
Spend image pixel minimum value, YavgFor gray level image pixel average.
4) detail enhancement algorithms
Input：The enhanced sequence of video images of low contrast, video image resolution ratio are 1920 × 1080 pixels, locating depth
For 24bit.
Output：The enhanced sequence of video images of details, video image resolution ratio are 1920 × 1080 pixels, and locating depth is
24bit。
Implementation process designs：Detail enhancement algorithms are an optional module in embodiment, aim at enhancing image
Detailed information.The important flow of algorithm using classical Unsharp Mask (USM) sharpening algorithm, concrete principle as shown in figure 5, with
Specific formula, which is expressed, is：
Y (n, m)=x (n, m)+λ z (n, m)
Wherein, x (n, m) is input picture, and y (n, m) is output image, and λ is for controlling enhancing effect scaling
The factor, and z (n, m) is correction signal, and high-pass filtering acquisition is carried out to x generally by input picture, in this algorithm：
Z (n, m)=x (n, m)-g (n, m)
Wherein, g (n, m) is the gaussian filtering result to x (n, m).
Using detail enhancement algorithms to image procossing during, tri- channels R, G, B of original image are carried out first
Then low-pass filtering utilizes the high frequency section in the difference of original image and low-pass filtered image extraction image, finally will be high
Frequency image weighted superposition realizes image detail enhancing.
5) Electronic magnification algorithm
Input：The enhanced sequence of video images of details, video image resolution ratio are 1920 × 1080 pixels, and locating depth is
24bit。
Output：Sequence of video images after 2x or 4x Electronic magnifications, video image resolution ratio are 1920 × 1080 pixels, position
Depth is 24bit.
Implementation process designs：The purpose of Electronic magnification is that picture centre region is amplified display, with meet monitoring and
The needs of observation are divided into two kinds of functional requirements of 2 × Electronic magnification and 4 × Electronic magnification.
The algorithm design synthesis considers efficiency and two aspect factor of effect, and Electronic magnification is realized using bilinear interpolation algorithm
Function.Mathematically, bilinear interpolation is the linear interpolation extension of the interpolating function there are two variable, and core concept is two
A direction carries out once linear interpolation respectively, as shown in Figure 6.
If expecting unknown function f in the value of point P=(x, y), and known function f is in Q11=(x1, y1), Q12=(x1,
y2), Q21=(x2, y1) and Q22=(x2, y2) four points value, calculating solution procedure is：
Linear interpolation is carried out in the directions x first, is obtained：
Then linear interpolation is carried out in the directions y, obtained：
Thus obtain the f (expression formulas of x, y：
If selection one coordinate system make f four known point coordinates be respectively (0,0), (0,1), (1,0) and (1,
1), then interpolation formula can abbreviation be：
F (x, y) ≈ f (0,0) (1-x) (1-y)+f (1,0) x (1-y)+f (0,1) (1-x) y+f (1,1) xy.
Or it is expressed as with matrix operation：
Unlike this interpolation method, the result of this interpolation method is not usually linear, is expressed as：
b1+b2x+b3y+b4xy.
Wherein, the number of constant both corresponds to the data of given f and counts out：
b1=f (0,0)
b2=f (1,0)-f (0,0)
b3=f (0,1)-f (0,0)
b4=f (1,1)-f (1,0)-f (0,1)+f (0,0)
The result of linear interpolation is unrelated with the sequence of interpolation.The interpolation in the directions y is carried out first, then carries out inserting for the directions x
Value, it is obtained the result is that the same.
6) gradient statistic algorithm
Input：Original video image sequence, video image resolution ratio are 192 × 108 pixels, locating depth 24bit.
Output：Gradient reference value.
Implementation process designs：Gradient statistic algorithm is mainly to pass through the wheel of detection image with the principle of contrast focusing
Realize auto-focusing in wide edge.The contour edge of image is more clear, then its brightness step is bigger, in other words edge scape
Contrast between object and background is bigger.Conversely, image out of focus, contour edge is smudgy, brightness step or contrast
Decline；Out of focus remoter, contrast is lower.Using this principle, when the contrast phase absolute value of the difference minimum of output, illustrate to focus
It completes.
This algorithm uses metric form of the Sobel gradient operators as amount of variation, calculates picture centre region (192 × 108
Pixel) Grad, when Grad reaches maximum, as focusing complete.Sobel operators are mainly used as edge detection, in technology
On, it is a discreteness difference operator, is used for the approximation of the gray scale of operation brightness of image function.Make in any point of image
With this operator, it will generate corresponding gray scale vector or its law vector.Sobel warp factors are：
The operator includes the matrix of two groups of 3x3, respectively transverse direction and longitudinal direction, it and image are made planar convolution, you can point
The brightness difference approximation of transverse direction and longitudinal direction is not obtained.If representing original image with A, Gx and Gy are respectively represented through lateral and vertical
To the gray value of image of edge detection.
The transverse direction and longitudinal direction gray value of each pixel of image is combined by following formula, to calculate the big of the gray scale
It is small：
Gradient statistic algorithm is Grad to improve efficiency using the approximation not extracted square root：
| G |=| Gx|+|Gy|
7) local clarity statistical module
The module is counted for the clarity about each pixel in image data, for judging Penetrating Fog enhancing
Effect completes record by ARM cpu.
Specific embodiment is presented above, but the present invention is not limited to embodiment described above.The present invention
Basic ideas be above-mentioned basic scheme, for those of ordinary skill in the art, introduction according to the present invention is designed each
The model of kind deformation, formula, parameter do not need to spend creative work.The case where not departing from the principle and spirit of the invention
Under to embodiment carry out variation, modification, replacement and deformation still fall in protection scope of the present invention.
Claims (10)
1. a kind of Penetrating Fog based on FPGA enhances image processing system, which is characterized in that including fpga chip and ARM chips, institute
It states fpga chip to connect with the ARM chips, image data acquiring interface is provided on the fpga chip, and to the figure of acquisition
As data carry out color cast correction processing, Penetrating Fog enhancing processing, low contrast enhancing processing and/or details enhancing processing；It is described
It is also set up on fpga chip for by the output interface of treated image data output.
2. the Penetrating Fog according to claim 1 based on FPGA enhances image processing system, which is characterized in that the ARM cores
On piece is provided with the network interface for being communicated to connect with host computer, and the ARM chips are used to be controlled according to the instruction that host computer issues
FPGA handles the image data of acquisition.
3. the Penetrating Fog according to claim 2 based on FPGA enhances image processing system, which is characterized in that further include gradient
Statistical module, the gradient statistical module is used to carry out gradient statistical disposition to the image data of acquisition, and will pass through gradient and unite
Image data after meter is sent to host computer by the ARM chips.
4. the Penetrating Fog according to claim 1 based on FPGA enhances image processing system, which is characterized in that the FPGA cores
On piece is provided with the first driving interface and the first debugging interface.
5. the Penetrating Fog according to claim 4 based on FPGA enhances image processing system, which is characterized in that the ARM cores
On piece is provided with the second driving interface and the second debugging interface.
6. the Penetrating Fog according to claim 1 based on FPGA enhances image processing system, which is characterized in that further include electronics
Zoom module and character adding module, the image data of acquisition is after details enhancing processing, by the zoom of Electronic magnification module
The output of the overlap-add procedure of processing and character adding module.
7. a kind of Penetrating Fog based on FPGA enhances image processing method, which is characterized in that include the following steps：
Image data is acquired, color cast correction processing, Penetrating Fog enhancing processing, low contrast enhancing processing are carried out to described image data
And/or details enhancing processing, and image data exports by treated.
8. the Penetrating Fog according to claim 7 based on FPGA enhances image processing method, which is characterized in that the figure of acquisition
As data have carried out gradient statistical disposition.
9. the Penetrating Fog according to claim 8 based on FPGA enhances image processing method, which is characterized in that the colour cast school
The process just handled is：The data in tri- channels R, G, B of the image of acquisition are obtained, the characterization partially green intensity map of image, choosing are established
Measurement of the pixel of the minimum containing green components in image as image entirety colour cast degree is taken, according to setting in the intensity map
Fixed ratio refers to quantization means of the corresponding intensity map values as image color cast.
10. the Penetrating Fog according to claim 7 based on FPGA enhances image processing method, which is characterized in that the figure of acquisition
Picture data are after details enhancing processing, by Electronic magnification and character adding processing output.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810608569.4A CN108810506B (en) | 2018-06-13 | 2018-06-13 | Fog-penetrating enhanced image processing method and system based on FPGA |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810608569.4A CN108810506B (en) | 2018-06-13 | 2018-06-13 | Fog-penetrating enhanced image processing method and system based on FPGA |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108810506A true CN108810506A (en) | 2018-11-13 |
| CN108810506B CN108810506B (en) | 2021-09-07 |
Family
ID=64085627
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810608569.4A Active CN108810506B (en) | 2018-06-13 | 2018-06-13 | Fog-penetrating enhanced image processing method and system based on FPGA |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108810506B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109544483A (en) * | 2018-12-26 | 2019-03-29 | 深圳朗田亩半导体科技有限公司 | Video image brightness, contrast enhancement process and device |
| CN111010497A (en) * | 2019-12-26 | 2020-04-14 | 苏州羿景睿图信息科技有限公司 | FPGA-based CLAHE fog penetration enhancement parallel operation method |
| CN112581509A (en) * | 2020-12-25 | 2021-03-30 | 北京环境特性研究所 | SOPC-based unmanned aerial vehicle-mounted ground target real-time tracking system and method |
| CN113970896A (en) * | 2021-10-25 | 2022-01-25 | 中国船舶重工集团公司第七一一研究所 | Control device based on FPGA chip and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202121711U (en) * | 2011-06-09 | 2012-01-18 | 南京觅踪电子科技有限公司 | Fog-penetrating monitoring system based on machine vision |
| CN105023256A (en) * | 2015-08-13 | 2015-11-04 | 丘璇 | Image defogging method and system |
| CN106506974A (en) * | 2016-12-13 | 2017-03-15 | 北京鸿合智能系统股份有限公司 | A kind of automatic photographing method and device |
| CN106998445A (en) * | 2017-03-28 | 2017-08-01 | 深圳市科美集成电路有限公司 | Haze penetrates camera circuit and system |
| CN206620225U (en) * | 2017-03-28 | 2017-11-07 | 深圳市科美集成电路有限公司 | Haze penetrates camera circuit and system |
| CN107895357A (en) * | 2017-12-25 | 2018-04-10 | 北京航空航天大学 | A kind of real-time water surface thick fog scene image Enhancement Method based on FPGA |
-
2018
- 2018-06-13 CN CN201810608569.4A patent/CN108810506B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202121711U (en) * | 2011-06-09 | 2012-01-18 | 南京觅踪电子科技有限公司 | Fog-penetrating monitoring system based on machine vision |
| CN105023256A (en) * | 2015-08-13 | 2015-11-04 | 丘璇 | Image defogging method and system |
| CN106506974A (en) * | 2016-12-13 | 2017-03-15 | 北京鸿合智能系统股份有限公司 | A kind of automatic photographing method and device |
| CN106998445A (en) * | 2017-03-28 | 2017-08-01 | 深圳市科美集成电路有限公司 | Haze penetrates camera circuit and system |
| CN206620225U (en) * | 2017-03-28 | 2017-11-07 | 深圳市科美集成电路有限公司 | Haze penetrates camera circuit and system |
| CN107895357A (en) * | 2017-12-25 | 2018-04-10 | 北京航空航天大学 | A kind of real-time water surface thick fog scene image Enhancement Method based on FPGA |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109544483A (en) * | 2018-12-26 | 2019-03-29 | 深圳朗田亩半导体科技有限公司 | Video image brightness, contrast enhancement process and device |
| CN109544483B (en) * | 2018-12-26 | 2021-09-24 | 深圳朗田亩半导体科技有限公司 | Video image brightness and contrast enhancement method and device |
| US11205252B2 (en) | 2018-12-26 | 2021-12-21 | Shenzhen Lontium Semiconductor Technology Co., Ltd. | Method and device for enhancing brightness and contrast of video image |
| CN111010497A (en) * | 2019-12-26 | 2020-04-14 | 苏州羿景睿图信息科技有限公司 | FPGA-based CLAHE fog penetration enhancement parallel operation method |
| CN112581509A (en) * | 2020-12-25 | 2021-03-30 | 北京环境特性研究所 | SOPC-based unmanned aerial vehicle-mounted ground target real-time tracking system and method |
| CN112581509B (en) * | 2020-12-25 | 2023-08-15 | 北京环境特性研究所 | Unmanned aerial vehicle ground target real-time tracking system and tracking method based on SOPC |
| CN113970896A (en) * | 2021-10-25 | 2022-01-25 | 中国船舶重工集团公司第七一一研究所 | Control device based on FPGA chip and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108810506B (en) | 2021-09-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108810506A (en) | A kind of Penetrating Fog enhancing image processing method and system based on FPGA | |
| Li et al. | Luminance-aware pyramid network for low-light image enhancement | |
| CN110717852B (en) | FPGA-based field video image real-time segmentation system and method | |
| CN107424198A (en) | Image processing method, device, mobile terminal and computer-readable recording medium | |
| CN107451969A (en) | Image processing method, device, mobile terminal and computer-readable recording medium | |
| CN107949850A (en) | Key point in detection image data | |
| CN101887592B (en) | Drawing method for vector graphic map symbols | |
| CN106791623A (en) | A kind of panoramic video joining method and device | |
| CN111079764B (en) | Low-illumination license plate image recognition method and device based on deep learning | |
| CN103646392B (en) | Backlighting detecting and equipment | |
| CN110675334A (en) | Image enhancement method and device | |
| CN103065334A (en) | Color cast detection and correction method and device based on HSV (Hue, Saturation, Value) color space | |
| US11887218B2 (en) | Image optimization method, apparatus, device and storage medium | |
| CN111489322A (en) | Method and device for adding sky filter to static picture | |
| CN107317967A (en) | Image processing method, device, mobile terminal and computer-readable recording medium | |
| CN106454014A (en) | Method and device for improving quality of vehicle image captured in backlighting scene | |
| CN107862672A (en) | The method and device of image defogging | |
| CN109949248A (en) | Modify method, apparatus, equipment and the medium of the color of vehicle in the picture | |
| Fu et al. | Multi-feature-based bilinear CNN for single image dehazing | |
| CN104680518A (en) | Blue screen image matting method based on chroma overflowing processing | |
| Khan et al. | A deep hybrid few shot divide and glow method for ill-light image enhancement | |
| CN110838088A (en) | Multi-frame noise reduction method and device based on deep learning and terminal equipment | |
| CN116453470B (en) | Image display method, device, electronic equipment and computer readable storage medium | |
| Liu et al. | Learning multiscale pipeline gated fusion for underwater image enhancement | |
| CN107392870A (en) | Image processing method, device, mobile terminal and computer-readable recording medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |