CN112785543B - Traffic camera image enhancement method with two-channel fusion - Google Patents

Traffic camera image enhancement method with two-channel fusion Download PDF

Info

Publication number
CN112785543B
CN112785543B CN202110224933.9A CN202110224933A CN112785543B CN 112785543 B CN112785543 B CN 112785543B CN 202110224933 A CN202110224933 A CN 202110224933A CN 112785543 B CN112785543 B CN 112785543B
Authority
CN
China
Prior art keywords
image
fusion
layer
enhancement
scale
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110224933.9A
Other languages
Chinese (zh)
Other versions
CN112785543A (en
Inventor
王宁
瞿关明
陈澎祥
陆翔云
杨东超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tiandy Technologies Co Ltd
Original Assignee
Tiandy Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tiandy Technologies Co Ltd filed Critical Tiandy Technologies Co Ltd
Priority to CN202110224933.9A priority Critical patent/CN112785543B/en
Publication of CN112785543A publication Critical patent/CN112785543A/en
Application granted granted Critical
Publication of CN112785543B publication Critical patent/CN112785543B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10048—Infrared image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention provides a traffic camera image enhancement method with two channels fused, which comprises the following steps: the image decomposition comprises the steps of decomposing the image data of two channels into N layers of image sequences respectively to obtain two groups of multi-scale image data; performing multi-scale fusion enhancement, namely correspondingly fusing two groups of multi-scale image data sequences layer by layer to obtain a group of N-layer image sequences; and image reconstruction, namely reconstructing the image sequence of the N layers obtained by fusion by using a reconstruction method to obtain a final fusion image. The method for enhancing the traffic camera image with the two channels fused generates the snap-shot image with clear characteristics and rich details, so that contents such as vehicle driving states, passenger information and the like on the road are listed, and accurate data are provided for road law enforcement.

Description

Traffic camera image enhancement method with two-channel fusion
Technical Field
The invention belongs to the technical field of video monitoring, and particularly relates to a traffic camera image enhancement method with two channels fused.
Background
In the road law enforcement process, the traffic camera can be used for providing law enforcement basis all the day, so that the road management efficiency is improved, the road driving environment is cleared, and the traffic illegal behaviors are reduced. However, when the existing traffic camera works, the adopted scheme needs to be matched with a strong light supplement lamp, and strong flash light during snapshot directly irradiates eyes of a driver, so that the driver is seriously blinded momentarily. The infrared light is invisible light for human eyes, and is used for supplementing light to a road scene, so that the problem of glare during traffic snapshot can be solved. With the development of the production and manufacturing process, the condition for producing the dual-channel lens in batches is provided. A beam splitter prism is integrated in the dual-channel lens to split a light beam into visible light and infrared light, and two photosensitive chips are loaded to form the color/infrared dual-channel image acquisition lens. And then, by utilizing the dual-channel fusion enhancement method provided by the invention, the original color and infrared images are subjected to fusion enhancement processing, and finally, the snap-shot picture with clear characteristics and rich details is obtained.
Disclosure of Invention
In view of the above, the invention aims to provide a traffic camera image enhancement method with two channels fused, so as to solve the adverse effect of flicker of a light supplement lamp on a vehicle driver during snapshot of a traffic camera, reduce light pollution and improve road safety.
In order to achieve the purpose, the technical scheme of the invention is realized as follows:
a dual-channel fused traffic camera image enhancement method comprises the following steps:
the image decomposition comprises the steps of decomposing the image data of two channels into N layers of image sequences respectively to obtain two groups of multi-scale image data;
performing multi-scale fusion enhancement, namely correspondingly fusing two groups of multi-scale image data sequences layer by layer respectively to obtain a group of N-layer image sequences;
and (4) image reconstruction, namely reconstructing the image sequence of the N layers obtained by fusion by using a reconstruction method to obtain a final fusion image.
Further, the image decomposition is performed by using a laplacian pyramid method.
Further, the specific method of image decomposition is as follows:
acquiring two paths of original image YUV data, namely a color image and an infrared image, wherein Y is brightness, and UV is chromatic aberration;
step (2), the brightness of the two paths of original images is respectively decomposed by utilizing a Laplacian pyramid decomposition method,
Figure BDA0002956899940000021
Figure BDA0002956899940000022
wherein G is 0 For the brightness of the original image, DOWN (-) is the DOWN sampling operation, UP (-) is the UP sampling operation,
Figure BDA0002956899940000023
represents the convolution, g 5×5 Is a 5 × 5 gaussian kernel.
Furthermore, a Gaussian fusion rule, a Laplace fusion rule and a color fusion rule are used for carrying out multi-scale fusion enhancement.
Further, the method for performing multi-scale fusion enhancement is as follows:
gaussian fusion, using Gaussian images of color images
Figure BDA0002956899940000024
Gauss plot with infrared images
Figure BDA0002956899940000025
Performing fusion to obtain a generated image
Figure BDA0002956899940000026
Figure BDA0002956899940000027
Wherein K is a Gaussian layer pixel coefficient satisfying 0 ≦ (K) ij ≤1；
Figure BDA00029568999400000210
Expressing the Hadamard product, i.e. multiplying the matrix element by element, the Gaussian layer pixel coefficient K is based on
Figure BDA0002956899940000028
And
Figure BDA0002956899940000029
the difference of (a);
laplace fusion, fusing Laplace maps of color images, respectively
Figure BDA0002956899940000031
Laplace map with infrared image
Figure BDA0002956899940000032
Obtaining a generated image
Figure BDA0002956899940000033
Figure BDA0002956899940000034
Wherein M is i Is the Laplace pixel coefficient of the i layer, is composed of
Figure BDA0002956899940000035
And
Figure BDA0002956899940000036
calculating the difference of (2);
color fusion, decomposition of brightness by generating a map
Figure BDA0002956899940000037
And of the original colour image
Figure BDA0002956899940000038
Color data UV are generated.
Further, a laplacian pyramid reconstruction method is used for image reconstruction, and the formula is as follows:
Figure BDA0002956899940000039
obtaining the final generated image
Figure BDA00029568999400000310
Compared with the prior art, the dual-channel fusion traffic camera image enhancement method has the following advantages:
(1) The traffic camera image enhancement method with the two-channel fusion technically solves the problem that a light supplement lamp flickers to cause 'stubborn disease' of light pollution in the process of road electronic eye law enforcement, so that an environment-friendly road traffic environment is constructed; meanwhile, the method generates the snap-shot image with clear characteristics and rich details, so that contents such as the driving state of the vehicle on the road, information of passengers and the like are listed, and accurate data are provided for road law enforcement.
(2) On the basis of a Laplace pyramid decomposition and reconstruction method, the invention provides three multi-scale fusion rules, and fuses a color fuzzy image with insufficient light at night with an infrared black-and-white image with clear details to obtain a snap picture with clear characteristics and rich details, thereby providing a reliable basis for road law enforcement.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate an embodiment of the invention and, together with the description, serve to explain the invention and not to limit the invention. In the drawings:
fig. 1 is a flowchart of a traffic camera image enhancement method with two-channel fusion according to an embodiment of the present invention.
Detailed Description
It should be noted that the embodiments and features of the embodiments of the present invention may be combined with each other without conflict.
The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.
As shown in fig. 1, a method for enhancing a traffic camera image by two-channel fusion includes the following steps:
the image decomposition comprises the steps of decomposing the image data of two channels into N layers of image sequences respectively to obtain two groups of multi-scale image data;
performing multi-scale fusion enhancement, namely correspondingly fusing two groups of multi-scale image data sequences layer by layer to obtain a group of N-layer image sequences;
and (4) image reconstruction, namely reconstructing the image sequence of the N layers obtained by fusion by using a reconstruction method to obtain a final fusion image.
Wherein, the image decomposition is carried out by utilizing a Laplacian pyramid method.
The specific method of image decomposition is as follows:
acquiring two paths of original image YUV data, namely a color image and an infrared image, wherein Y is brightness, and UV is chromatic aberration;
step (2), the brightness of the two paths of original images is decomposed respectively by utilizing a Laplacian pyramid decomposition method,
Figure BDA0002956899940000041
Figure BDA0002956899940000042
wherein G 0 For the brightness of the original image, DOWN (-) is a DOWN sampling operation (removing even rows and even columns), UP (-) is an UP sampling operation (inserting a row/column 0 between each row/column),
Figure BDA0002956899940000051
represents the convolution, g 5×5 Is a 5 × 5 gaussian kernel.
And performing multi-scale fusion enhancement by using a Gaussian fusion rule, a Laplace fusion rule and a color fusion rule.
The method for performing multi-scale fusion enhancement is as follows:
gaussian fusion, using Gaussian images of color images
Figure BDA0002956899940000052
Gaussian with infrared image
Figure BDA0002956899940000053
Performing fusion to obtain a generated image
Figure BDA0002956899940000054
Figure BDA0002956899940000055
Wherein K is the Gaussian layer pixel coefficient (the size of the image row and column is consistent), and the K is more than or equal to 0 ij ≤1；
Figure BDA00029568999400000516
Expressing the Hadamard product, multiplying the matrix element by element, and calculating the Gaussian layer pixel coefficient
Figure BDA0002956899940000056
And
Figure BDA0002956899940000057
the difference of (a);
laplace fusion, fusing Laplace maps of color images, respectively
Figure BDA0002956899940000058
Laplace map with infrared image
Figure BDA0002956899940000059
Obtaining a generated image
Figure BDA00029568999400000510
Figure BDA00029568999400000511
Wherein M is i The coefficients of Laplace pixels of the i-layer (corresponding to the size of the image rows and columns) are determined by
Figure BDA00029568999400000512
And
Figure BDA00029568999400000513
calculating the difference of (2);
color fusion, decomposition of brightness by generating a map
Figure BDA00029568999400000514
And of the original colour image
Figure BDA00029568999400000515
Color data UV are generated.
The image reconstruction is carried out by utilizing a Laplacian pyramid reconstruction method, and the formula is as follows:
Figure BDA0002956899940000061
obtaining the final generated image
Figure BDA0002956899940000062
Those of ordinary skill in the art will appreciate that the elements and method steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of clearly illustrating the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the technical solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided in the present application, it should be understood that the disclosed method and system may be implemented in other ways. For example, the above described division of elements is merely a logical division, and other divisions may be realized, for example, multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not executed. The units may or may not be physically separate, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiments of the present invention.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the present invention, and they should be construed as being included in the following claims and description.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and should not be taken as limiting the scope of the present invention, which is intended to cover any modifications, equivalents, improvements, etc. within the spirit and scope of the present invention.

Claims (5)

1. A dual-channel fused traffic camera image enhancement method is characterized by comprising the following steps:
the image decomposition comprises the steps of decomposing the image data of two channels into N layers of image sequences respectively to obtain two groups of multi-scale image data;
performing multi-scale fusion enhancement, namely correspondingly fusing two groups of multi-scale image data sequences layer by layer respectively to obtain a group of N-layer image sequences;
reconstructing an image, namely reconstructing the image sequence of the N layers obtained by fusion by using a reconstruction method to obtain a final fusion image;
the method for performing multi-scale fusion enhancement is as follows:
gaussian fusion, using Gaussian images of color images
Figure FDA0003791718120000011
Gauss plot with infrared images
Figure FDA0003791718120000012
Performing fusion to obtain a generated image
Figure FDA0003791718120000013
Figure FDA0003791718120000014
Wherein K is a Gaussian layer pixel coefficient satisfying 0 ≦ (K) ij ≤1；
Figure FDA0003791718120000015
Expressing the Hadamard product, i.e. multiplying the matrix element by element, the Gaussian layer pixel coefficient K is based on
Figure FDA0003791718120000016
And
Figure FDA0003791718120000017
the difference of (a);
laplace fusion, fusing the Laplace maps of color images separately
Figure FDA0003791718120000018
Laplace map with infrared image
Figure FDA0003791718120000019
Obtaining a generated image
Figure FDA00037917181200000110
Figure FDA00037917181200000111
Wherein M is i Is the Laplace pixel coefficient of the i layer, is composed of
Figure FDA00037917181200000112
And
Figure FDA00037917181200000113
calculating the difference of (A);
color fusion, decomposition of luminance by generating a map
Figure FDA00037917181200000114
And of the original colour image
Figure FDA00037917181200000115
Color data UV are generated.
2. The dual-channel fused traffic camera image enhancement method according to claim 1, characterized in that: and performing image decomposition by using a Laplacian pyramid method.
3. The dual-channel fused traffic camera image enhancement method according to claim 1 or 2, wherein the specific method of image decomposition is as follows:
acquiring two paths of original image YUV data, namely a color image and an infrared image, wherein Y is brightness, and UV is chromatic aberration;
step (2), the brightness of the two paths of original images is decomposed respectively by utilizing a Laplacian pyramid decomposition method,
Figure FDA0003791718120000021
Figure FDA0003791718120000022
wherein G is 0 For the brightness of the original image, DOWN (-) is the DOWN-sampling operation, UP (-) is the UP-sampling operation,
Figure FDA0003791718120000023
represents the convolution, g 5×5 Is a 5 × 5 gaussian kernel.
4. The dual-channel fused traffic camera image enhancement method according to claim 1, characterized in that: and performing multi-scale fusion enhancement by using a Gaussian fusion rule, a Laplace fusion rule and a color fusion rule.
5. The dual-channel fused traffic camera image enhancement method of claim 1, wherein: reconstructing an image by using a Laplacian pyramid reconstruction method, wherein the formula is as follows:
Figure FDA0003791718120000024
obtaining the final generated image
Figure FDA0003791718120000025
CN202110224933.9A 2021-03-01 2021-03-01 Traffic camera image enhancement method with two-channel fusion Active CN112785543B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110224933.9A CN112785543B (en) 2021-03-01 2021-03-01 Traffic camera image enhancement method with two-channel fusion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110224933.9A CN112785543B (en) 2021-03-01 2021-03-01 Traffic camera image enhancement method with two-channel fusion

Publications (2)

Publication Number Publication Date
CN112785543A CN112785543A (en) 2021-05-11
CN112785543B true CN112785543B (en) 2023-01-24

Family

ID=75762049

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110224933.9A Active CN112785543B (en) 2021-03-01 2021-03-01 Traffic camera image enhancement method with two-channel fusion

Country Status (1)

Country Link
CN (1) CN112785543B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111369486A (en) * 2020-04-01 2020-07-03 浙江大华技术股份有限公司 Image fusion processing method and device
CN111563552A (en) * 2020-05-06 2020-08-21 浙江大华技术股份有限公司 Image fusion method and related equipment and device
CN112184604A (en) * 2020-09-15 2021-01-05 杭州电子科技大学 Color image enhancement method based on image fusion
CN112365431A (en) * 2020-09-24 2021-02-12 广东外语外贸大学 Image enhancement method, device and equipment based on pyramid decomposition and storage medium

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8411938B2 (en) * 2007-11-29 2013-04-02 Sri International Multi-scale multi-camera adaptive fusion with contrast normalization
US9697588B2 (en) * 2010-11-15 2017-07-04 Intuitive Surgical Operations, Inc. System and method for multi-resolution sharpness transport across color channels
CN108154493B (en) * 2017-11-23 2021-11-30 南京理工大学 FPGA-based dual-waveband infrared image pseudo-color fusion algorithm

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111369486A (en) * 2020-04-01 2020-07-03 浙江大华技术股份有限公司 Image fusion processing method and device
CN111563552A (en) * 2020-05-06 2020-08-21 浙江大华技术股份有限公司 Image fusion method and related equipment and device
CN112184604A (en) * 2020-09-15 2021-01-05 杭州电子科技大学 Color image enhancement method based on image fusion
CN112365431A (en) * 2020-09-24 2021-02-12 广东外语外贸大学 Image enhancement method, device and equipment based on pyramid decomposition and storage medium

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
A Fast Fusion Method for Multi-exposure Image in YUV Color Space;Yanjie Liu et al.;《2018 IEEE 3rd Advanced Information Technology, Electronic and Automation Control Conference(IAEAC 2018)》;20181231;第1685-1689页 *
余明伟.红外与可见光实时融合的FPGA实现.《中国优秀博硕士学位论文全文数据库(硕士) 信息科技辑》.2017,(第02期), *
红外与可见光实时融合的FPGA实现;余明伟;《中国优秀博硕士学位论文全文数据库(硕士) 信息科技辑》;20170215(第02期);论文第20-26页 *

Also Published As

Publication number Publication date
CN112785543A (en) 2021-05-11

Similar Documents

Publication Publication Date Title
CN110866879B (en) Image rain removing method based on multi-density rain print perception
CN108416292B (en) Unmanned aerial vehicle aerial image road extraction method based on deep learning
CN111915530A (en) End-to-end-based haze concentration self-adaptive neural network image defogging method
US20220351344A1 (en) Multi-exposure image fusion method based on feature distribution weight of multi-exposure image
CN109410144B (en) End-to-end image defogging processing method based on deep learning
CN113505772B (en) License plate image generation method and system based on generation countermeasure network
CN113762264A (en) Multi-encoder fused multispectral image semantic segmentation method
Zheng et al. Low-light image and video enhancement: A comprehensive survey and beyond
CN112785543B (en) Traffic camera image enhancement method with two-channel fusion
CN112950521B (en) Image defogging method and generator network
CN113989785A (en) Driving scene classification method, device, equipment and storage medium
CN114155165A (en) Image defogging method based on semi-supervision
CN108648144A (en) A kind of FPM high-resolution colour picture method for reconstructing based on deep learning algorithm
CN116596792B (en) Inland river foggy scene recovery method, system and equipment for intelligent ship
CN110930343A (en) SR-MDCNN-based remote sensing image fusion method
CN114283288B (en) Method, system, equipment and storage medium for enhancing night vehicle image
CN116935214A (en) Space-time spectrum fusion method for satellite multi-source remote sensing data
CN116912485A (en) Scene semantic segmentation method based on feature fusion of thermal image and visible light image
Dong et al. Infrared image colorization using an edge aware auto encoder decoder with the multi-resolution fusion
CN112184566B (en) Image processing method and system for removing adhered water mist and water drops
CN113657183A (en) Vehicle 24 color identification method under smooth neural network based on multilayer characteristics
CN115601280A (en) Image processing method and vehicle
CN113191991A (en) Multi-modal image fusion method, system, device and medium based on information bottleneck
CN113191971A (en) Unmanned aerial vehicle image defogging method based on YUV color space
Shen et al. Light Field Reflection and Background Separation Network Based on Adaptive Focus Selection

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant