CN107623831A - A kind of building visual phone imaging device full-color round the clock - Google Patents
A kind of building visual phone imaging device full-color round the clock Download PDFInfo
- Publication number
- CN107623831A CN107623831A CN201710687819.3A CN201710687819A CN107623831A CN 107623831 A CN107623831 A CN 107623831A CN 201710687819 A CN201710687819 A CN 201710687819A CN 107623831 A CN107623831 A CN 107623831A
- Authority
- CN
- China
- Prior art keywords
- image
- color
- color image
- sensor
- infrared light
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 25
- 230000000007 visual effect Effects 0.000 title abstract description 9
- 230000003287 optical effect Effects 0.000 claims abstract description 9
- 238000000034 method Methods 0.000 claims description 27
- 238000001914 filtration Methods 0.000 claims description 21
- 238000009499 grossing Methods 0.000 claims description 15
- 238000005286 illumination Methods 0.000 claims description 14
- 238000005259 measurement Methods 0.000 claims description 11
- 230000004927 fusion Effects 0.000 claims description 9
- 230000009466 transformation Effects 0.000 claims description 9
- 230000005540 biological transmission Effects 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 6
- 238000000354 decomposition reaction Methods 0.000 claims description 5
- 230000006835 compression Effects 0.000 claims description 4
- 238000007906 compression Methods 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 4
- 238000003672 processing method Methods 0.000 claims description 4
- 230000003044 adaptive effect Effects 0.000 claims description 3
- 230000000295 complement effect Effects 0.000 claims description 2
- 238000005070 sampling Methods 0.000 claims description 2
- 238000000926 separation method Methods 0.000 claims description 2
- 239000013598 vector Substances 0.000 claims description 2
- 239000000203 mixture Substances 0.000 abstract 1
- 239000013589 supplement Substances 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 125000001475 halogen functional group Chemical group 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
The present invention discloses a kind of building visual phone imaging device full-color round the clock, belongs to building visual telephony field, for solving the problems, such as that existing building visual phone can not obtain night coloured image in the case of near-infrared light filling.Present invention building visual phone imaging device full-color round the clock, illuminated using near-infrared light filling, using the anti-reflection camera lens of near-infrared and monochrome image sensor shooting near-infrared+visible ray gray level image, using the visible light lens parallel with the anti-reflection camera lens optical axis of near-infrared and color image sensor shooting otherwise visible light color image, denoising is carried out to coloured image using gray level image as guiding, again color space is separated in color with brightness, image detail and color blend are carried out to gray level image and coloured image, obtain coloured image in the case of low-light (level) near-infrared light filling.The present invention can be used for realizing full-color building visual intercommunication round the clock.
Description
Technical Field
The invention belongs to the technical field of building videophones, and particularly relates to a day and night full-color building videophone imaging device.
Background
The video telephone for the building outputs the image and the audio of the visitor to the house by installing the imaging device and the audio device at the door of the building or the door of the room, so that the resident of the building can screen the visitor without opening the door, and accidents such as indoor robbery and the like are avoided.
The existing building video telephone can shoot high-quality color images when the illumination is sufficient, and can only shoot black and white images by supplementing near infrared light when the illumination is insufficient. The color image has important value for the discrimination of visitors. When the illumination is insufficient, the white light supplement can obtain a color image. However, the light supplement mode can be perceived by human eyes, and halo can be caused to the human eyes. Therefore, the problem that how to shoot color images by building videophones without using white light supplementary lighting under the condition of insufficient optical axis is urgently needed to be solved.
Disclosure of Invention
The invention discloses a day and night full-color building video telephone imaging device, aiming at solving the problem that the existing building video telephone imaging device cannot shoot color images under the conditions of insufficient illumination and no use of white light for light supplement.
The technical scheme for solving the problems is as follows: a day and night full-color building video telephone imaging device is composed of 1 casing (1), 1 photoelectric sensor (2), 1 near-infrared light source (3), 1 near-infrared anti-reflection lens (4), 1 monochrome image sensor (5), 1 visible light lens (6), 1 IRCUT (7), 1 color image sensor (8), 1 processing circuit (9), 1 power supply interface (10) and 1 group of data interface (11).
The front end of the shell (1) is provided with two lens interfaces for fixing the near-infrared anti-reflection lens (1) and the visible light lens (2), and the front end of the shell is provided with the photoelectric sensor (2).
The photoelectric sensor (2) is a visible light sensor and is used for measuring the intensity of visible light in a scene so as to judge whether the near infrared light source (3) is turned on or off.
The near-infrared light source (3) is fixed in front of the shell (1), complementary illumination is carried out on a scene under the condition of low illumination, the wavelength range of the near-infrared light source (3) is 800 nm-1000 nm, the illumination visual field angle is larger than the shooting visual field, the near-infrared light source (3) is provided with a control signal input interface and is connected with the processing circuit (9), and the processing circuit (9) controls the near-infrared light source to be turned on or turned off.
The rear end of the near-infrared anti-reflection lens (4) is provided with a monochrome image sensor (5) for shooting a gray image A.
A color image sensor (8) is arranged at the rear end of the visible light lens (6) and used for shooting a color image B, and an IRCUT (7) is arranged at the front end of the color image sensor (8) and used for filtering near infrared light under the condition that the near infrared light source (3) is turned on for illumination.
The color image sensor (8) is a Bayer image sensor, and the color image B is obtained by demosaicing the Bayer image Q shot by the Bayer image sensor.
The processing circuit (9) is embedded hardware and can be realized by chips such as ARM, DSP, FPGA, ASIC, singlechip and the like; as shown in fig. 2, the acquisition circuit (9) is connected with the control circuit interface of the photoelectric sensor (2) and the near-infrared light source (3), the monochrome image sensor (5), the color image sensor (8), the IRCUT (7), the power supply interface (10) and the data interface (11), supplies power to the photoelectric sensor (2), the monochrome image sensor (5), the color image sensor (8), the IRCUT (7) and the data interface (11), completes acquisition and processing of the measurement signal of the photoelectric sensor (2), image acquisition of the monochrome image sensor (5) and the color image sensor (8), completes operations such as ISP processing, image fusion, image transmission and the like, and performs drive control of the IRCUT and the near-infrared light source.
The power supply interface (10) is connected with the near-infrared light source (3) and the processing circuit (9) and supplies power to the near-infrared light source (3) and the processing circuit (9).
The data interface (11) is used for camera control and data transmission, and the related interface comprises: BNC, network.
The working process of the day and night full-color building video telephone imaging device is as follows: the processing circuit (9) collects the measurement result v of the photoelectric sensor (2), the value range is 0-255, when the measurement result v is greater than t, the camera enters a daytime working mode, when the measurement value v is less than t, the camera enters a nighttime working mode, wherein t is a judgment threshold value, and the value range is 0-255.
In the daytime working mode, the processing circuit (9) closes the near infrared light source (3) and the monochrome image sensor (5), drives the IRCUT to move away from an IR filter at the front end of the color image sensor, collects a Bayer image Q of the color image sensor (8), executes a standard ISP processing flow, acquires a daytime color image C, and performs operations such as compression, storage, transmission and the like on the image C.
In a night working mode, a processing circuit (9) starts a near infrared light source (3) to illuminate, a monochrome image sensor (5) and a color image sensor (8), drives an IRCUT to move an IR filter to the front end of the color image sensor, synchronously acquires a gray image A shot by the monochrome image sensor (5) and a Bayer image Q shot by the color image sensor (8), demosaices the Bayer image Q to obtain a color image B, performs image fusion on the gray image A and the color image B to obtain a night color image C ', and then compresses and transmits the image C'.
In the night working mode, the specific method for demosaicing the Bayer image Q to obtain the color image B is as follows: with 2 × 2 pixels as processing units, according to the Bayer imaging pattern, r and B channel pixels in 2 × 2 pixel blocks are extracted as r and B channel values of the color image B, and g channel pixel mean values in 2 × 2 pixel blocks are extracted as g channel values of the color image B, so that the resolution of the obtained color image B is 1/4 of that of the Bayer image Q.
And in the night working mode, fusing the gray image A and the color image B to obtain a night color image C', wherein the method comprises a method a and a method B.
The method a comprises the following operation steps:
step a-1: adopting an image interpolation method to perform up-sampling on the color image B to obtain a color image B ', so that the resolution of the color image B' is the same as that of the gray image A; calibrating a binocular stereo imaging system formed by a near infrared anti-reflection lens (4) and a visible light lens (6), and acquiring a pixel coordinate mapping relation P between a gray level image A and a color image B': (i) A ,j A )＝P(i B' ,j B' ) Wherein (i) A ,j A )、(i B' ,j B' ) Is the pixel coordinates of the grayscale image a and the color image B';
the image interpolation method comprises linear interpolation, bilinear interpolation, gaussian interpolation, B spline interpolation and RBF interpolation;
step a-2: according to the coordinate mapping relation P, carrying out coordinate transformation on the color image B ' to obtain a color image B ', extracting the overlapping area of the gray image A and the color image B ' to respectively obtain a gray image E and a color image D, wherein at the moment, the field of view of the gray image E is overlapped with that of the color image D;
step a-3: taking the gray image E as a guide image, and performing color matching on r, g and b color channels D of the color image D r ,D g ,D b Carrying out local smooth filtering to obtain a denoised color image
Step a-4, in the color space lambda of color and brightness component, the color image after denoising is processedConversion to imagesWhereinIs a component of the luminance that is,is a color component, is replaced with a grayscale image EObtaining an imageImage processing methodInto an rgb color space image C' in which the luminance information originates from image E and the color information originates from image E
In the steps a-3, the specific operation steps of performing local smoothing filtering on the color image D by using the image E as a guide image are as follows:
step a-3-1:
calculating the local neighborhood mean of the image E:
μ E ＝f m (E) (1)
calculating three color channels D of an image D r ,D g ,D b Local neighborhood mean of (c):
step a-3-2:
calculating the local neighborhood variance of image E:
σ E ＝f m (E.*E)-μ E .*μ E (5)
calculating three color channels D of an image E and an image D r ,D g ,D b Local neighborhood covariance of the corresponding pixel in (1):
step a-3-3:
calculating linear transformation coefficients:
wherein, tau r ,τ g ,τ b Is a penalty coefficient with the value range of (0-10);
step a-3-4:
calculating the local neighborhood mean of the linear transformation coefficient:
step a-3-5:
for image D three color channels D r ,D g ,D b Performing linear transformation:
wherein, f m And the window size is s, the value range of s is 3-111, the point multiplication operation represents the multiplication of elements at the same position in the matrix, and the point division operation represents the division of the elements at the same position in the matrix.
Method b is to obtain the color image from step a-3 in method aAnd gray scaleAnd (3) carrying out image detail fusion on the image E to obtain a high-quality low-illumination color image C ", and specifically operating as follows:
b-1, step: color image is processed in color space lambda of color and brightness componentsConversion to imagesWhereinIs the component of the luminance that is,is a color component;
the color and brightness separation color space lambda comprises YUV, YCbCr, HSI, lab and CIEXYZ color spaces;
b-2, step: performing image edge-preserving multi-scale decomposition on the image E to obtain a base imageAnd detail imagesWherein (n + 1) is the number of layers of image decomposition, and the value range is 1-10;
the base imageThe image edge-preserving smooth filtering results in:
the image edge-preserving smoothing filter f smooth (. Includes weighted least squares edge-preserving smoothing filtering, local extremum-based edge-preserving smoothing filtering, local laplacian pyramid-based edge-preserving smoothing filteringMedian filtering, weighted median filtering methods;
multi-layer detail imagesFrom the base imageAnd obtaining by difference with smooth images of different scales:
multi-layer detail imageIt can also be obtained by differencing two smooth images of different scales:
wherein, f smooth (. Is) image edge-preserving smoothing filtering, superscript 0, z 1 、z 2 The index is a label of a filter scale layer, the larger the label is, the smaller the scale is, and the label is 0 to represent the maximum filtering scale;
b-3, step: will be the base imageDetail imageAnd a luminance componentAfter weighted fusion, a new brightness component is obtained
Wherein, { omega } 0 ,ω 1 ,…,ω n+1 The weighting coefficient is in the value range of 0 to 1;
step b-4: based on new luminance componentsImage processing methodConversion to rgb color space color image C ", the luminance information in image C" is derived from image E and imageThe color information being derived from the image
Preferably, the monochrome image sensor (5) and the color image sensor (7) have the same target surface size and resolution, the imaging planes are coplanar, the near infrared anti-reflection lens (4) and the visible lens (6) have the same focal length, the same aperture F number and the same adaptive target surface size, the optical axes (12-1 and 12-2) of the infrared anti-reflection lens (4) and the visible lens (6) are parallel and vertically arranged, and the optical axis distance g is less than 30mm.
Preferably, the step a-2 can be realized by table look-up operation, and according to the coordinate mapping relation P: (i) A ,j A )＝P(i B' ,j B' ) And overlapping the field of view range, calculating the coordinate correspondence (i ') from the gray image E to the color image D' B' ,j′ B' )＝Q(i E ,j E ) Then, the look-up table LUT is used for pre-storage; the look-up table LUT is of length w h and the memory elements are 2-dimensional vectors (i' B' ,j′ B' ) W, h are the width and height of the grayscale image E, and the value range is: 1 to 10000, unit is pixel, (i) E ,j E ) Is the pixel coordinate of the gray scale image E with L = i E *w+j E As an inputThe output is (i' B' ,j′ B' ) (ii) a The specific operation flow is as follows: firstly, cutting an overlapped area of the gray level A and the color image B' as a gray level image E, and establishing a blank color image D with the same size as the gray level image E; next, the pixel coordinates (i) of the color image D are taken D ,j D ) Calculating an input value L = i of a look-up table LUT D *w+j D From the look-up table LUT, the pixel coordinate (i) is found D ,j D ) The corresponding value of (c): (i' B' ,j′ B' ) (i 'in color image B' B' ,j′ B' ) Filling of the at-position image into the color image D (i) D ,j D ) At a location; and traversing the color image D to complete color image D filling.
Compared with the prior art, the invention has the beneficial effects that: compared with the existing visual building telephone imaging device which can only shoot black and white images under the condition of near-infrared supplementary lighting at night, the imaging device can realize color imaging, and the near-infrared supplementary lighting is not perceived by human eyes, so that the human eyes cannot feel dazzling light.
Drawings
FIG. 1 is a schematic diagram of a day and night full color video phone imaging device for buildings according to the present invention;
FIG. 2 is a connection diagram of units of a day-night full-color video phone imaging device for a building;
in the figure, 1-a shell, 2-a photoelectric sensor, 3-a near-infrared light source, 4-a near-infrared anti-reflection lens, 5-a monochrome image sensor, 6-a visible light lens, 7-IRCUT, 8-a color image sensor, 9-a processing circuit, 10-a power supply interface, 11-a data interface (11) and 12-a lens optical axis.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and examples.
Example 1
As shown in fig. 2, the day and night full-color building videophone imaging device includes a casing (1), a photoelectric sensor (2), a near-infrared light source (3), a near-infrared anti-reflection lens (4), a monochrome image sensor (5), a visible light lens (6), an IRCUT (7), a color image sensor (8), a processing circuit (9), a power supply interface (10), and a data interface (11). Wherein the power of the near-infrared light source (3) is 5W, and the wavelength is 808nm; the near-infrared anti-reflection lens (4) and the visible light lens (6) are M12 interfaces, the aperture F =1.4, the size of the adaptive target surface is 1/3 inch, and the focal length is F =3mm; the monochrome image sensor adopts a 100 ten thousand pixel Mono CMOS starlight level image sensor, the frame rate is 30fps, the color image sensor adopts a 100 ten thousand pixel Bayer CMOS starlight level image sensor, the frame rate is 30fps, and the chip areas of the monochrome image sensor and the color image sensor are 1/3 inch; the processing circuit is realized based on FPGA, and the data interface is a network interface. The optical axes of the near-infrared anti-reflection lens (4) and the visible light lens (6) are parallel and are horizontally arranged at the front end of the shell. The optical axis distance g =25mm of the near infrared anti-reflection lens (4) and the visible light lens (6).
The imaging device work flow is as follows: the processing circuit (9) collects the measurement result v of the photoelectric sensor (2), when the measurement result v > t, t is set to be 10, the camera enters a daytime working mode, and when the measurement value v < t, the camera enters a nighttime working mode.
In the daytime working mode, the processing circuit (9) closes the near infrared light source (3) and the monochrome image sensor (5), drives the IRCUT to move away from an IR filter at the front end of the color image sensor, collects a Bayer image Q of the color image sensor (8), executes a standard ISP processing flow, acquires a daytime color image C, and performs operations such as compression, storage, transmission and the like on the image C.
In a night working mode, a processing circuit (9) starts a near infrared light source (3) to illuminate, a monochrome image sensor (5) and a color image sensor (8), drives an IRCUT to move an IR filter to the front end of the color image sensor, synchronously acquires a gray image A shot by the monochrome image sensor (5) and a Bayer image Q shot by the color image sensor (8), demosaices the Bayer image Q to obtain a color image B, performs image fusion on the gray image A and the color image B to obtain a night color image C ', and then compresses and transmits the image C'.
In the night working mode, 2 × 2 pixels are used as processing units, r and B channel pixels in 2 × 2 pixel blocks are extracted as r and B channel values of a color image B according to a Bayer imaging mode, g channel pixel average values in the 2 × 2 pixel blocks are extracted as g channel values of the color image B, and the resolution of the obtained color image B is 1/4 of that of a Bayer image Q.
In the night working mode, the gray image A and the color image B are fused by a method a to obtain a night color image C', wherein f m The size of the filtering window of the mean value of the (DEG) image is 5, and the linear interpolation is selected by the image interpolation method.
Example 2
The difference from embodiment 1 is that in the night working mode, the grayscale image A and the color image B are fused by the method B to obtain the high-quality night color image C ", wherein f m The size of an image mean filtering window is 5, the color space lambda selects a YUV color space, the number of layers n =4 of image edge-preserving multi-scale decomposition is calculated, and smooth filtering f of image edge-preserving smooth (. The) select weighted least squares to ensure edge smoothing.
Example 3
The difference from example 2 is that the color space λ is a Lab color space.
Example 4
The difference from embodiment 2 is that the color space λ is an HSV color space.
Example 5
The difference from embodiment 2 is that the edge-preserving smoothing filter f smooth (. The) select a local extremum based preserved edge smoothing filter.
Example 6
The difference from the embodiment 1 is that in the step a-2 of the method a, the image coordinate transformation is accelerated by using a lookup table method.
Claims (3)
1. A day and night full-color building video telephone imaging device is characterized in that: the system comprises 1 camera shell (1), 1 photoelectric sensor (2), 1 near-infrared light source (3), 1 near-infrared anti-reflection lens (4), 1 monochrome image sensor (5), 1 visible light lens (6), 1 IRCUT (7), 1 color image sensor (8), 1 processing circuit (9), 1 power supply interface (10) and 1 group of data interface (11);
the front end of the camera shell (1) is provided with two lens interfaces for fixing the near-infrared anti-reflection lens (1) and the visible light lens (2), and the front end of the camera shell is provided with the photoelectric sensor (2);
the photoelectric sensor (2) is a visible light sensor and is used for measuring the intensity of visible light in a scene so as to judge whether the near infrared light source (3) is turned on or off;
the near-infrared light source (3) is fixed in front of the camera shell (1), complementary illumination is carried out on a scene under the condition of low illumination, the wavelength range of the near-infrared light source (3) is 800 nm-1000 nm, the illumination field angle is larger than the shooting field angle, the near-infrared light source (3) is provided with a control signal input interface and is connected with the processing circuit (9), and the processing circuit (9) controls the near-infrared light source to be turned on or turned off;
a monochrome image sensor (5) is arranged at the rear end of the near-infrared anti-reflection lens (4) and used for shooting a gray level image A;
a color image sensor (8) is arranged at the rear end of the visible light lens (6) and used for shooting a color image B, and an IRCUT (7) is arranged at the front end of the color image sensor (8) and used for filtering near infrared light when the near infrared light source (3) is turned on for illumination;
the color image sensor (8) is a Bayer image sensor, and a color image B is obtained by demosaicing a Bayer image Q shot by the Bayer image sensor;
the processing circuit (9) is embedded hardware and can be realized by chips such as ARM, DSP, FPGA, ASIC, singlechip and the like; the acquisition circuit (9) is connected with the photoelectric sensor (2), a control circuit interface of the near-infrared light source (3), the monochrome image sensor (5), the color image sensor (8), the IRCUT (7), a power supply interface (10) and a data interface (11) to supply power to the photoelectric sensor (2), the monochrome image sensor (5), the color image sensor (8), the IRCUT (7) and the data interface (11) so as to finish acquisition and processing of measurement signals of the photoelectric sensor (2), acquire images of the monochrome image sensor (5) and the color image sensor (8), finish operations such as ISP processing, image fusion, image compression, image storage, image transmission and the like, and carry out drive control of the IRCUT and the near-infrared light source;
the power supply interface (10) is connected with the near-infrared light source (3) and the processing circuit (9) and supplies power to the near-infrared light source (3) and the processing circuit (9);
the data interface (11) is used for camera control and data transmission, and the related interface comprises: BNC, network, HD-SDI, HDMI;
the camera of the invention has the following working procedures: the processing circuit (9) collects the measurement result v of the photoelectric sensor (2), the value range is 0-255, when the measurement result v is greater than t, the camera enters a daytime working mode, when the measurement value v is less than t, the camera enters a night working mode, wherein t is a judgment threshold value, and the value range is 0-255; in a daytime working mode, the processing circuit (9) closes the near infrared light source (3) and the monochrome image sensor (5), drives the IRCUT to move away from an IR filter at the front end of the color image sensor, collects a Bayer image Q of the color image sensor (8), executes a standard ISP processing flow, acquires a daytime color image C, and performs operations such as compression, storage, transmission and the like on the image C; in a night working mode, a processing circuit (9) starts a near infrared light source (3) to illuminate, a monochrome image sensor (5) and a color image sensor (8), drives an IRCUT to move an IR filter to the front end of the color image sensor, synchronously acquires a gray image A shot by the monochrome image sensor (5) and a Bayer image Q shot by the color image sensor (8), demosaices the Bayer image Q to obtain a color image B, performs image fusion on the gray image A and the color image B to obtain a night color image C ', and then compresses, stores and transmits the image C';
in the night working mode, the specific method for demosaicing the Bayer image Q to obtain the color image B is as follows: taking 2 × 2 pixels as a processing unit, extracting r and B channel pixels in 2 × 2 pixel blocks as r and B channel values of a color image B according to a Bayer imaging mode, and extracting g channel pixel mean values in the 2 × 2 pixel blocks as g channel values of the color image B, wherein the resolution of the obtained color image B is 1/4 of that of a Bayer image Q;
a method for fusing the gray image A and the color image B to obtain a night color image C' in a night working mode, which comprises a method a and a method B;
the method a comprises the following operation steps:
step a-1: adopting an image interpolation method to perform up-sampling on the color image B to obtain a color image B ', so that the resolution of the color image B' is the same as that of the gray image A; calibrating a binocular stereo imaging system formed by a near infrared anti-reflection lens (4) and a visible light lens (6), and acquiring a pixel coordinate mapping relation P between a gray level image A and a color image B': (i) A ,j A )＝P(i B' ,j B' ) Wherein (i) A ,j A )、(i B' ,j B' ) Is the pixel coordinates of the grayscale image a and the color image B';
the image interpolation method comprises linear interpolation, bilinear interpolation, gaussian interpolation, B spline interpolation and RBF interpolation;
step a-2: according to the coordinate mapping relation P, carrying out coordinate transformation on the color image B ' to obtain a color image B ', extracting the overlapping area of the gray image A and the color image B ' to respectively obtain a gray image E and a color image D, wherein at the moment, the view field of the gray image E is superposed with that of the color image D;
step a-3: taking the gray image E as a guide image, and performing color matching on r, g and b color channels D of the color image D r ,D g ,D b Carrying out local smooth filtering to obtain a denoised color image
Step a-4, in the color space lambda of color and brightness component, the color image is processedConversion to imagesWhereinIs the component of the luminance that is,is a color component, is replaced with a grayscale image EObtaining an imageImage processing methodInto an rgb color space image C' in which the luminance information originates from image E and the color information originates from image E
The specific operation steps of performing local smooth filtering on the color image D by taking the image E as a guide image in the step a-3 are as follows:
step a-3-1:
calculating the local neighborhood mean of the image E:
μ E ＝f m (E) (1)
calculating three color channels D of an image D r ,D g ,D b Local neighborhood mean of (c):
step a-3-2:
calculating the local neighborhood variance of the image E:
σ E ＝f m (E.*E)-μ E .*μ E (5)
calculating three color channels D of an image E and an image D r ,D g ,D b Local neighborhood covariance of the corresponding pixel in (1):
step a-3-3:
calculating linear transformation coefficients:
wherein, tau r ,τ g ,τ b Is a penalty coefficient with the value range of (0-10);
step a-3-4:
calculating the local neighborhood mean of the linear transformation coefficient:
a-3-5 step:
for image D three color channels D r ,D g ,D b Performing linear transformation:
wherein, f m (. Cndot.) is image mean filtering, window size s, s ranges from 3 to 111, dot multiplication, represents multiplication of elements at the same position in the matrix,/is a dot division operation, represents division of elements at the same position in the matrix;
method b is to obtain the color image from step a-3 in method aAnd carrying out image detail fusion with the gray level image E to obtain a high-quality low-illumination color image C ", and specifically operating as follows:
b-1 step: color image is processed in color space lambda of color and brightness componentsConversion to imagesWhereinIs a component of the luminance that is,is a colorA component;
b-2, step: performing image edge-preserving multi-scale decomposition on the image E to obtain a base imageAnd detail imagesWherein (n + 1) is the number of layers of image decomposition, and the value range is 1-10;
the base imageThe image edge-preserving smoothing filter obtains:
the image edge-preserving smoothing filter f smooth (. Comprises a weighted least square edge smoothing, a local extremum based edge smoothing, a local laplacian pyramid based edge smoothing, a median filtering, a weighted median filtering method;
multi-layer detail imagesFrom the base imageAnd obtaining by difference with smooth images of different scales:
multi-layer detail imageIt can also be obtained by differencing two smooth images of different scales:
wherein, f smooth (. Is) image edge-preserving smoothing filtering, superscript 0, z 1 、z 2 The reference number refers to the reference number of a filter scale layer, the larger the reference number is, the smaller the scale is, and the reference number is 0 to represent the maximum filtering scale;
b-3, step: will be the base imageDetail imageAnd a luminance componentAfter weighted fusion, a new brightness component is obtained
Wherein, { omega } 0 ,ω 1 ,…,ω n+1 The value range is 0 to 1;
b-4, steps: based on new luminance componentsImage processing methodConversion into rgb color space color image C ", image C" where the luminance information is derived from image E and imageThe color information being derived from the image
The color-to-brightness separation color space λ includes YUV, YCbCr, HSI, lab, CIEXYZ color spaces.
2. According to claim 1, the monochrome image sensor (5) and the color image sensor (7) have the same target surface size, resolution and coplanar imaging plane, the near infrared antireflection lens (4) and the visible lens (6) have the same focal length, F number and adaptive target surface size, the optical axes (12-1, 12-2) of the infrared antireflection lens (4) and the visible lens (6) are parallel and vertically arranged, and the optical axis spacing g is less than 50mm, so that the transverse fields of view coincide.
3. According to claim 1, the steps a-2 can be realized by table look-up operation, according to the coordinate mapping relation P: (i) A ,j A )＝P(i B' ,j B' ) And overlapping the field of view range, calculating the coordinate correspondence (i ') from the gray image E to the color image D' B' ,j′ B' )＝Q(i E ,j E ) Then, the look-up table LUT is used for pre-storage; the look-up table LUT is of length w h and the memory elements are 2-dimensional vectors (i' B' ,j′ B' ) W, h are the width and height of the grayscale image E, and the value range is: 1 to 10000, unit is pixel, (i) E ,j E ) Is the pixel coordinate of the gray scale image E with L = i E *w+j E As input, the output is (i' B' ,j′ B' ) (ii) a The specific operation flow is as follows: firstly, cutting an overlapped area of the gray level A and the color image B' as a gray level image E, and establishing a blank color image D with the same size as the gray level image E; next, the pixel coordinates (i) of the color image D are taken D ,j D ) Calculating an input value L = i of a look-up table LUT D *w+j D From the look-up table LUT, the pixel coordinate (i) is found D ,j D ) The corresponding value of (c): (i' B' ,j′ B' ) (i 'in color image B' B' ,j′ B' ) Image filling at a locationInto the color image D (i) D ,j D ) At a location; and traversing the color image D to complete color image D filling.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710687819.3A CN107623831A (en) | 2017-08-12 | 2017-08-12 | A kind of building visual phone imaging device full-color round the clock |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710687819.3A CN107623831A (en) | 2017-08-12 | 2017-08-12 | A kind of building visual phone imaging device full-color round the clock |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107623831A true CN107623831A (en) | 2018-01-23 |
Family
ID=61088820
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710687819.3A Pending CN107623831A (en) | 2017-08-12 | 2017-08-12 | A kind of building visual phone imaging device full-color round the clock |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107623831A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110248105A (en) * | 2018-12-10 | 2019-09-17 | 浙江大华技术股份有限公司 | A kind of image processing method, video camera and computer storage medium |
| CN110891138A (en) * | 2018-09-10 | 2020-03-17 | 杭州萤石软件有限公司 | Black light full-color realization method and black light full-color camera |
| CN111679283A (en) * | 2020-06-10 | 2020-09-18 | 南京凯基特电气有限公司 | Multifunctional anti-interference wide-range laser sensor system and use method |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202475619U (en) * | 2012-02-03 | 2012-10-03 | 深圳市保千里电子有限公司 | Night vision system for eliminating stray light |
| KR20140127631A (en) * | 2013-04-25 | 2014-11-04 | 삼성테크윈 주식회사 | Photographing apparatus and method for night photography |
| CN204836402U (en) * | 2015-07-29 | 2015-12-02 | 山东神戎电子股份有限公司 | Laser night -time vision device suitable for small -size aircraft |
| CN205017431U (en) * | 2015-09-24 | 2016-02-03 | 广州市巽腾信息科技有限公司 | Multi -functional removal image processing device |
| CN205160681U (en) * | 2015-10-28 | 2016-04-13 | 杭州富居数码科技有限公司 | Building unit machine of talkbacking with night vision function |
| WO2016080003A1 (en) * | 2014-11-20 | 2016-05-26 | シャープ株式会社 | Solid-state imaging element |
| CN107580163A (en) * | 2017-08-12 | 2018-01-12 | 四川精视科技有限公司 | A kind of twin-lens black light camera |
| CN107820066A (en) * | 2017-08-12 | 2018-03-20 | 四川聚强创新科技有限公司 | A kind of low-luminance color video camera |
| CN207283655U (en) * | 2017-08-12 | 2018-04-27 | 四川精视科技有限公司 | A kind of twin-lens black light camera |
| CN207321433U (en) * | 2017-08-12 | 2018-05-04 | 四川精视科技有限公司 | A kind of building visual phone imaging device full-color round the clock |
-
2017
- 2017-08-12 CN CN201710687819.3A patent/CN107623831A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202475619U (en) * | 2012-02-03 | 2012-10-03 | 深圳市保千里电子有限公司 | Night vision system for eliminating stray light |
| KR20140127631A (en) * | 2013-04-25 | 2014-11-04 | 삼성테크윈 주식회사 | Photographing apparatus and method for night photography |
| WO2016080003A1 (en) * | 2014-11-20 | 2016-05-26 | シャープ株式会社 | Solid-state imaging element |
| CN204836402U (en) * | 2015-07-29 | 2015-12-02 | 山东神戎电子股份有限公司 | Laser night -time vision device suitable for small -size aircraft |
| CN205017431U (en) * | 2015-09-24 | 2016-02-03 | 广州市巽腾信息科技有限公司 | Multi -functional removal image processing device |
| CN205160681U (en) * | 2015-10-28 | 2016-04-13 | 杭州富居数码科技有限公司 | Building unit machine of talkbacking with night vision function |
| CN107580163A (en) * | 2017-08-12 | 2018-01-12 | 四川精视科技有限公司 | A kind of twin-lens black light camera |
| CN107820066A (en) * | 2017-08-12 | 2018-03-20 | 四川聚强创新科技有限公司 | A kind of low-luminance color video camera |
| CN207283655U (en) * | 2017-08-12 | 2018-04-27 | 四川精视科技有限公司 | A kind of twin-lens black light camera |
| CN207321433U (en) * | 2017-08-12 | 2018-05-04 | 四川精视科技有限公司 | A kind of building visual phone imaging device full-color round the clock |
Non-Patent Citations (1)
| Title |
|---|
| 吴海兵;陶声祥;顾国华;王书宇;: "基于四波段图像融合的彩色夜视方法研究", 光子学报, no. 05 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110891138A (en) * | 2018-09-10 | 2020-03-17 | 杭州萤石软件有限公司 | Black light full-color realization method and black light full-color camera |
| CN110248105A (en) * | 2018-12-10 | 2019-09-17 | 浙江大华技术股份有限公司 | A kind of image processing method, video camera and computer storage medium |
| WO2020119082A1 (en) * | 2018-12-10 | 2020-06-18 | Zhejiang Dahua Technology Co., Ltd. | Systems and methods for image acquisition |
| CN110248105B (en) * | 2018-12-10 | 2020-12-08 | 浙江大华技术股份有限公司 | Image processing method, camera and computer storage medium |
| US11240443B2 (en) | 2018-12-10 | 2022-02-01 | Zhejiang Dahua Technology Co., Ltd. | Systems and methods for image acquisition |
| CN111679283A (en) * | 2020-06-10 | 2020-09-18 | 南京凯基特电气有限公司 | Multifunctional anti-interference wide-range laser sensor system and use method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107580163A (en) | A kind of twin-lens black light camera | |
| US11877086B2 (en) | Method and system for generating at least one image of a real environment | |
| US20210217212A1 (en) | Method and system for automatically colorizing night-vision images | |
| CN108712608B (en) | Terminal equipment shooting method and device | |
| CN107925751B (en) | System and method for multiple views noise reduction and high dynamic range | |
| CN108154514B (en) | Image processing method, device and equipment | |
| US8111946B2 (en) | Image processing apparatus, imaging apparatus, and image processing method | |
| CN108694709B (en) | Image fusion method and device | |
| WO2012067028A1 (en) | Image input device and image processing device | |
| CN110490811B (en) | Image noise reduction device and image noise reduction method | |
| CN108055452A (en) | Image processing method, device and equipment | |
| CN108269244B (en) | Image defogging system based on deep learning and prior constraint | |
| CN109804619A (en) | Image processing apparatus, image processing method and camera | |
| CN105447838A (en) | Method and system for infrared and low-level-light/visible-light fusion imaging | |
| CN107820066A (en) | A kind of low-luminance color video camera | |
| CN107113408A (en) | Image processing apparatus, image processing method, program and system | |
| CN108024054A (en) | Image processing method, device and equipment | |
| CN105245785A (en) | Brightness balance adjustment method of vehicle panoramic camera | |
| CN107623831A (en) | A kind of building visual phone imaging device full-color round the clock | |
| CN106846258A (en) | A kind of single image to the fog method based on weighted least squares filtering | |
| CN113556526A (en) | RGBW filter array-based color enhancement method for color night vision equipment | |
| CN108093175A (en) | A kind of adaptive defogging method of real-time high-definition video and device | |
| CN114782502A (en) | Multispectral multi-sensor cooperative processing method and device and storage medium | |
| JP2005229317A (en) | Image display system and imaging device | |
| CN113936017A (en) | Image processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |