CN109040720B

CN109040720B - A kind of method and device generating RGB image

Info

Publication number: CN109040720B
Application number: CN201810820523.9A
Authority: CN
Inventors: 程敏
Original assignee: Zhejiang Dahua Technology Co Ltd
Current assignee: Zhejiang Dahua Technology Co Ltd
Priority date: 2018-07-24
Filing date: 2018-07-24
Publication date: 2019-11-19
Anticipated expiration: 2038-07-24
Also published as: CN109040720A

Abstract

The invention discloses a kind of method and devices for generating RGB image, comprising: carries out interpolation processing to the pixel value of the channel IR pixel in the first image of RGBIR sensor output, obtains the second image；The IR ingredient in the first image is removed, third image is obtained；Determine the first average brightness value of the first image and the second average brightness value of the second image；According to the ratio of the first average brightness value and the second average brightness value, Steerable filter intensity is determined, wherein the bigger Steerable filter intensity of ratio is bigger；Steerable filter processing is carried out to third image using Steerable filter intensity, obtains the 4th image；Interpolation processing is carried out to the 4th image, generates RGB image.Since the bigger Steerable filter intensity of the ratio of the first average brightness value and the second average brightness value of the second image is bigger.The image poorer for signal-to-noise ratio, the bigger Steerable filter intensity of use carry out Steerable filter processing to it, meet the requirements the signal-to-noise ratio of all RGB images can.

Description

A kind of method and device generating RGB image

Technical field

The present invention relates to technical field of image processing more particularly to a kind of method and devices for generating RGB image.

Background technique

Conventional image sensor arranges in red R, green G, each channel indigo plant B according to certain format, and each pixel incudes R, G, B In a channel value, then by image signal processing blocks carry out interpolation processing, obtain the RGB image of silent frame；RGBIR Sensor arranges in R, G, B, each channel infrared light IR according to certain format, a channel in each pixel induction R, G, B, IR Then value carries out interpolation processing by image signal processing blocks, obtains the RGB image and IR image of silent frame.RGBIR is passed Sensor has compared more than traditional imaging sensor the channel IR, more functions may be implemented, for example switch without optical filter It realizes and day and night switches, fog penetrating function can be achieved, low-light (level) performance can be improved etc..

RGBIR sensor has compared more than conventional image sensor the channel IR all the way, and R, G, B of RGBIR sensor are logical The pollution of IR ingredient is inevitably mixed in road to need to sense RGBIR to obtain RGB image based on RGBIR sensor IR ingredient removal in R, G of device, channel B.Then interpolation processing is carried out to the image after removal IR ingredient, obtains RGB image.

The noise of image after removing IR ingredient is poor, and the noise for also allowing for finally obtained RGB image is poor. In the prior art in order to improve the signal-to-noise ratio of RGB image, using fixed Steerable filter intensity to the image after removal IR ingredient Steerable filter processing is carried out, to improve the signal-to-noise ratio of RGB image.But IR ingredient is different, the image after removing IR ingredient Signal-to-noise ratio is also different, and wherein IR ingredient is more, and the signal-to-noise ratio of the image after removing IR ingredient is poorer, therefore the prior art can not protect The signal-to-noise ratio for demonstrate,proving all RGB images generated based on RGBIR sensor can be met the requirements.

Summary of the invention

The embodiment of the invention provides a kind of method and devices for generating RGB image, can not protect to solve the prior art The problem of demonstrate,proving the signal-to-noise ratio of all RGB images generated based on RGBIR sensor can meet the requirements.

The embodiment of the invention provides a kind of methods for generating RGB image, which comprises

Interpolation processing is carried out to the pixel value of the channel IR pixel in the first image of RGBIR sensor output, obtains the Two images；The IR ingredient in the first image is removed, third image is obtained；

The first average brightness value of the first image and the second average brightness value of the second image are determined respectively；According to institute State the ratio of the first average brightness value and second average brightness value and pair of preset ratio range and Steerable filter intensity Should be related to, determine Steerable filter intensity comprising the biggish preset ratio range of ratio, corresponding Steerable filter is strong It spends larger；

It is reference with the first image, the third image is carried out at Steerable filter using the Steerable filter intensity Reason, obtains the 4th image；

Respectively in the 4th image the channel R, the channel G, channel B pixel pixel value carry out interpolation processing, really The R component of each channel B pixel and the B component of each channel IR pixel in fixed 4th image；According to the described 4th The R component of the pixel value of each channel G pixel, the pixel value of each channel R pixel, each channel B pixel in image With the B component of each channel IR pixel, RGB image is generated.

Further, the second of the first average brightness value and the second image of the first image determining respectively is average Brightness value includes:

According to the first brightness value of the pixel in the channel R, the channel G and channel B in the first image, determine that first is flat Equal brightness value；

According to the coordinate information of pixel in the first image and second image, determine in second image Pixel corresponding with the channel R in the first image, the pixel in the channel G and channel B, according to the corresponding pixel The second brightness value, determine the second average brightness value.

Further, according to the first brightness value of the pixel in the channel R, the channel G and channel B in the first image, Before determining the first average brightness value, the method also includes:

First brightness value of the pixel in the channel R, the channel G and channel B in identification the first image, it is bright to filter out first Angle value is greater than the pixel of preset first luminance threshold；

According to the first brightness value of the pixel in the channel R, the channel G and channel B in the first image, determine that first is flat Equal brightness value includes:

According to first brightness value in the channel R, the channel G and the remaining pixel of channel B in the first image, is determined One average brightness value.

Further, described before determining the second average brightness value according to the second brightness value of the corresponding pixel Method further include:

The second brightness value for identifying the corresponding pixel filters out the second brightness value greater than preset second luminance threshold Pixel；

According to the second brightness value of the corresponding pixel, determine that the second average brightness value includes:

According to the second brightness value of remaining pixel in the corresponding pixel, the second average brightness value is determined.

Further, it is described respectively in the 4th image the channel R, the channel G, channel B pixel pixel value into Row interpolation processing, determines the B component of the R component of each channel B pixel and each channel IR pixel in the 4th image Include:

For the pixel in each channel IR, according to including each pixel in the first default neighborhood including the pixel Pixel value and the pixel pixel value, determine horizontal edge gradient value and vertical edge gradient value；According to the horizontal sides Edge gradient value and vertical edge gradient value, determine interpolation direction, on the interpolation direction to the pixel value of the channel G pixel into Row interpolation processing, obtains the G component of the pixel；

Determine that the G component of the pixel in each channel B and the channel R, the B component of the pixel in each channel R and each B are logical The R component of the pixel in road；

For the pixel in each channel IR, according to the following formula:

Determine the pixel B component；

In formula, B_B1To B_BnThe pixel value of n channel B pixel in respectively the second default neighborhood；B_R1To B_RnRespectively institute State the B component of the n channel R pixel in the second default neighborhood；G_B1To G_BnN channel B in respectively described second default neighborhood The G component of pixel, G_R1To G_RnThe G component of the n channel R pixel, G in respectively described second default neighborhood_IRFor the pixel The G component of point, B_IRFor the B component of the pixel.

On the other hand, the embodiment of the invention provides a kind of device for generating RGB image, described device includes:

First determining module, in the first image for being exported to RGBIR sensor the pixel value of the channel IR pixel into Row interpolation processing, obtains the second image；The IR ingredient in the first image is removed, third image is obtained；

Second determining module, for determine respectively the first image the first average brightness value and the second image second Average brightness value；According to the ratio and preset ratio range of first average brightness value and second average brightness value With the corresponding relationship of Steerable filter intensity, determine Steerable filter intensity comprising the biggish preset ratio model of ratio It encloses, corresponding Steerable filter intensity is larger；

Module is filtered, for being reference with the first image, using the Steerable filter intensity to the third Image carries out Steerable filter processing, obtains the 4th image；

Generation module, for respectively in the 4th image the channel R, the channel G, channel B pixel pixel value into Row interpolation processing, determines the B component of the R component of each channel B pixel and each channel IR pixel in the 4th image； According to the pixel value of each channel G pixel, the pixel value of each channel R pixel, each channel B picture in the 4th image The B component of the R component of vegetarian refreshments and each channel IR pixel generates RGB image.

Further, second determining module, specifically for according to the channel R, the channel G and the B in the first image First brightness value of the pixel in channel, determines the first average brightness value；According in the first image and second image The coordinate information of pixel determines the picture with the channel R, the channel G and channel B in the first image in second image The corresponding pixel of vegetarian refreshments determines the second average brightness value according to the second brightness value of the corresponding pixel.

Further, described device further include:

First filters out module, for identification the first of the pixel of the channel R in the first image, the channel G and channel B Brightness value filters out the pixel that the first brightness value is greater than preset first luminance threshold；

Second determining module, specifically for according in the first image the channel R, the channel G and channel B it is remaining First brightness value of pixel, determines the first average brightness value.

Further, described device further include:

Second filters out module, for identification the second brightness value of the corresponding pixel, filters out the second brightness value and is greater than The pixel of preset second luminance threshold；

Second determining module, specifically for the second brightness according to remaining pixel in the corresponding pixel Value, determines the second average brightness value.

Further, the generation module, specifically for being directed to the pixel in each channel IR, according to including the pixel The pixel value of the pixel value of each pixel and the pixel in the first default neighborhood inside, determine horizontal edge gradient value and Vertical edge gradient value；According to the horizontal edge gradient value and vertical edge gradient value, interpolation direction is determined, in the interpolation Interpolation processing is carried out to the pixel value of the channel G pixel on direction, obtains the G component of the pixel；Determine each channel B and R The G component of the pixel in channel, each channel R pixel B component and each channel B pixel R component；For every The pixel in a channel IR, according to the following formula:

Determine the pixel B component；

The embodiment of the invention provides a kind of method and devices for generating RGB image, which comprises passes to RGBIR The pixel value of the channel IR pixel carries out interpolation processing in first image of sensor output, obtains the second image；Remove described IR ingredient in one image, obtains third image；The first average brightness value and the second image of the first image are determined respectively The second average brightness value；According to the ratio and preset ratio of first average brightness value and second average brightness value Be worth range and Steerable filter intensity corresponding relationship, determine Steerable filter intensity comprising the biggish preset ratio of ratio It is worth range, corresponding Steerable filter intensity is larger；It is reference with the first image, using the Steerable filter intensity to described Third image carries out Steerable filter processing, obtains the 4th image；Respectively to the channel R, the channel G, channel B in the 4th image The pixel value of pixel carries out interpolation processing, determines that the R component of each channel B pixel and each IR are logical in the 4th image The B component of road pixel；According to the picture of the pixel value of each channel G pixel, each channel R pixel in the 4th image Element value, the B component of the R component of each channel B pixel and each channel IR pixel, generate RGB image.

Due in embodiments of the present invention, being averaged according to the second of the first average brightness value of the first image and the second image The ratio of brightness value and preset ratio range and the corresponding relationship of Steerable filter intensity determine Steerable filter intensity, and Ratio is bigger, and Steerable filter intensity is bigger.Due to the poorer image of signal-to-noise ratio, corresponding average brightness value ratio is bigger, at this In inventive embodiments, the image poorer for signal-to-noise ratio, the bigger Steerable filter intensity of use carries out Steerable filter processing to it, So that the signal-to-noise ratio of all RGB images based on the generation of RGBIR sensor can be met the requirements.

Detailed description of the invention

To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment Attached drawing is briefly introduced, it should be apparent that, drawings in the following description are only some embodiments of the invention, for this For the those of ordinary skill in field, without creative efforts, it can also be obtained according to these attached drawings other Attached drawing.

Fig. 1 is the process schematic for the generation RGB image that the embodiment of the present invention 1 provides；

Fig. 2 is the schematic diagram of the G component of the pixel in the channel determination IR that the embodiment of the present invention 5 provides；

Fig. 3 is the schematic diagram of the G component of the pixel in the channel determination R that the embodiment of the present invention 5 provides；

Fig. 4 is the schematic diagram of the G component of the pixel for the determination channel B that the embodiment of the present invention 5 provides；

Fig. 5 is the schematic diagram of the B component of the pixel in the channel determination R that the embodiment of the present invention 5 provides；

Fig. 6 is the schematic diagram of the R component of the pixel for the determination channel B that the embodiment of the present invention 5 provides；

Fig. 7 is the schematic diagram of the B component of the pixel in the channel determination IR that the embodiment of the present invention 5 provides；

Fig. 8 is the apparatus structure schematic diagram provided in an embodiment of the present invention for generating RGB image.

Specific embodiment

The present invention will be describe below in further detail with reference to the accompanying drawings, it is clear that described embodiment is only this Invention a part of the embodiment, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art exist All other embodiment obtained under the premise of creative work is not made, shall fall within the protection scope of the present invention.

Embodiment 1:

Fig. 1 be it is provided in an embodiment of the present invention generate RGB image process schematic, the process the following steps are included:

S101: interpolation processing is carried out to the pixel value of the channel IR pixel in the first image of RGBIR sensor output, is obtained To the second image；The IR ingredient in the first image is removed, third image is obtained.

It is provided in an embodiment of the present invention generate RGB image method be applied to electronic equipment, the electronic equipment can be PC, The equipment such as tablet computer are also possible to RGBIR sensor.

RGBIR sensor arranges in each channel R, G, B, IR according to certain format, in each pixel induction R, G, B, IR One channel value, RGBIR sensor can acquire RGBIR image, that is, the first image.If electronic equipment is PC, plate Apparatus such as computer, RGBIR sensor is after collecting the first image, by the first image transmitting to electronic equipment, electronic equipment pair First image carries out the treatment process for being subsequently generated RGB image.If electronic equipment is RGBIR sensor, RGBIR sensor After collecting the first image, the treatment process for being subsequently generated RGB image is directly carried out to the first image.

Electronic equipment carries out interpolation processing to the pixel value of each channel IR pixel in the first image, obtains the second figure Picture, wherein the second image includes the channel the IR pixel of silent frame.Wherein, electronic equipment can use single linear interpolation algorithm Interpolation processing is carried out to the pixel value of each channel IR pixel, preferably, the clarity in order to guarantee the second image, can adopt Interpolation processing is carried out with pixel value of the bilinear interpolation algorithm to each channel IR pixel, it is logical to obtain the IR comprising silent frame Second image of road pixel.

Electronic equipment can determine in the first image according to the pixel value of the channel the IR pixel of the silent frame of the second image Each channel R, the channel G and channel B the corresponding IR ingredient of pixel.Specifically, according to the first image and the second figure As in each pixel coordinate information, can determine in the second image with each channel R, the channel G and the B in the first image The pixel value of each corresponding pixel is obtained each R multiplied by preset coefficient by the corresponding pixel of the pixel in channel The corresponding IR ingredient of the pixel in channel, the channel G and channel B.Electronic equipment leads to each channel R in the first image, G Corresponding IR ingredient is individually subtracted in the pixel value of road and the pixel of channel B, obtains third image.

Wherein, the pixel value of the channel IR pixel carries out slotting in the first image that electronic equipment exports RGBIR sensor Value processing, the process for obtaining the second image, the pixel value according to the channel the IR pixel of the silent frame of the second image, determines first The first image of process and removal of the corresponding IR ingredient of pixel in each channel R, the channel G and channel B in image In IR ingredient, the process for obtaining third image belongs to the prior art, no longer repeats herein the above process.

S102: the first average brightness value of the first image and the second average brightness value of the second image are determined respectively； It is strong according to the ratio and preset ratio range of first average brightness value and second average brightness value and Steerable filter The corresponding relationship of degree, determine Steerable filter intensity comprising the biggish preset ratio range of ratio, corresponding guiding Filtering strength is larger.

Electronic equipment can determine the first average brightness value according to the first brightness value of pixel each in the first image. According to the second brightness value of pixel each in the second image, the second average brightness value can be determined.Wherein it is possible to calculate first The average value of first brightness value of the pixel in each channel R, the channel G, channel B and the channel IR in image, this is first bright The average value of angle value is as the first average brightness value.Calculate the second brightness value of the pixel in each channel IR in the second image Average value, using the average value of second brightness value as the second average brightness value.

Electronic equipment calculates the first average brightness value and the after determining the first average brightness value and the second average brightness value The ratio of two average brightness values.The corresponding relationship of preset ratio range Yu Steerable filter intensity is preserved in electronic equipment, In, the biggish preset ratio range of the ratio for including, corresponding Steerable filter intensity is larger.Electronic equipment is calculating first After the ratio of average brightness value and the second average brightness value, ratio range belonging to the ratio can be determined, with the ratio range Corresponding Steerable filter intensity, as determining Steerable filter intensity.

It is illustrated below by a specific example.

For example, the mapping table of preset ratio range Yu Steerable filter intensity is preserved in electronic equipment, the relationship Table is as follows:

Ratio m	Steerable filter intensity
		0 m≤1.15 <	0
1.15 m≤1.25 <	1
		1.25 m≤1.35 <	2
1.35 m≤1.45 <	3
		1.45 m≤1.85 <	6
1.85 < m	8

Such as it is 1.2 that electronic equipment, which calculates the first average brightness value and the ratio of the second average brightness value, it is determined that should Ratio range belonging to ratio is 1.15 m≤1.25 <, and Steerable filter intensity corresponding with the ratio range is 1, it is thus determined that Steerable filter intensity be 1.Such as electronic equipment calculates the ratio of the first average brightness value and the second average brightness value and is 1.5, it is determined that ratio range belonging to the ratio is 1.45 m≤1.85 <, Steerable filter intensity corresponding with the ratio range Be 6, it is thus determined that Steerable filter intensity be 6.

S103: it is reference with the first image, the third image is oriented to using the Steerable filter intensity Filtering processing, obtains the 4th image.

Electronic equipment needs to carry out Steerable filter to third image to improve the signal-to-noise ratio of the RGB image ultimately generated Processing.Specifically, electronic equipment is reference with the first image, third image is led using the Steerable filter intensity determined To filtering processing, the 4th image is obtained.Wherein, Steerable filter processing is carried out by reference pair third image of the first image, obtained The process of 4th image belongs to the prior art, no longer repeats herein the process.

S104: respectively in the 4th image the channel R, the channel G, channel B pixel pixel value carry out interpolation at Reason, determines the B component of the R component of each channel B pixel and each channel IR pixel in the 4th image；According to described The R of the pixel value of each channel G pixel, the pixel value of each channel R pixel, each channel B pixel in 4th image The B component of ingredient and each channel IR pixel generates RGB image.

Electronic equipment is logical to the channel R in the 4th image, G respectively after determining Steerable filter treated the 4th image Road, channel B pixel pixel value carry out interpolation processing, determine in the 4th image R component of each channel B pixel and The B component of each channel IR pixel.Wherein it is possible to be inserted using single linear difference arithmetic, bilinearity difference arithmetic, posterior probability Value-based algorithm, Continuous Approximation interpolation algorithm, median filtering interpolation algorithm etc. are logical to the channel R in the 4th image, the channel G, B respectively The pixel value of road pixel carries out interpolation processing.Also, it for the channel R, the channel G, channel B in the 4th image, can use Identical interpolation algorithm carries out interpolation processing, can also carry out interpolation processing using different interpolation algorithms out.

The pixel value of each channel G pixel in the 4th image that electronic equipment exports, each channel R pixel The B component of pixel value, the R component of each channel B pixel and each channel IR pixel produces RGB image.Wherein, The RGB image is the RGB image of Bayer format.

Due in embodiments of the present invention, being averaged according to the second of the first average brightness value of the first image and the second image The ratio of brightness value and preset ratio range and the corresponding relationship of Steerable filter intensity determine Steerable filter intensity, and The biggish preset ratio range of the ratio for including, corresponding Steerable filter intensity are larger.Due to the poorer image of signal-to-noise ratio, Corresponding average brightness value ratio is bigger, in embodiments of the present invention, the image poorer for signal-to-noise ratio, the bigger guiding of use Filtering strength carries out Steerable filter processing to it, so that the signal-to-noise ratio of all RGB images based on the generation of RGBIR sensor can Enough meet the requirements.

In addition, in embodiments of the present invention, first carrying out Steerable filter processing to third image, obtaining the 4th image, then right 4th image carries out interpolation processing, ultimately generates RGB image, so that the data volume of Steerable filter processing processing is smaller, therefore can To save memory overhead.

Embodiment 2:

In order to reduce the data volume during generating RGB image, on the basis of the above embodiments, in the embodiment of the present invention In, the first average brightness value of the first image determining respectively and the second average brightness value of the second image include:

Since the first brightness value of the pixel in the channel R, the channel G and channel B in the first image schemes the RGB of generation The SNR influence of picture is larger, and the first brightness value of the pixel in the channel IR in the first image is for the RGB image of generation SNR influence very little.In order to reduce the data volume during generating RGB image, it can reduce and determine the first average brightness value Data volume in the process.Due to the channel IR in the first image pixel the first brightness value for generation RGB image SNR influence very little, therefore can not consider the pixel in the channel IR, therefore in embodiments of the present invention, electronic equipment is only counted The average value for calculating the first brightness value of the pixel of the channel R in the first image, the channel G and channel B, by first brightness value Average value is as the first average brightness value.Wherein, the pixel of each channel R in the first image, the channel G and channel B has pair The first brightness value answered, the of the pixel in each channel R, the channel G and channel B in available first image of electronic equipment One brightness value, then calculates the average value of the first brightness value got, which is the first average brightness value.

Electronic equipment identifies the coordinate information of pixel in the first image and the second image, due to the first image and the second figure The resolution ratio of picture is identical, therefore according to the coordinate information of pixel in the first image and the second image, can determine the second figure Pixel corresponding with the channel R in the first image, the pixel in the channel G and channel B as in.Wherein, electronic equipment identifies It is above-mentioned by the coordinate in the second image to after the coordinate information of the pixel in the channel R, the channel G and channel B in the first image The pixel of coordinate information, as pixel corresponding with the channel R in the first image, the pixel in the channel G and channel B respectively Point.Electronic equipment calculates the average value of the second brightness value of the corresponding pixel in the second image, by second brightness value Average value is as the second average brightness value.

For example, the coordinate information of the pixel in some channel R in the first image is (10,5), then in the second image Coordinate information is that the pixel of (10,5) is pixel corresponding with the pixel in the channel R.

Due in embodiments of the present invention, according to biggish first image of the SNR influence of the RGB image for generation First brightness value of the pixel in the middle channel R, the channel G and channel B, determines the first average brightness value, according in the second image with The second brightness value of the corresponding pixel of the pixel in the channel R, the channel G and channel B, determines the second average brightness in first image Value.Without considering the pixel in the channel IR in the first image, therefore reduce the data determined during the first average brightness value Amount, and then reduce the data volume generated during RGB image.

Embodiment 3:

In order to avoid exposing the adverse effect generated for determining the first average brightness value, further make the first average brightness Be worth it is more acurrate, on the basis of the various embodiments described above, in embodiments of the present invention, according to the channel R in the first image, G First brightness value of the pixel of channel and channel B, before determining the first average brightness value, the method also includes:

First brightness value of the pixel in each channel R, the channel G and channel B in electronic equipment the first image of identification, electricity Preset first luminance threshold is preserved in sub- equipment, wherein preset first luminance threshold is biggish brightness value, such as 240,245 etc..Electronic equipment for the pixel in each channel R, the channel G and channel B in the first image the first brightness value, Judge whether first brightness value is greater than preset first luminance threshold, if it is, illustrating the corresponding picture of the first brightness value Vegetarian refreshments is overexposure pixel, filters out the pixel.Then each channel R, the channel G and the channel B calculated in the first image is remaining Pixel the first brightness value average value, obtain the first average brightness value.

For example, preset first luminance threshold that electronic equipment saves is 240, if electronic equipment recognizes some R and leads to First brightness value of the pixel in road is 250, then illustrates that the pixel in the channel R is overexposure pixel, filter out the picture in the channel R Vegetarian refreshments.If the first brightness value that electronic equipment recognizes the pixel of some channel B is 180, illustrate the pixel of the channel B Point is not overexposure pixel, retains the pixel of the channel B.It is finally surplus according to the channel R, the channel G and the channel B in the first image First brightness value of remaining pixel, determines the first average brightness value.

Since in embodiments of the present invention, electronic equipment filters out the picture that the first brightness value is greater than preset first luminance threshold Vegetarian refreshments determines that first is average according to first brightness value in the channel R, the channel G and the remaining pixel of channel B in the first image Brightness value.Therefore the adverse effect that can be generated to avoid exposure for determining the first average brightness value, further makes first to be averaged Brightness value is more acurrate.

Embodiment 4:

In order to avoid exposing the adverse effect generated for determining the second average brightness value, further make the second average brightness Be worth it is more acurrate, it is in embodiments of the present invention, bright according to the second of the corresponding pixel on the basis of the various embodiments described above Angle value, before determining the second average brightness value, the method also includes:

Electronic equipment identifies the pixel pair with the channel R each in the first image, the channel G and channel B in the second image Second brightness value of the pixel answered preserves preset second luminance threshold in electronic equipment, wherein preset second brightness Threshold value is biggish brightness value, and preset second luminance threshold can be identical or different with preset first luminance threshold. Electronic equipment judges whether second brightness value is greater than for the second brightness value of the corresponding pixel of each of the second image It is right to filter out this if it is, illustrating that the corresponding pixel of the second brightness value is overexposure pixel for preset second luminance threshold The pixel answered.Then being averaged for the second brightness value of remaining pixel in the corresponding pixel in the second image is calculated Value, obtains the second average brightness value.

For example, preset second luminance threshold that electronic equipment saves is 245, if electronic equipment recognizes the second image In some corresponding pixel the second brightness value be 250, then illustrate the corresponding pixel be overexposure pixel, it is right to filter out this The pixel answered.If the second brightness value that electronic equipment recognizes some corresponding pixel is 240, illustrate that this is corresponding Pixel is not overexposure pixel, retains the corresponding pixel.Finally according to pixel remaining in corresponding pixel Second brightness value determines the second average brightness value.

Since in embodiments of the present invention, electronic equipment filters out pair that the second brightness value is greater than preset second luminance threshold The pixel answered determines the second average brightness value according to the second brightness value of pixel remaining in corresponding pixel.Therefore The adverse effect that can be generated to avoid exposure for determining the second average brightness value, further makes the second average brightness value more quasi- Really.

Embodiment 5:

In order to keep the clarity of the RGB image generated higher, need accurately to determine each channel B pixel in the 4th image The R component of point and the B component of each channel IR pixel.In order to make the R of each channel B pixel in the 4th determining image at Divide and the B component of each channel IR pixel is more acurrate, on the basis of the various embodiments described above, in embodiments of the present invention, institute State respectively in the 4th image the channel R, the channel G, channel B pixel pixel value carry out interpolation processing, determine described in The R component of each channel B pixel and the B component of each channel IR pixel include: in 4th image

For the pixel in each channel IR, according to the following formula:

Determine the pixel B component；

It is first in order to determine the B component of the R component of each channel B pixel and each channel IR pixel in the 4th image It is first it needs to be determined that the pixel in each channel IR, channel B and the channel R G component, in order to make generate RGB image clarity It is higher, in embodiments of the present invention, interpolation processing is carried out using pixel value of the edge guiding interpolation algorithm to the channel G pixel, Obtain the G component of the pixel in each channel IR, channel B and the channel R.

Specifically, being directed to the pixel in each channel IR, determine comprising every in the first default neighborhood including the pixel A pixel, preferably, each pixel in the first default neighborhood centered on the pixel can be determined.First default neighbour Domain can be the neighborhood of 5*5, the neighborhood of 5*7, neighborhood of 7*7 etc..It is default according to first for the pixel in each channel IR The pixel value of the pixel of the pixel value and channel IR of the pixel in each channel G and the channel IR, determines horizontal sides in neighborhood Edge gradient value and vertical edge gradient value.According to the size relation of the horizontal edge gradient value and vertical edge gradient value, really Determine interpolation direction.Wherein, the biggish direction of edge gradient value is interpolation direction.I.e. horizontal edge gradient value is greater than vertical edge When gradient value, interpolation direction is horizontal direction；When horizontal edge gradient value is less than vertical edge gradient value, interpolation direction is vertical Direction.Interpolation processing is carried out to the pixel value of the channel G pixel on determining interpolation direction, obtains the channel the IR pixel G component.

It is illustrated below by specific example.

Fig. 2 is the schematic diagram for determining the G component of pixel in the channel IR, and the first default neighborhood shown in Fig. 2 is logical with IR The neighborhood of 5*5 centered on the pixel in road.G, IR in Fig. 2 are respectively gap marker, and the number in Fig. 2 is respectively channel sequence Number mark, the channel of the blank in Fig. 2 is the channel R or channel B.Based on Fig. 2, it is thus necessary to determine that the G component of the pixel at IR7 G7。

Horizontal edge gradient value and vertical edge gradient value are determined according to fig. 2, specifically, horizontal edge gradient value Hg7=| G6-G8|+(|2*G3-G16-G17|+|2G11-G18-G19|)/4+|2IR7-IR5-IR9|；Vertical edge gradient value Vg7=| G3-G11|+(|2*G6-G14-G20|+|2G8-G15-G21|)/4+|2IR7-IR1-IR13|.It is closed according to the size of Hg7 and Vg7 It is to determine interpolation direction, if Hg7 is greater than Vg7, interpolation direction is horizontal direction, in the horizontal direction to the channel G pixel Pixel value carry out interpolation processing, obtain the pixel G component be G7=(G3+G11)/2+ (2*IR7-IR1-IR13)/4. If Hg7 is less than Vg7, interpolation direction is vertical direction, carries out interpolation to the pixel value of the channel G pixel in vertical direction Processing, the G component for obtaining the pixel is G7=(G6+G8)/2+ (2*IR7-IR5-IR9)/4.And if Hg7 is equal to Vg7, G7=(G6+G8+G3+G11)/4+ (4*IR7-IR5-IR9-IR1-IR13)/8.

Electronic equipment determines the G component of the pixel in each channel B and the channel R, and uses edge guiding interpolation algorithm Interpolation processing is carried out to the pixel value of the channel G pixel, obtains the G component of the pixel in each channel B and the channel R.Wherein, it adopts Interpolation processing is carried out with pixel value of the edge guiding interpolation algorithm to the channel G pixel, obtains the picture of each channel B and the channel R The process of the G component of vegetarian refreshments belongs to the prior art, is only illustrated herein by specific example.

Fig. 3 is the schematic diagram for determining the G component of pixel in the channel R, and the first default neighborhood shown in Fig. 3 is with the channel R Pixel centered on 5*5 neighborhood.G, R in Fig. 3 are respectively gap marker, and the number in Fig. 3 is respectively channel position It identifies, the channel of the blank in Fig. 3 is the channel IR or channel B.Based on Fig. 3, it is thus necessary to determine that the G component G7 of the pixel at R7.

Horizontal edge gradient value and vertical edge gradient value are determined according to Fig. 3, specifically, horizontal edge gradient value Hg7=| G6-G8|+(|2*G3-G16-G17|+|2G11-G18-G19|)/4+|2R7-G6-G8|；Vertical edge gradient value Vg7=| G3- G11|+(|2*G6-G14-G20|+|2G8-G15-G21|)/4+|2R7-G3-G11|.It is determined according to the size relation of Hg7 and Vg7 Interpolation direction, if Hg7 is greater than Vg7, interpolation direction is horizontal direction, in the horizontal direction to the pixel of the channel G pixel Value carry out interpolation processing, obtain the pixel G component be G7=(G3+G11)/2+ (R7- (R22+R23+R24+R25)/4)/ 2.If Hg7 is less than Vg7, interpolation direction is vertical direction, is carried out in vertical direction to the pixel value of the channel G pixel slotting Value processing, the G component for obtaining the pixel is G7=(G6+G8)/2+ (R7- (R22+R23+R24+R25)/4)/2.And if Hg7 is equal to Vg7, then G7=(G6+G8+G3+G11)/4+ (R7- (R22+R23+R24+R25)/4)/2.

Fig. 4 is the schematic diagram for determining the G component of pixel of channel B, and the first default neighborhood shown in Fig. 4 is with channel B Pixel centered on 5*5 neighborhood.G, B in Fig. 4 are respectively gap marker, and the number in Fig. 4 is respectively channel position It identifies, the channel of the blank in Fig. 4 is the channel IR or the channel R.Based on Fig. 4, it is thus necessary to determine that the G component G7 of the pixel at B7.

Horizontal edge gradient value and vertical edge gradient value are determined according to Fig. 4, specifically, horizontal edge gradient value Hg7=| G6-G8|+(|2*G3-G16-G17|+|2G11-G18-G19|)/4+|2B7-G6-G8|；Vertical edge gradient value Vg7=| G3- G11|+(|2*G6-G14-G20|+|2G8-G15-G21|)/4+|2B7-G3-G11|.It is determined according to the size relation of Hg7 and Vg7 Interpolation direction, if Hg7 is greater than Vg7, interpolation direction is horizontal direction, in the horizontal direction to the pixel of the channel G pixel Value carry out interpolation processing, obtain the pixel G component be G7=(G3+G11)/2+ (B7- (B22+B23+B24+B25)/4)/ 2.If Hg7 is less than Vg7, interpolation direction is vertical direction, is carried out in vertical direction to the pixel value of the channel G pixel slotting Value processing, the G component for obtaining the pixel is G7=(G6+G8)/2+ (B7- (B22+B23+B24+B25)/4)/2.And if Hg7 is equal to Vg7, then G7=(G6+G8+G3+G11)/4+ (B7- (B22+B23+B24+B25)/4)/2.

In turn, it in order to keep the clarity of the RGB image generated higher, in embodiments of the present invention, is inserted using color difference is constant Value-based algorithm carries out interpolation processing to the pixel value of channel B pixel, the B component of the pixel in each channel R is obtained, using color difference Constant interpolation algorithm carries out interpolation processing to the pixel value of the channel R pixel, obtains the R component of the pixel of each channel B.

Wherein, interpolation processing is carried out using pixel value of the constant interpolation algorithm of color difference to channel B pixel, obtains each R The B component of the pixel in channel, and interpolation processing is carried out to the pixel value of the channel R pixel, obtain the pixel of each channel B The process of the R component of point belongs to the prior art, is only illustrated herein by specific example.

Fig. 5 is the schematic diagram for determining the B component of pixel in the channel R, and default neighborhood shown in fig. 5 is with the picture in the channel R The neighborhood of 5*5 centered on vegetarian refreshments.R, B in Fig. 5 are respectively gap marker, and the number in Fig. 5 is respectively channel position mark, The channel of blank in Fig. 5 is the channel IR or the channel G or the channel R.Determine that the B component B4 of the pixel at R4 is B4 based on Fig. 5 =(B0+B1+B2+B3-G0-G1-G2-G3)/4+G4.

Fig. 6 is the schematic diagram for determining the R component of pixel of channel B, and default neighborhood shown in fig. 6 is with the picture of channel B The neighborhood of 5*5 centered on vegetarian refreshments.R, B in Fig. 6 are respectively gap marker, and the number in Fig. 6 is respectively channel position mark, The channel of blank in Fig. 6 is the channel IR or the channel G or channel B.Determine that the R component R4 of the pixel at B4 is R4 based on Fig. 6 =(R0+R1+R2+R3-G0-G1-G2-G3)/4+G4.

In turn, the B component of the pixel in each channel IR is obtained using the constant interpolation algorithm of color difference, specifically, for every The pixel in a channel IR, according to the following formula:

Determine the pixel B component；

Fig. 7 is the schematic diagram for determining the B component of pixel in the channel IR, and the second default neighborhood shown in Fig. 7 is logical with IR The neighborhood of 3*3 centered on the pixel in road.Include 2 channel Bs, respectively B2 and B6 in Fig. 7, includes 4 channels G, respectively For G1, G3, G5 and G7, B2 and B6 in Fig. 7 are respectively the B component in the channel R2 and the channel R6 pixel.IR4 is determined based on Fig. 7 The B component B4 of the pixel at place is B4=(B0+B2+B6+B8-G0-G2-G6-G8)/4+G4.

Finally, the pixel value of each channel G pixel, each channel R picture in the 4th image that electronic equipment exports The B component of the pixel value of vegetarian refreshments, the R component of each channel B pixel and each channel IR pixel produces RGB image.

Due in embodiments of the present invention, being carried out using pixel value of the edge guiding interpolation algorithm to the channel G pixel slotting Value processing, obtains the G component of the pixel in each channel IR, channel B and the channel R, using the constant interpolation algorithm of color difference to channel B The pixel value of pixel carries out interpolation processing, obtains the B component of the pixel in each channel R, and the picture to the channel R pixel Element value carries out interpolation processing, obtains the R component of the pixel of each channel B, obtains each IR using the constant interpolation algorithm of color difference The B component of the pixel in channel.So that the clarity of the RGB image ultimately generated is higher.

Fig. 8 is the apparatus structure schematic diagram provided in an embodiment of the present invention for generating RGB image, which includes:

First determining module 81, the pixel value of the channel IR pixel in the first image for being exported to RGBIR sensor Interpolation processing is carried out, the second image is obtained；The IR ingredient in the first image is removed, third image is obtained；

Second determining module 82, for determine respectively the first image the first average brightness value and the second image Two average brightness values；According to the ratio and preset ratio model of first average brightness value and second average brightness value Enclose the corresponding relationship with Steerable filter intensity, determine Steerable filter intensity comprising the biggish preset ratio model of ratio It encloses, corresponding Steerable filter intensity is larger；

Module 83 is filtered, for being reference with the first image, using the Steerable filter intensity to described the Three images carry out Steerable filter processing, obtain the 4th image；

Generation module 84, for respectively in the 4th image the channel R, the channel G, channel B pixel pixel value Carry out interpolation processing, determine in the 4th image B of the R component of each channel B pixel and each channel IR pixel at Point；It is logical according to the pixel value of each channel G pixel, the pixel value of each channel R pixel, each B in the 4th image The B component of the R component of road pixel and each channel IR pixel generates RGB image.

Second determining module 82, specifically for the picture according to the channel R, the channel G and channel B in the first image First brightness value of vegetarian refreshments, determines the first average brightness value；According to pixel in the first image and second image Coordinate information determines corresponding with the pixel in the channel R, the channel G and channel B in the first image in second image Pixel the second average brightness value is determined according to the second brightness value of the corresponding pixel.

Described device further include:

First filters out module 85, for identification the of the pixel of the channel R in the first image, the channel G and channel B One brightness value filters out the pixel that the first brightness value is greater than preset first luminance threshold；

Second determining module 82, specifically for remaining according to the channel R, the channel G and the channel B in the first image Pixel the first brightness value, determine the first average brightness value.

Described device further include:

Second filters out module 86, and the second brightness value of the corresponding pixel, it is big to filter out the second brightness value for identification In the pixel of preset second luminance threshold；

Second determining module 82, specifically for bright according to second of remaining pixel in the corresponding pixel Angle value determines the second average brightness value.

The generation module 84, specifically for being directed to the pixel in each channel IR, including comprising the pixel The pixel value of the pixel value of each pixel and the pixel in first default neighborhood, determines horizontal edge gradient value and vertical edges Edge gradient value；According to the horizontal edge gradient value and vertical edge gradient value, interpolation direction is determined, on the interpolation direction Interpolation processing is carried out to the pixel value of the channel G pixel, obtains the G component of the pixel；Determine each channel B and the channel R The G component of pixel, each channel R pixel B component and each channel B pixel R component；It is logical for each IR The pixel in road, according to the following formula:

Determine the pixel B component；

The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates, Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or The function of being specified in multiple boxes.

These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one The step of function of being specified in a box or multiple boxes.

Although preferred embodiments of the present invention have been described, it is created once a person skilled in the art knows basic Property concept, then additional changes and modifications may be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as It selects embodiment and falls into all change and modification of the scope of the invention.

Obviously, various changes and modifications can be made to the invention without departing from essence of the invention by those skilled in the art Mind and range.In this way, if these modifications and changes of the present invention belongs to the range of the claims in the present invention and its equivalent technologies Within, then the present invention is also intended to include these modifications and variations.

Claims

1. a kind of method for generating RGB image, which is characterized in that the described method includes:

According to the pixel value of the channel IR pixel in the first image of RGBIR sensor output, to R, G, B in the first image Channel pixel carries out interpolation processing, obtains the second image；The IR ingredient for removing R, G in the first image, channel B pixel, obtains To third image；

The first average brightness value of the first image and the second average brightness value of the second image are determined respectively；According to described The ratio and preset ratio range of one average brightness value and second average brightness value and the corresponding of Steerable filter intensity are closed System, determine Steerable filter intensity comprising the biggish preset ratio range of ratio, corresponding Steerable filter intensity compared with Greatly；

It is reference with the first image, Steerable filter processing is carried out to the third image using the Steerable filter intensity, Obtain the 4th image；

Respectively in the 4th image the channel R, the channel G, channel B pixel pixel value carry out interpolation processing, determine institute State the B component of the R component of each channel B pixel and each channel IR pixel in the 4th image；According to the 4th image In the pixel value of each channel G pixel, the pixel value of each channel R pixel, the R component of each channel B pixel and every The B component of a channel IR pixel generates RGB image.

2. the method as described in claim 1, which is characterized in that first average brightness for determining the first image respectively Second average brightness value of value and the second image includes:

According to the first brightness value of the pixel in the channel R, the channel G and channel B in the first image, determine that first is average bright Angle value；

According to the coordinate information of pixel in the first image and second image, determine in second image with institute The corresponding pixel of pixel for stating the channel R in the first image, the channel G and channel B, according to the of the corresponding pixel Two brightness values determine the second average brightness value.

3. method according to claim 2, which is characterized in that according to the channel R, the channel G and the channel B in the first image Pixel the first brightness value, before determining the first average brightness value, the method also includes:

First brightness value of the pixel in the channel R, the channel G and channel B in identification the first image, filters out the first brightness value Greater than the pixel of preset first luminance threshold；

According to the first brightness value of the pixel in the channel R, the channel G and channel B in the first image, determine that first is average bright Angle value includes:

According to first brightness value in the channel R, the channel G and the remaining pixel of channel B in the first image, determine that first is flat Equal brightness value.

4. method according to claim 2, which is characterized in that according to the second brightness value of the corresponding pixel, determine Before second average brightness value, the method also includes:

The second brightness value for identifying the corresponding pixel filters out the picture that the second brightness value is greater than preset second luminance threshold Vegetarian refreshments；

5. the method as described in claim 1, which is characterized in that it is described respectively in the 4th image the channel R, the channel G, The pixel value of channel B pixel carries out interpolation processing, determines in the 4th image R component of each channel B pixel and every The B component of a channel IR pixel includes:

For the pixel in each channel IR, according to the picture comprising each pixel in the first default neighborhood including the pixel The pixel value of element value and the pixel, determines horizontal edge gradient value and vertical edge gradient value；According to the horizontal edge ladder Angle value and vertical edge gradient value, determine interpolation direction, carry out on the interpolation direction to the pixel value of the channel G pixel slotting Value processing, obtains the G component of the pixel；

Determine the G component of the pixel in each channel B and the channel R, the B component of the pixel in each channel R and each channel B The R component of pixel；

For the pixel in each channel IR, according to the following formula:

Determine the B of the pixel at Point；

In formula, B_B1To B_BnThe pixel value of n channel B pixel in respectively the second default neighborhood；B_R1To B_RnRespectively described The B component of the n channel R pixel in two default neighborhoods；G_B1To G_BnN channel B pixel in respectively described second default neighborhood The G component of point, G_R1To G_RnThe G component of the n channel R pixel, G in respectively described second default neighborhood_IRFor the pixel G component, B_IRFor the B component of the pixel.

6. a kind of device for generating RGB image, which is characterized in that described device includes:

First determining module, the pixel value of the channel IR pixel in the first image for being exported according to RGBIR sensor, to institute R, G in the first image, channel B pixel progress interpolation processing are stated, the second image is obtained；It is logical to remove R, G, B in the first image The IR ingredient of road pixel, obtains third image；

Second determining module, for determine respectively the first image the first average brightness value and the second image it is second average Brightness value；According to the ratio of first average brightness value and second average brightness value and preset ratio range with lead To the corresponding relationship of filtering strength, determine Steerable filter intensity comprising the biggish preset ratio range of ratio, it is right The Steerable filter intensity answered is larger；

Module is filtered, for being reference with the first image, using the Steerable filter intensity to the third image Steerable filter processing is carried out, the 4th image is obtained；

Generation module, for respectively in the 4th image the channel R, the channel G, channel B pixel pixel value carry out it is slotting Value processing, determines the B component of the R component of each channel B pixel and each channel IR pixel in the 4th image；According to The pixel value of each channel G pixel, the pixel value of each channel R pixel, each channel B pixel in 4th image R component and each channel IR pixel B component, generate RGB image.

7. device as claimed in claim 6, which is characterized in that second determining module is specifically used for according to described first First brightness value of the pixel in the channel R, the channel G and channel B in image, determines the first average brightness value；According to described The coordinate information of pixel in one image and second image, determine in second image in the first image The corresponding pixel of the pixel in the channel R, the channel G and channel B is determined according to the second brightness value of the corresponding pixel Second average brightness value.

8. device as claimed in claim 7, which is characterized in that described device further include:

First filters out module, for identification the first brightness of the pixel of the channel R in the first image, the channel G and channel B Value filters out the pixel that the first brightness value is greater than preset first luminance threshold；

Second determining module, specifically for according to the channel R, the channel G and the remaining pixel of channel B in the first image First brightness value of point, determines the first average brightness value.

9. device as claimed in claim 7, which is characterized in that described device further include:

Second filters out module, for identification the second brightness value of the corresponding pixel, filters out the second brightness value greater than default The second luminance threshold pixel；

Second determining module, specifically for the second brightness value according to remaining pixel in the corresponding pixel, Determine the second average brightness value.

10. device as claimed in claim 6, which is characterized in that the generation module, specifically for for each channel IR Pixel, according to the pixel of the pixel value comprising each pixel in the first default neighborhood including the pixel and the pixel Value, determines horizontal edge gradient value and vertical edge gradient value；According to the horizontal edge gradient value and vertical edge gradient value, It determines interpolation direction, interpolation processing is carried out to the pixel value of the channel G pixel on the interpolation direction, obtains the pixel G component；Determine the B component and each channel B of the G component of the pixel in each channel B and the channel R, the pixel in each channel R Pixel R component；For the pixel in each channel IR, according to the following formula:

Determine the B of the pixel at Point；