CN110493583A

CN110493583A - Image processing method, device, electronic equipment and computer readable storage medium

Info

Publication number: CN110493583A
Application number: CN201810875810.XA
Authority: CN
Inventors: 范蒙; 俞海; 浦世亮
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2018-08-03
Filing date: 2018-08-03
Publication date: 2019-11-22
Anticipated expiration: 2038-08-03
Also published as: CN114422766A; CN110493583B

Abstract

The embodiment of the invention provides a kind of image processing method, device, electronic equipment and computer readable storage medium, this programme can obtain the raw image data of imaging sensor acquisition；Wherein, the raw image data includes at least original all-pass channel data, original first color channel data and original second color channel data, any channel data all has empty pixel, and first color channel and second color channel are the two of them in R, G, B color channel；Based on the raw image data, the all-pass channel image of full resolution and the RGB image of full resolution are generated；The all-pass channel image is merged with the RGB image, obtains fused image.Using scheme provided in an embodiment of the present invention, the preferable clear image of quality can be collected.

Description

Image processing method, device, electronic equipment and computer readable storage medium

Technical field

The present invention relates to image technique fields, more particularly to a kind of image processing method, device, electronic equipment and calculating Machine readable storage medium storing program for executing.

Background technique

In the prior art, image capture device acquires image often through imaging sensor.It is, for example, possible to use Bayers Visible images of the imaging sensor acquisition comprising three kinds of color channels in rgb color channel of format.However, environment can It in the light-exposed lower situation of intensity of illumination, is limited by sensor photosensitive performance, this mode visible images collected are made an uproar Sound is larger, not enough clearly.

Summary of the invention

The embodiment of the present invention is designed to provide a kind of image processing method, device, electronic equipment and computer-readable Storage medium, to collect the preferable clear image of quality.Specific technical solution is as follows:

In a first aspect, the embodiment of the invention provides a kind of image processing methods, which comprises

Obtain the raw image data of imaging sensor acquisition；Wherein, the raw image data includes at least original complete Track data, original first color channel data and original second color channel data, any channel data all have cavity all Pixel, first color channel and second color channel are the two of them in R, G, B color channel；

Based on the raw image data, the all-pass channel image of full resolution and the RGB image of full resolution are generated；

The all-pass channel image is merged with the RGB image, obtains fused image.

Optionally, described the step of being based on the raw image data, generating the all-pass channel image of full resolution, packet It includes:

Interpolation processing is carried out to the original all-pass channel data, obtains the all-pass channel image of the full resolution.

Optionally, described that interpolation processing is carried out to the original all-pass channel data, obtain the all-pass of the full resolution The step of channel image, comprising:

Interpolation interpolation processing is carried out to the original all-pass channel data, is determined each in the original all-pass track data Channel value at a cavity pixel, and then obtain the all-pass channel image of the full resolution.

Optionally, described that interpolation processing is carried out to the original all-pass channel data, obtain the all-pass of the full resolution Channel image includes:

According to original first color channel data and original second color channel data, the first weight is determined；

According to the original all-pass channel data and first weight, interpolation is carried out to the original all-pass channel data Processing determines the channel value at each empty pixel in the original all-pass track data, and then obtains the full resolution The all-pass channel image of rate.

Optionally, described the step of being based on the raw image data, generating the RGB image of full resolution, comprising:

Interpolation processing is carried out to original first color channel data, obtains the first color channel figure of full resolution Picture；

Interpolation processing is carried out to original second color channel data, obtains the second color channel figure of full resolution Picture；

According to the all-pass channel image, first color channel images and second color channel images, determine The third color channel image of full resolution, the third color channel are that first color is removed in R, G, B color channel A color channel except color channel and second color channel；

According to first color channel images, second color channel images and the third color channel image, Obtain the RGB image of the full resolution.

Optionally, described that interpolation processing is carried out to original first color channel data, obtain the first of full resolution The step of color channel images, comprising:

Interpolation interpolation processing is carried out to original first color channel data, determines original first color channel The channel value at each empty pixel in data, and then obtain the first color channel images of the full resolution；Alternatively,

According to the original all-pass channel data and original second color channel data, the second weight is determined；According to Second weight and original first color channel data carry out at interpolation original first color channel data Reason is determined the channel value at each empty pixel in original first color channel data, and then is obtained described complete First color channel images of resolution ratio.

Optionally, described that interpolation processing is carried out to original second color channel data, obtain the second of full resolution The step of color channel images, comprising:

Interpolation interpolation processing is carried out to original second color channel data, determines original second color channel The channel value at each empty pixel in data, and then obtain the second color channel images of the full resolution；Alternatively,

According to the original all-pass channel data and original first color channel data, third weight is determined, according to The third weight and original second color channel data carry out at interpolation original second color channel data Reason is determined the channel value at each empty pixel in original second color channel data, and then is obtained described complete Second color channel images of resolution ratio.

Optionally, described logical according to the all-pass channel image, first color channel images and second color Road image, the step of determining the third color channel image of full resolution, comprising:

It determines the channel value of any pixel point in the third color channel image of the full resolution, and then is differentiated entirely The third color channel image of rate；Wherein, the channel value of any pixel point is according to the all-pass channel image, described In first color channel images and second color channel images with any pixel point same position pixel What channel value determined.

Optionally, the channel value of any pixel point is determined by following formula in the third color channel image:

S₁=[C₁-(S₂+S₃-α)*K₁+β]*K₂；

In formula, S₁Indicate the channel value of any pixel point in the third color channel image, C₁、S₂And S₃It respectively corresponds Indicate in the all-pass channel image, first color channel images and second color channel images with any picture Channel value of the vegetarian refreshments in the pixel of same position, α, β, K₁And K₂It is preset constant value.

According to the raw image data, third color channel data is determined, the third color channel is described R, G, B A color channel in color channel in addition to first color channel and second color channel；

To original first color channel data, original second color channel data and the third color channel Data carry out interpolation processing respectively, obtain the RGB image of the full resolution.

Optionally, described according to the raw image data, the step of determining third color channel data, comprising:

By the non-empty pixel in the all-pass channel data, the non-empty being determined as in the third color channel data Hole pixel；

It determines the channel value of any non-empty pixel in the third color channel data, and then obtains third color Channel data；Wherein, the channel value of any non-empty pixel in the third color channel data, is according to described original The channel value of pixel in all-pass channel data with any non-empty pixel in same position, original first color In color channel data with any non-empty pixel in the corresponding default field of pixel of same position each pixel In the average channel value of point and second color channel data with any non-empty pixel same position picture The average channel value of each pixel determines in the corresponding default field of vegetarian refreshments.

Optionally, the channel value of any non-empty pixel in the third color channel data is true by following formula It is fixed:

S₄=[C₂-(S₅+S₆-α)*K₁+β]*K₂；

In formula, S₄Indicate the channel value of any non-empty pixel in the third color channel data, C₂Described in expression The channel value of pixel in original all-pass channel data with any non-empty pixel in same position, S₅Described in expression With any non-empty pixel in the corresponding default field of pixel of same position in original first color channel data In each pixel average channel value, S₆It indicates to exist in second color channel data with any non-empty pixel The average channel value of each pixel in the corresponding default field of the pixel of same position；α,β,K₁And K₂It is preset Constant value.

Optionally, described to original first color channel data, original second color channel data and described Third color channel data carries out interpolation processing respectively, obtains the RGB image of the full resolution, comprising:

Interpolation processing is carried out to original first color channel data, determines original first color channel data In each empty pixel at channel value, and then obtain the first color channel images of full resolution；

Interpolation processing is carried out to original second color channel data, determines original second color channel data In each empty pixel at channel value, and then obtain the second color channel images of full resolution；

Interpolation processing is carried out to the third color channel data, is determined each in the third color channel data Channel value at empty pixel, and then obtain the third color channel image of full resolution；

It is optionally, described that interpolation processing is carried out to original first color channel data, comprising:

Interpolation interpolation processing is carried out to original first color channel data；

Alternatively, according to the original all-pass channel data, original second color channel data and the third color Channel data determines the 4th weight；According to the 4th weight and original first color channel data, to described original One color channel data carries out interpolation processing；

It is described that interpolation processing is carried out to original second color channel data, comprising:

Interpolation interpolation processing is carried out to default second color channel data；

Alternatively, according to the original all-pass channel data, original first color channel data and the third color Channel data determines the 5th weight；According to the 5th weight and original second color channel data, to described original Two color channel datas carry out interpolation processing；

It is described that interpolation processing is carried out to the third color channel data, comprising:

Interpolation interpolation processing is carried out to the third color channel data；

Alternatively, according to the original all-pass channel data, original first color channel data and described original second Color channel data determines the 6th weight；According to the 6th weight and the third color channel data, to the third color Color channel data carries out interpolation processing.

Optionally, described to merge the all-pass channel image with the RGB image, obtain fused image Step, comprising:

Obtain the color information in the RGB image；

Obtain the second luminance information in the first luminance information and the all-pass channel image in the RGB image；

The color information, first luminance information and second luminance information are merged, described melt is obtained Image after conjunction.

Second aspect, the embodiment of the invention provides a kind of image processing apparatus, described device includes:

Module is obtained, for obtaining the raw image data of image capture device acquisition；Wherein, the raw image data Including at least original all-pass channel data, original first color channel data and original second color channel data, any channel Data all have empty pixel, and first color channel and second color channel are its in R, G, B color channel In two kinds；

Generation module generates the all-pass channel image and full resolution of full resolution for being based on the raw image data The RGB image of rate；

Fusion Module obtains fused figure for merging the all-pass channel image with the RGB image Picture.

Optionally, the generation module, comprising:

All-pass channel image generates submodule, for carrying out interpolation processing to the original all-pass channel data, obtains institute State the all-pass channel image of full resolution.

Optionally, the all-pass channel image generates submodule, is specifically used for:

Optionally, the generation module, comprising: the first RGB image generates submodule, is used for:

Optionally, first RGB image generates submodule and carries out at interpolation to original first color channel data Reason, obtains the first color channel images of full resolution, specifically:

Optionally, first RGB image generates submodule and carries out at interpolation to original second color channel data Reason, obtains the second color channel images of full resolution, specifically:

Optionally, first RGB image generates submodule according to the all-pass channel image, first color channel Image and second color channel images, determine the third color channel image of full resolution, specifically:

Optionally, the channel value of any pixel point in third color channel image is determined by following formula:

S₁=[C₁-(S₂+S₃-α)*K₁+β]*K₂；

Optionally, the generation module, comprising: the second RGB image generates submodule, is used for:

Optionally, described second RGB image generates submodule according to the raw image data, determines that third color is logical Track data specifically includes:

It determines the channel value of any non-empty pixel in the third color channel data, and then obtains the third Color channel data, wherein the channel value of any non-empty pixel in the third color channel data, is basis Channel value, the original of pixel in the original all-pass channel data with any non-empty pixel in same position Begin in the first color channel data with any non-empty pixel in the corresponding default field of pixel of same position With any non-empty pixel same in the average channel value of each pixel and second color channel data The average channel value of each pixel determines in the corresponding default field of the pixel of position.

Optionally, the channel value of any non-empty pixel in third color channel data is determined by following formula:

S₄=[C₂-(S₅+S₆-α)*K₁+β]*K₂；

Optionally, second RGB image generates submodule to original first color channel data, described original the Two color channel datas and the third color channel data carry out interpolation processing respectively, obtain the RGB figure of the full resolution Picture, specifically:

Optionally, second RGB image generates submodule and carries out at interpolation to original first color channel data Reason specifically includes:

Interpolation interpolation processing is carried out to original first color channel data；Alternatively, according to the original all-pass channel Data, original second color channel data and the third color channel data, determine the 4th weight；According to the described 4th Weight and original first color channel data carry out interpolation processing to original first color channel data；

Second RGB image generates submodule and specifically wraps to the original second color channel data progress interpolation processing It includes:

Interpolation interpolation processing is carried out to default second color channel data；Alternatively, according to the original all-pass channel Data, original first color channel data and the third color channel data, determine the 5th weight；According to the described 5th Weight and original second color channel data carry out interpolation processing to original second color channel data；

Second RGB image generates submodule and specifically includes to third color channel data progress interpolation processing:

Interpolation interpolation processing is carried out to the third color channel data；Alternatively, according to the original all-pass channel data, Original first color channel data and original second color channel data, determine the 6th weight；According to the described 6th Weight and the third color channel data carry out interpolation processing to the third color channel data.

Optionally, the Fusion Module, is specifically used for:

Obtain the color information in the RGB image；

The third aspect, the embodiment of the invention provides a kind of image processing systems, comprising:

Image capture device, including at least one imaging sensor, described image sensor include the colour in all-pass channel Filter array, described image sensor, for acquiring raw image data, wherein the raw image data includes at least original Beginning all-pass channel data, original first color channel data and original second color channel data, any channel data all have Empty pixel, first color channel and second color channel are the two of them in R, G, B color channel；

Image processing equipment, for obtaining the raw image data of described image acquisition equipment acquisition；Based on described original Image data generates the all-pass channel image of full resolution and the RGB image of full resolution；By the all-pass channel image and institute It states RGB image to be merged, obtains fused image.

Fourth aspect, the embodiment of the invention provides a kind of electronic equipment, including processor, communication interface, memory and Communication bus, wherein processor, communication interface, memory complete mutual communication by communication bus；

Memory, for storing computer program；

Processor when for executing the program stored on memory, is realized described in any of the above-described image processing method Method and step.

5th aspect, the embodiment of the invention provides a kind of computer readable storage medium, the computer-readable storage Dielectric memory contains computer program, and the computer program realizes any of the above-described image processing method institute when being executed by processor The method and step stated.

Scheme provided in an embodiment of the present invention can obtain the original all-pass channel data, original of imaging sensor acquisition The raw image data of first color channel data and original second color channel data；Based on raw image data, generate complete The all-pass channel image of resolution ratio and the RGB image of full resolution；All-pass channel image is merged with RGB image, is obtained Fused image.Include all-pass channel in scheme provided in an embodiment of the present invention, in raw image data, and then can obtain All-pass channel image is obtained to melt all-pass channel image with RGB image due to advantage of the all-pass channel image in sensitivity Closing obtained blending image has the advantages that signal-to-noise ratio is high, clarity is high, and therefore, scheme provided in an embodiment of the present invention can adopt Collect the preferable clear image of quality.

Detailed description of the invention

In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with It obtains other drawings based on these drawings.

Fig. 1 is a kind of flow diagram of image processing method provided in an embodiment of the present invention；

Fig. 2 be the present embodiments relate to spectral response schematic diagram；

Fig. 3 be the present embodiments relate to bayer array and RCCG array schematic diagram；

Fig. 4 is to merge all-pass channel image with RGB image in a kind of implementation, obtain the place of blending image Manage block diagram；

Fig. 5 is at the first of the all-pass channel image provided in an embodiment of the present invention for generating full resolution and RGB image Manage block diagram；

Fig. 6 be the present embodiments relate to interpolation processing schematic；

Fig. 7 is at the all-pass channel image of generation full resolution provided in an embodiment of the present invention and second of RGB image Manage block diagram；

Fig. 8 be the present embodiments relate to the schematic diagram for supplying missing color channel；

Fig. 9 is at the third of the all-pass channel image provided in an embodiment of the present invention for generating full resolution and RGB image Manage block diagram；

Figure 10 is a kind of structural schematic diagram of image processing apparatus provided in an embodiment of the present invention；

Figure 11 is a kind of structural schematic diagram of image processing system provided in an embodiment of the present invention；

Figure 12 is the structural schematic diagram of a kind of electronic equipment provided in an embodiment of the present invention.

Specific embodiment

Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other Embodiment shall fall within the protection scope of the present invention.

First below to this specification relates to technical term simply introduced.

Raw image data, also referred to as RAW data refer to that the light signal captured is converted digital letter by imaging sensor Number obtained initial data.In raw image data, if being separated into channel data, there are parts in single channel data Empty pixel does not include any channel value at empty pixel, the cavity for example, left figure shown in fig. 6, in R channel data Any R (Red) channel value is not present in pixel, and any G (Green) channel value is not present in the empty pixel in G channel data, There is no any B (Blue) channel values etc. for empty pixel in channel B data.

The image of full resolution refers to for each channel in the presence of the image, in each pixel in the image All with the channel value in the channel.For example, in embodiments of the present invention, each pixel in the all-pass channel image of full resolution Point all includes all-pass channel value；Likewise, each pixel in the RGB image of full resolution includes R channel value, G Channel value and channel B value.

In order to solve the problems, such as that mentioned by above-mentioned background technique, the embodiment of the invention provides provide a kind of image processing method Method, device, electronic equipment and computer readable storage medium.

It describes in detail first below to a kind of image processing method provided in an embodiment of the present invention.

It should be noted that a kind of executing subject of image processing method provided by the embodiment of the present invention can be one kind Image processing apparatus, wherein the image processing apparatus is run in electronic equipment.In a particular application, which can be with Are as follows: the image capture device comprising imaging sensor is not limited thereto certainly.

A kind of image processing method provided in an embodiment of the present invention, as shown in Figure 1, the method comprising the steps of S101~ S103:

S101: the raw image data of imaging sensor acquisition is obtained.

Wherein, raw image data includes original all-pass channel data, original first color channel data and original second Color channel data, any raw image data all have empty pixel, first color channel and second color Channel is the two of them in R, G, B color channel.

It should be noted that including not only all-pass channel in raw image data, also includes rgb color channel, wrapped The rgb color channel contained can be three kinds of channel R, the channel G and channel B color channels, or the channel R, the channel G and Any two kinds in three kinds of color channels of channel B.For example, the raw image data can wrap containing the channel R, the channel G and complete Circulation passage.

It is appreciated that the image capture device can wrap containing optical lens, optical filter and imaging sensor.Wherein, The optical lens can be can be by visible light and the common lens of infrared light.In a kind of situation, which may include filter Mating plate switching device, when ambient light illumination is more than preset threshold, switching device of optical fiber is switched to filtering infrared fileter；Work as ring When border illumination is no more than preset threshold, such as night, switching device of optical fiber is switched to all-pass optical filter；In another case, should Optical filter can also be directly filtering infrared fileter.

In one implementation, raw image data can be with are as follows: using containing all-pass channel road and at least two RGB colors The image data of the imaging sensor acquisition of the color filter lens array in color channel.Above-mentioned imaging sensor can be include all-pass The imaging sensor of channel (C-channel, also known as W channel) color filter lens array, as RGBW (Red-Green-Blue-White, four Color framework) imaging sensor, RCCG (Red-Complete Color-Green, red green all-pass channel) imaging sensor, BCCG (Green-Complete Color-Blue, bluish-green all-pass channel) imaging sensor RCCB (Red-Complete Color- Blue, red indigo plant all-pass channel) imaging sensor etc..

Wherein, the channel photoreception of spectrum range ratio R in all-pass channel, the channel G, channel B and IR (infrared) channel are respective Spectral region is wide, as shown in Fig. 2, to respectively correspond all-pass channel, the channel R, G logical by W, Red, Green, Blue and IR in figure Road, channel B and the channel IR, wider array of photoreception of spectrum range can guarantee that more fully optical signal enters imaging sensor, And then obtain the higher raw image data of quality.

Color filter lens array containing all-pass channel and tradition Bayer (Bayer) array are different, by taking Fig. 3 as an example, Fig. 3 In schematic diagram (a) be tradition Bayer array schematic diagram, schematic diagram (b) be RCCG array schematic diagram, it is found that compared to Traditional Bayer array is substituted the channel B of traditional Bayer array in RCCG array using the channel G, and is substituted using C-channel The channel G of traditional Bayer array.Using the rainbow filter array containing all-pass channel when Image Acquisition, guarantee more abundant Light enter sensor, obtain higher-quality original image.

S102: it is based on raw image data, generates the all-pass channel image of full resolution and the RGB image of full resolution.

It is appreciated that in the raw image data, there are corresponding original all-pass channel data only on partial pixel, On afterimage element and original all-pass channel data is not present, i.e. raw image data has cavity on all-pass channel, and of the invention In embodiment in the all-pass channel image of full resolution, there is all-pass channel value in each pixel.

It is above-mentioned to be based on raw image data as the optional implementation of one of embodiment of the present invention, generate complete differentiate The step of all-pass channel image of rate, may include:

Interpolation processing is carried out to original all-pass channel data, obtains the all-pass channel image of full resolution.

Specifically, in one implementation, original all-pass channel data can be based on, to original all-pass channel data into Row interpolation processing, obtains the all-pass channel image of full resolution.

It is understood that being based on original all-pass channel data, the step of interpolation processing is carried out to original all-pass channel data Suddenly, it is specifically as follows: based on original all-pass channel data, original all-pass channel data is carried out at interpolation using interpolation algorithm Reason determines the channel value at each empty pixel in the original all-pass track data, and then obtains the complete of full resolution Circulation passage image.

Wherein, used interpolation algorithm can be bilinear interpolation method, bicubic interpolation interpolation method etc..It is exemplary , for the pixel that all-pass channel value is each not present in original image, calculate all pixels in the predetermined neighborhood of the pixel The average value of all-pass channel value, and using the average value as the all-pass channel value of the pixel, and then obtain the all-pass of full resolution Channel image.Wherein, the predetermined neighborhood of pixel refers to centered on the pixel, the pixel set in preset range, for example, 3 × 3 Neighborhood.

In another implementation, be also based on original all-pass channel data, original first color channel data and Original second color channel data carries out interpolation processing to original all-pass channel data, obtains the all-pass channel figure of full resolution Picture.It should be understood that, not only with reference to original all-pass channel data, being also referred to when carrying out interpolation processing to all-pass channel data Original first color channel data and original second color channel data, can be improved interpolation, obtain the complete of better quality The all-pass channel image of resolution ratio.

Specifically, interpolation can be carried out to original all-pass channel data according to original all-pass channel data and the first weight Processing, obtains the all-pass channel image of full resolution, wherein the first weight are as follows: according to original first color channel data and original Begin what the second color channel data determined.

Illustratively, for each there is no the pixel of all-pass channel data (i.e. empty pixel), head in original image First according to original first color channel data and original second color channel data of all pixels in the predetermined neighborhood of the pixel The first weight is determined, then according to the all-pass channel of all pixels in the predetermined neighborhood of the identified first weight computing pixel The average value of data, and using the average value as the all-pass channel data of the pixel, and then obtain the all-pass channel of full resolution Image.

May include in step S101 raw image data obtained in the embodiment of the present invention all-pass channel data and Two of them color channel data in rgb color channel.However, the full resolution to be generated in the embodiment of the present invention In RGB image, there are R channel value, G channel value and channel B values in each pixel.

It should be understood that the first situation, when in raw image data including two kinds of color channel datas, in order to obtain The RGB image of full resolution can first supply a kind of color channel data lacked, then to all colors Channel data carries out interpolation, to obtain the RGB image of full resolution, or first to two kinds of existing color channel datas Interpolation is carried out, then a kind of color channel data lacked is supplied, to obtain the RGB image of full resolution.

And second situation does not need then to carry out color when in raw image data including three kinds of color channel datas Channel data supplies processing, but directly carries out interpolation processing to three color channel datas, and the RGB of full resolution can be obtained Image.

In view of length problem, does not describe in detail herein to both the above situation temporarily, be discussed in detail and see below.

S103: all-pass channel image is merged with RGB image, obtains fused image.

In the embodiment of the present invention, all-pass channel image is carried out to merge used fusion with the RGB image of full resolution Mode can be diversified, and the present invention does not limit specific amalgamation mode herein.

It is above-mentioned to melt all-pass channel image with RGB image as a kind of optional implementation of the embodiment of the present invention The step of closing, obtaining blending image, may include following step c1~c3:

Step c1: the color information in RGB image is obtained.

When fusion treatment, need to extract color information to RGB image, then merged with the luminance information of step C2, with Generate while having the output image of exact color and preferable low-light (level) effect.

The color information of pixel and the color space of image are related, for example, can execute to the RGB image of the full resolution Then rgb color space will to the transformation of HSV (Hue, form and aspect, Saturation, saturation degree, Value, lightness) color space The H channel value of each pixel and color of the channel S value as the pixel in blending image in the image obtained after colour space transformation Multimedia message.

For another example, the transformation of rgb color space to YUV color space can be executed to the RGB image of the full resolution, then Using the U channel value of pixel each in the image obtained after colour space transformation and V channel value as the pixel in blending image Color information.

It is appreciated that image capture device calculate blending image in each pixel color information be not limited to it is above two Implementation is also possible to the RGB image of full resolution being converted to other color spaces, as long as the color space can be drawn It is divided into color information and luminance information.

Step c2: the luminance information in the luminance information and all-pass channel image in RGB image is obtained.

When fusion treatment, need that all-pass image and RGB image is combined to generate luminance information, since all-pass image is in sensitivity On advantage so that the luminance picture picture quality with higher under low-light (level) environment generated.

In a kind of optional implementation, the luminance information of each pixel in blending image can directly use all-pass channel Channel value possessed by the pixel of same position indicates in image, for example, the channel value of pixel a is x in all-pass channel image, then The luminance information of pixel a is directly x in blending image.

In order to guarantee the generation effect of luminance information, another kind optionally in implementation, is based on all-pass channel image, meter In calculation blending image the step of luminance information (c2) of each pixel, may include:

Based on all-pass channel image and RGB image, the luminance information of each pixel in blending image is calculated.

Specifically, the luminance information of the pixel in RGB image can be calculated for each pixel in blending image, make For the first luminance information, and using the channel value of the pixel in all-pass channel image as the second luminance information, according still further to preset The first luminance information of weight coefficient and the second luminance information weighted sum obtain the luminance information of the pixel in blending image.

Wherein, the luminance information of pixel can be obtained according to mode in RGB image: execute rgb color space to RGB image To the transformation of YUV color space, believe the corresponding transformed V channel value of each pixel as the brightness of pixel in RGB image Breath.

Illustratively, weight coefficient corresponding to the first luminance information and the second luminance information is all 0.5, then blending image In the luminance information of each pixel can be with are as follows: the first luminance information of the pixel and the second luminance information average value.

Step c3: color information is merged with luminance information, obtains fused image.

Wherein, directly merge blending image in each pixel color information and the obtained blending image of luminance information simultaneously Non- is the image under rgb color space, so the image that fusion obtains can also be converted to the image of rgb format at this time.

Illustratively, if step c1 color information calculated belongs under HSV color space, fusion is being calculated In image after the luminance information of each pixel, using the luminance information of pixel each in blending image as the V channel value of the pixel, Then H channel value, channel S value and the V channel value for combining blending image, obtain the blending image of HSV format, then by the fusion Image is converted to the image of rgb format.

For another example, if step c1 color information calculated belongs under YUV color space, blending image is being calculated In each pixel luminance information after, using the luminance information of pixel each in blending image as the Y channel value of the pixel, then Y channel value, U channel value and the V channel value for combining blending image, obtain the blending image of yuv format, then by the blending image Be converted to the image of rgb format.

It is shown in Figure 4 to merge all-pass channel image with RGB image in a kind of implementation, it is merged The processing block diagram of image it is possible, firstly, to extract the color information in RGB image, obtains color image, together in above process When available RGB image in and all-pass channel image in luminance information, obtain luminance picture, in turn, by color image with Luminance picture is merged, and fused image is obtained.From the foregoing, it will be observed that scheme provided in an embodiment of the present invention, can obtain figure The original of the original all-pass channel data, original first color channel data and original second color channel data that are acquired as sensor Beginning image data；Based on raw image data, the all-pass channel image of full resolution and the RGB image of full resolution are generated；It will All-pass channel image is merged with RGB image, obtains fused image.It is original in scheme provided in an embodiment of the present invention Include all-pass channel in image data, and then all-pass channel image can be obtained, since all-pass channel image is in sensitivity Advantage, it is excellent that the blending image that all-pass channel image is merged with RGB image has that signal-to-noise ratio is high, clarity is high Point, therefore, the preferable clear image of the available quality of scheme provided in an embodiment of the present invention.

In addition, the embodiment of the present invention obtains visible images using an imaging sensor, and obtained from visible images The RGB image of full resolution and the all-pass channel image of full resolution, then by RGB image obtained and all-pass channel image Fusion, so that the higher target image of clarity is obtained, so not needing in the case where the environment illumination intensities such as night are weaker It is additional to carry out light filling, the scheme also preferable clear image of available quality provided by the embodiment of the present invention using infrared light.

Referring to Fig. 5, Fig. 7, it is shown in Fig. 9 generate full resolution all-pass channel image and RGB image processing block diagram, In, processing block diagram when Fig. 5 and Fig. 7 correspond in raw image data comprising two kinds of color channel datas, Fig. 9 corresponds to original Processing block diagram when including three kinds of color channel datas in image data.

In Fig. 5, Fig. 7, processing block diagram shown in Fig. 9, for generate full resolution all-pass channel image the step of, It is by carrying out interpolation processing to original all-pass channel data, to obtain the all-pass channel image of full resolution.

The step of RGB channel image for generating full resolution, in processing block diagram shown in Fig. 5, first to original the One color channel data and original second color channel data carry out interpolation processing, then according to the first color after interpolation processing Channel image, the second color channel images and all-pass channel image supply the third color channel data lacked, from And obtain the RGB image of full resolution；In processing block diagram shown in Fig. 7, first to the third lacked in raw image data Kind color channel data is supplied, then to original first color channel data, original second color channel data and benefit The third color channel data obtained after foot carries out difference processing respectively, to obtain the RGB image of full resolution.

In processing block diagram shown in Fig. 9, due to having existed three kinds of color channel datas in raw image data, no It needs to carry out color channel and supplies processing, difference processing directly is carried out to three kinds of color channel datas, full resolution can be obtained RGB image, optionally, when carrying out differences processing to three kinds of color channel datas and carrying out interpolation processing, see also all-pass Channel data.

When the color channel to missing is supplied, from known original color channel data and original all-pass channel Data supply the color channel of missing, it is ensured that accurate color rendition.When carrying out interpolation to color channel, need to refer to The information of original all-pass channel data, so that interpolation is more accurate, the rgb color image of generation has image detail abundant.

Below by Fig. 5, Fig. 7, processing block diagram shown in Fig. 9, respectively for logical comprising rgb color in raw image data Two kinds of color channel datas, three kinds of color channel data both of these case in road are generated to above-mentioned according to raw image data The step of RGB image of full resolution, is described in detail.

For the first case, raw image data are as follows: comprising all-pass channel and original first color channel data, original Second color channel data, wherein original first color channel data and original second color channel data are rgb color channel In two kinds of color channels image data, that is to say, that: in raw image data other than including all-pass channel data, only It also include two kinds of channel datas in R channel data, G channel data and channel B data, if raw image data includes R Channel data and G channel data, but do not include channel B data, raw image data includes R channel data and channel B number for another example According to, but do not include G channel data, for another example raw image data includes G channel data and channel B data, but does not include R port number According to.It is appreciated that in this case it is desirable to lacked color channel is supplied, to guarantee accurate color rendition.

As the first optional implementation of the embodiment of the present invention, processing block diagram as shown in Figure 5, in original image number It is above-mentioned to be based on raw image data in the case where being lacked in there are color channel, generate the step of the RGB image of full resolution Suddenly, may include following step a1 and a2:

Step a1: it is carried out at interpolation respectively for original first color channel data and original second color channel data Reason, obtains the first color channel images and the second color channel images of full resolution.

It in one implementation, can be to original first color channel number by taking original first color channel data as an example According to interpolation processing is carried out, the channel value at each empty pixel in original first color channel data is determined, and then obtain To the first color channel images of full resolution.Wherein, used interpolation algorithm can be bilinear interpolation method, bicubic Interpolation Interpolation method etc..

Illustratively, the raw image data be RCCG imaging sensor imaging obtain, then the first color channel and Second color channel is respectively the channel R and the channel G, and the third color channel lacked is channel B.Then at this point, step a1 needs For in raw image data the channel R and the channel G carry out interpolation processing respectively, obtain full resolution R channel image and The G channel image of full resolution.

By taking Fig. 6 as an example, Fig. 6 indicates that the interpolation processing schematic in the channel R has five before interpolation processing in array on the right side of Fig. 6 Asterisk note pixel do not have R channel value, then interpolation handle when, should with five-pointed star label pixel R channel value be its 3 × 3 The average value of all R channel values in neighborhood.

In another implementation, by taking original first color channel data as an example, it is also based on original all-pass channel Data and original first color channel data and original second color channel data carry out original first color channel data Interpolation processing obtains the first color channel data image of full resolution.It should be understood that any rgb color port number When according to carrying out interpolation processing, not only with reference to the rgb color all-pass channel data, other RGB in raw image data are also referred to Color channel data and full tunnel channel data, can be improved interpolation, obtain the first color channel number of better quality According to image and the second color channel data image.

Specifically, it is directed to original first color channel data, it can be based on original first color channel data and the second power Value carries out interpolation processing to original first color channel data, obtains the first color channel data image, wherein the second weight Are as follows: it is determined according to original all-pass channel data and original second color channel data.

Illustratively, for each rgb color channel, for the empty pixel in the channel, first according to the sky In the predetermined neighborhood of hole pixel another rgb color channel data of all pixels point in the presence of raw image data with And original full tunnel channel data determines the second weight, then according to the pre- of the identified second weight computing cavity pixel Determine the average value of the channel value of all pixels in neighborhood, and using the average value as the channel value of the cavity pixel.

Step a2: according to all-pass channel image, the first color channel images and the second color channel images, complete differentiate is determined The third color channel image of rate, and then obtain the RGB image of full resolution.

For the third color channel image for not including in raw image data, each pixel is in third color channel Channel value on image can use original all-pass channel image and the first color channel images, the second color channel images In calculate and obtain with the channel value that the pixel of the pixel same position is respectively provided with, the embodiment of the present invention does not limit this The specific implementation of step a2.

It is above-mentioned according to all-pass channel image, first as a kind of specific implementation of step a2 in the embodiment of the present invention Color channel images and the second color channel images, determine the third color channel image of full resolution, and then are differentiated entirely The step of RGB image of rate, may include:

The channel value of any pixel point in third color channel image, predetermined formula are calculated according to predetermined formula are as follows:

S₁=[C₁-(S₂+S₃-α)*K₁+β]*K₂；

In formula, S₁Indicate the channel value of any pixel point in third color channel image, C₁、S₂And S₃Respectively correspond expression In all-pass channel image, the first color channel images and the second color channel images with any pixel point same position picture The channel value of vegetarian refreshments；α,β,K₁And K₂It is preset constant value.

Above-mentioned α, β, K₁And K₂It is preset constant value, the embodiment of the present invention does not limit α, β, K₁And K₂Tool Body value, those skilled in the art can be set as needed, for example, presetting α, β, K₁And K₂Value be respectively 0, 0,1 and 1.

Illustratively, above-mentioned S₁、S₂And S₃Respectively channel B value, R channel value and G channel value, then need polishing channel B Channel data, above-mentioned predetermined formula can be at this time are as follows:

B=[C- (R+G- α) * K₁+β]*K₂；

Wherein, B represents the channel value of the channel B of a certain pixel, and C represents channel of the pixel in all-pass channel image Value, R represent channel value of the pixel in R channel image, and G represents channel value of the pixel in G channel image.

As second of optional implementation of the embodiment of the present invention, processing block diagram as shown in Figure 7, in original image number It is above-mentioned to be based on raw image data in the case where being lacked in there are color channel, generate the step of the RGB image of full resolution Suddenly, may include following step b1 and b2:

Step b1: according to raw image data, third color channel data is determined.

In one implementation, can for the pixel in raw image data each including all-pass channel value, The first color channel of each pixel is logical in the default neighborhood of all-pass channel value and the pixel based on the pixel The channel value of road value and the second color channel calculates the channel value in the third color channel of the pixel.

Identical with the concept of aforementioned predetermined neighborhood is that the default neighborhood of pixel refers to centered on the pixel, in advance If the pixel set in range, for example, 3 × 3 neighborhoods of pixel.By taking Fig. 8 as an example, the pixel institute of array center is located in Fig. 8 The channel value having is C-channel value, it is clear that, above-mentioned first color channel values and the second color channel be respectively R channel value with And G channel value, third color channel value are channel B value, it is assumed that the default neighborhood is 3 × 3 neighborhoods, then for being located at battle array in Fig. 8 The channel B value of the pixel at column center, can according in the C-channel value of the pixel, Fig. 8 two R channel values and two channels G Value, which calculates, to be obtained.

It is diversified, this hair that the specific implementation of the third color channel value of pixel is calculated in the embodiment of the present invention Bright embodiment does not limit herein.It is above-mentioned to be directed to raw image data as a kind of optional implementation of the embodiment of the present invention In each include all-pass channel value pixel, it is each in the default neighborhood of all-pass channel value and the pixel based on the pixel The first color channel values and the second color channel values of pixel, can wrap the step of calculating the third color channel value of the pixel It includes:

By the non-empty pixel in all-pass channel data, the non-empty pixel that is determined as in third color channel data Point determines the channel value of any non-empty pixel in third color channel data, wherein any non-empty pixel in turn The channel value of point be according in original all-pass channel data with any non-empty pixel same position pixel it is logical In road value, original first color channel data with any non-empty pixel same position the corresponding default neck of pixel In domain in the average channel value of each pixel and the second color channel data with any non-empty pixel in same position The average channel value of each pixel determines in the corresponding default field of the pixel set.

Specifically, for the pixel in raw image data each including all-pass channel value, according to following formula meter Calculate the third color channel value of the pixel:

S₄=[C₂-(S₅+S₆-α)*K₁+β]*K₂；

In formula, S₄Indicate the channel value of any non-empty pixel in third color channel data, C₂Indicate original all-pass The channel value of pixel in channel data with any non-empty pixel in same position, S₅Indicate original first color channel The average of each pixel is led in the corresponding default field of pixel of same position with any non-empty pixel in data Road value, S₆It indicates in the second color channel data with any non-empty pixel in the corresponding default neck of the pixel of same position The average channel value of each pixel in domain；α,β,K₁And K₂It is preset constant value.

It is understood that the type of third color channel value belongs to any of R channel value, G channel value and channel B value One kind, but there are two types of the type of the first color channel values and the second color channel values includes, therefore above-mentioned S₅And S₆Respectively correspond One color channel values and the second color channel values, such as when the type of third color channel value belongs to R channel value, the first color The type of channel value and the second color channel values is respectively G channel value and channel B value, then S₅It can indicate the pre- of the pixel If the average value of the G channel value of each pixel, S in neighborhood₆It can indicate the B of each pixel in the default neighborhood of the pixel The average value of channel value.

Likewise, above-mentioned α, β, K₁And K₂It is preset constant value, the embodiment of the present invention does not limit α, β, K₁And K₂Specific value, those skilled in the art can be set as needed, for example, presetting α, β, K₁And K₂Value point It Wei 0,0,1 and 1.

For this implementation, for the target pixel points of the array center shown in Fig. 8, it is assumed that the default neighborhood is 3 × 3 neighborhoods, then above-mentioned S₄Indicate the channel B value of the target pixel points, C₂Indicate the C-channel value of the target pixel points, S₅It can be with table Show the average value of two G channel values shown in fig. 7, S₆It can indicate the average value of two R channel values shown in fig. 7.

Step b2: to original first color channel data, original second color channel data and third color channel data Interpolation processing is carried out respectively, obtains the RGB image of full resolution.

It include the channel R, the channel G and channel B, institute in raw image data after image capture device executes step b1 Interpolation processing can be carried out respectively to the channel R, the channel G and the channel B in raw image data, full resolution is obtained RGB image.

In one implementation, for original first color channel data, original second color channel data and third Totally three kinds of rgb color channel datas for each rgb color channel can be based on the rgb color channel to color channel data Data carry out interpolation processing to the rgb color channel data, to obtain the RGB image of full resolution.

Specifically, each rgb color channel can be directed to, it is based on the rgb color channel data, it is logical to the rgb color Track data carries out interpolation processing using interpolation algorithm, obtains the corresponding full resolution image of the color channel.Wherein, used Interpolation algorithm can be bilinear interpolation method, bicubic interpolation interpolation method etc..

In another implementation, each rgb color channel, the all-pass based on raw image data can also be directed to Channel data and other two rgb color channel data carry out interpolation processing to the rgb color channel, obtain comprising three kinds of RGB The RGB image of the full resolution of color channel.It should be understood that when carrying out interpolation processing to any rgb color channel data, Not only with reference to the rgb color channel data, other rgb color channel datas and full tunnel channel data, Ke Yiti are also referred to High interpolation obtains the RGB image of the full resolution comprising three kinds of rgb color channels of better quality.

Specifically, each rgb color channel can be directed to, based on the rgb color channel data in raw image data With the 4th weight, interpolation processing is carried out to the color channel data, obtains the full resolution comprising three kinds of rgb color channels RGB image, wherein the 4th weight are as follows: according to original all-pass channel data, original first color channel data and original second color What color channel data determined.

Illustratively, for each rgb color channel, for the empty pixel in the channel, first according to the sky The other two rgb color channel data of all pixels point and full tunnel channel data determine in the predetermined neighborhood of hole pixel Then four weights are averaged according to the channel value of all pixels point in the predetermined neighborhood of the identified third weight computing pixel Value, and using the average value as the channel value of the pixel, and then the RGB image of full resolution.

It should be noted that all-pass port number is all utilized in the optional implementation of aforementioned two kinds of embodiment of the present invention According to supplying the color channel data lacked in raw image data, and then the interpolation processing of all color channels is completed, guaranteed The accuracy of color channel interpolation, so that the RGB image of the full resolution generated has image detail abundant.

For second situation, raw image data can be with are as follows: includes three kinds of colors in all-pass channel and rgb color channel The image data in color channel, if the raw image data is the image data that the imaging of RGBW imaging sensor obtains, then, such as Fig. 9 Shown in handle block diagram, above-mentioned to be based on raw image data, the step of generating the RGB image of full resolution, can be with are as follows:

Interpolation processing is carried out respectively for three kinds of color channels in the presence of raw image data, is obtained comprising three kinds The RGB image of the full resolution in rgb color channel.

Specifically, in one implementation, it can be logical for each rgb color in the presence of raw image data Road is based on the rgb color channel data, carries out interpolation processing to the rgb color channel data, obtains comprising three kinds of rgb colors The RGB image of the full resolution in channel.

It is understood that being based on the RGB color for each rgb color channel in the presence of raw image data Color channel data, is specifically as follows: for raw image data the step of carrying out interpolation processing to the rgb color channel data In the presence of each rgb color channel, be based on the rgb color channel data, to the rgb color channel data use interpolation Algorithm carries out interpolation processing, obtains the RGB image of the full resolution comprising three kinds of rgb color channels.Wherein, used interpolation Algorithm can be bilinear interpolation method, bicubic interpolation interpolation method etc..

In another implementation, it can also be directed to each rgb color channel in the presence of raw image data, Rgb color channel data in the presence of all-pass channel data based on raw image data and raw image data, to this Rgb color channel carries out interpolation processing, obtains the RGB image of the full resolution comprising three kinds of rgb color channels.It is understood that , when carrying out interpolation processing to any rgb color channel data, not only with reference to the rgb color all-pass channel data, also refer to Other rgb color channel datas and full tunnel channel data in raw image data, can be improved interpolation, obtain matter Amount preferably includes the RGB image of the full resolution in three kinds of rgb color channels.

Specifically, original image number can be based on for each rgb color channel in the presence of raw image data The rgb color channel data and third weight in carry out interpolation processing to the color channel data of raw image data, obtain To the RGB image of the full resolution comprising three kinds of rgb color channels, wherein third weight are as follows: according to complete in raw image data The other two rgb color channel data in the presence of track data and raw image data determines all.

Illustratively, first for the empty pixel in the channel in original image for each rgb color channel First according to the other two rgb color channel data of all pixels point and full tunnel channel in the predetermined neighborhood of the cavity pixel Data determine third weight, then according to all channels in the predetermined neighborhood of the identified third weight computing cavity pixel The average value of value, and using the average value as the channel value of the cavity pixel, and then obtain including three kinds of rgb color channels The RGB image of full resolution.

Corresponding to embodiment of the method shown in Fig. 1, the embodiment of the invention also provides a kind of image processing apparatus, such as Figure 10 institute Show, described device includes:

Module 110 is obtained, for obtaining the raw image data of image capture device acquisition；Wherein, the original image Data include at least original all-pass channel data, original first color channel data and original second color channel data, any Channel data all has empty pixel, and first color channel and second color channel are in R, G, B color channel Two of them；

Generation module 120, for being based on the raw image data, generating the all-pass channel image of full resolution and dividing entirely The RGB image of resolution；

Fusion Module 130 obtains fused for merging the all-pass channel image with the RGB image Image.

In a kind of implementation, the generation module 120, comprising:

All-pass channel image generates submodule 121, for carrying out interpolation processing to the original all-pass channel data, obtains The all-pass channel image of the full resolution.

In a kind of implementation, the all-pass channel image generates submodule 121, is specifically used for:

In a kind of implementation, the generation module 120, comprising: the first RGB image generates submodule 122, is used for:

In a kind of implementation, first RGB image generates submodule 122 to original first color channel data Interpolation processing is carried out, the first color channel images of full resolution are obtained, specifically:

In a kind of implementation, first RGB image generates submodule 122 to original second color channel data Interpolation processing is carried out, the second color channel images of full resolution are obtained, specifically:

In a kind of implementation, first RGB image generates submodule 122 according to the all-pass channel image, described First color channel images and second color channel images, determine the third color channel image of full resolution, specifically:

In a kind of implementation, the channel value of any pixel point in third color channel image is determined by following formula:

S₁=[C₁-(S₂+S₃-α)*K₁+β]*K₂；

In a kind of implementation, the generation module 120, comprising: the second RGB image generates submodule 123, is used for:

In a kind of implementation, described second RGB image generates submodule 123 according to the raw image data, really Determine third color channel data, specifically include:

In a kind of implementation, any non-empty pixel in third color channel data is determined by following formula Channel value:

S₄=[C₂-(S₅+S₆-α)*K₁+β]*K₂；

In a kind of implementation, second RGB image generates submodule 123 to the original first color channel number Interpolation processing is carried out respectively according to, original second color channel data and the third color channel data, is obtained described complete The RGB image of resolution ratio, specifically:

In a kind of implementation, second RGB image generates submodule 123 to original first color channel data Interpolation processing is carried out to specifically include:

Second RGB image generates submodule 123 and carries out interpolation processing tool to original second color channel data Body includes:

Second RGB image generates submodule 123 and specifically wraps to third color channel data progress interpolation processing It includes:

In a kind of implementation, the Fusion Module 130 is specifically used for:

Obtain the color information in the RGB image；

Scheme provided in an embodiment of the present invention, can obtain imaging sensor acquisition includes all-pass channel and rgb color The raw image data of at least two color channels in channel；Based on raw image data, the all-pass for generating full resolution is logical The RGB image of road image and full resolution；All-pass channel image is merged with RGB image, obtain with exact color and The good blending image of low-light (level) effect, as collected target image.In scheme provided in an embodiment of the present invention, original image Include all-pass channel in data, and then all-pass channel image can be obtained, since all-pass channel image is excellent in sensitivity The blending image that all-pass channel image is merged with RGB image is had the advantages that signal-to-noise ratio is high, clarity is high by gesture, Therefore, scheme provided in an embodiment of the present invention can collect the preferable clear image of quality.

Corresponding to embodiment of the method shown in Fig. 1, the embodiment of the invention also provides a kind of image processing systems, such as Figure 11 institute Show, the system comprises:

Image capture device 210, including at least one imaging sensor 211, described image sensor include all-pass channel Color filter lens array, described image sensor, for acquiring raw image data, wherein the raw image data is at least Including original all-pass channel data, original first color channel data and original second color channel data, any channel data All have empty pixel, first color channel and second color channel are wherein two in R, G, B color channel Kind；

Image processing equipment 220, for obtaining the raw image data of described image acquisition equipment acquisition；Based on the original Beginning image data generates the all-pass channel image of full resolution and the RGB image of full resolution；By the all-pass channel image with The RGB image is merged, and fused image is obtained.

Include all-pass channel in scheme provided in an embodiment of the present invention, in raw image data, and then can obtain complete All-pass channel image merge with RGB image by circulation passage image due to advantage of the all-pass channel image in sensitivity The blending image arrived has the advantages that signal-to-noise ratio is high, clarity is high, and therefore, scheme provided in an embodiment of the present invention can collect The preferable clear image of quality.

The embodiment of the invention also provides a kind of electronic equipment, as shown in figure 12, including processor 301, communication interface 302, memory 303 and communication bus 304, wherein processor 301, communication interface 302, memory 303 pass through communication bus 304 complete mutual communication,

Memory 303, for storing computer program；

Processor 301 when for executing the program stored on memory 303, realizes following steps:

Obtain the raw image data of imaging sensor acquisition；Wherein, the raw image data are as follows: include all-pass channel With the image data of at least two color channels in rgb color channel；

The all-pass channel image is merged with the RGB image, blending image is obtained, as collected target Image.

The method that specific implementation and relevant explanation content about each step of this method may refer to above-mentioned Fig. 1 is implemented Example, this will not be repeated here.

In addition, processor 301 executes other realities of the program stored on memory 303 and the image processing method realized Existing mode, it is identical as implementation mentioned by preceding method embodiment part, it also repeats no more here.

The communication bus that above-mentioned electronic equipment is mentioned can be Peripheral Component Interconnect standard (Peripheral Component Interconnect, PCI) bus or expanding the industrial standard structure (Extended Industry Standard Architecture, EISA) bus etc..The communication bus can be divided into address bus, data/address bus, control bus etc..For just It is only indicated with a thick line in expression, figure, it is not intended that an only bus or a type of bus.

Communication interface is for the communication between above-mentioned electronic equipment and other equipment.

Memory may include random access memory (Random Access Memory, RAM), also may include non-easy The property lost memory (Non-Volatile Memory, NVM), for example, at least a magnetic disk storage.Optionally, memory may be used also To be storage device that at least one is located remotely from aforementioned processor.

Above-mentioned processor can be general processor, including central processing unit (Central Processing Unit, CPU), network processing unit (Network Processor, NP) etc.；It can also be digital signal processor (Digital Signal Processing, DSP), it is specific integrated circuit (Application Specific Integrated Circuit, ASIC), existing It is field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete Door or transistor logic, discrete hardware components.

In another embodiment provided by the invention, a kind of computer readable storage medium is additionally provided, which can It reads to be stored with instruction in storage medium, when run on a computer, so that computer executes any institute in above-described embodiment The image processing method stated.

It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that There is also other identical elements in process, method, article or equipment including the element.

Each embodiment in this specification is all made of relevant mode and describes, same and similar portion between each embodiment Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for device, For electronic equipment and computer readable storage medium embodiment, since it is substantially similar to the method embodiment, so description It is fairly simple, the relevent part can refer to the partial explaination of embodiments of method.

The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the scope of the present invention.It is all Any modification, equivalent replacement, improvement and so within the spirit and principles in the present invention, are all contained in protection scope of the present invention It is interior.

Claims

1. a kind of image processing method, which is characterized in that the described method includes:

Obtain the raw image data of imaging sensor acquisition；Wherein, the raw image data is logical including at least original all-pass Track data, original first color channel data and original second color channel data, any channel data all have empty pixel Point, first color channel and second color channel are the two of them in R, G, B color channel；

The all-pass channel image is merged with the RGB image, obtains fused image.

2. generating complete differentiate the method according to claim 1, wherein described be based on the raw image data The step of all-pass channel image of rate, comprising:

3. according to the method described in claim 2, it is characterized in that, described carry out at interpolation the original all-pass channel data The step of managing, obtaining the all-pass channel image of the full resolution, comprising:

Interpolation interpolation processing is carried out to the original all-pass channel data, determines each sky in the original all-pass track data Channel value at the pixel of hole, and then obtain the all-pass channel image of the full resolution.

4. according to the method described in claim 2, it is characterized in that, described carry out at interpolation the original all-pass channel data Reason, the all-pass channel image for obtaining the full resolution include:

According to the original all-pass channel data and first weight, the original all-pass channel data is carried out at interpolation Reason determines the channel value at each empty pixel in the original all-pass track data, and then obtains the full resolution All-pass channel image.

5. method according to claim 1 to 4, which is characterized in that it is described to be based on the raw image data, The step of generating the RGB image of full resolution, comprising:

Interpolation processing is carried out to original first color channel data, obtains the first color channel images of full resolution；

Interpolation processing is carried out to original second color channel data, obtains the second color channel images of full resolution；

According to the all-pass channel image, first color channel images and second color channel images, complete point is determined The third color channel image of resolution, the third color channel are in R, G, B color channel except first color is logical A color channel except road and second color channel；

6. according to the method described in claim 5, it is characterized in that, it is described original first color channel data is carried out it is slotting The step of value is handled, and obtains the first color channel images of full resolution, comprising:

Interpolation interpolation processing is carried out to original first color channel data, determines original first color channel data In each empty pixel at channel value, and then obtain the first color channel images of the full resolution；Alternatively,

According to the original all-pass channel data and original second color channel data, the second weight is determined；According to described Second weight and original first color channel data carry out interpolation processing to original first color channel data, really The channel value at each empty pixel in original first color channel data is made, and then obtains the full resolution The first color channel images.

7. according to the method described in claim 5, it is characterized in that, it is described original second color channel data is carried out it is slotting The step of value is handled, and obtains the second color channel images of full resolution, comprising:

Interpolation interpolation processing is carried out to original second color channel data, determines original second color channel data In each empty pixel at channel value, and then obtain the second color channel images of the full resolution；Alternatively,

According to the original all-pass channel data and original first color channel data, third weight is determined, according to described Third weight and original second color channel data carry out interpolation processing to original second color channel data, really The channel value at each empty pixel in original second color channel data is made, and then obtains the full resolution The second color channel images.

8. according to the method described in claim 5, it is characterized in that, described according to the all-pass channel image, first color Color channel image and second color channel images, the step of determining the third color channel image of full resolution, comprising:

It determines the channel value of any pixel point in the third color channel image of the full resolution, and then obtains full resolution Third color channel image；Wherein, the channel value of any pixel point is according to the all-pass channel image, described first The channel of pixel in color channel images and second color channel images with any pixel point in same position Value determination.

9. according to the method described in claim 8, it is characterized in that, in the third color channel image any pixel point it is logical Road value is determined by following formula:

S₁=[C₁-(S₂+S₃-α)*K₁+β]*K₂；

In formula, S₁Indicate the channel value of any pixel point in the third color channel image, C₁、S₂And S₃Respectively corresponding indicates institute It states in all-pass channel image, first color channel images and second color channel images and exists with any pixel point The channel value of the pixel of same position, α, β, K₁And K₂It is preset constant value.

10. method according to claim 1 to 4, which is characterized in that described to be based on the original image number According to the step of generating the RGB image of full resolution, comprising:

According to the raw image data, third color channel data is determined, the third color channel is R, G, B color A color channel in channel in addition to first color channel and second color channel；

To original first color channel data, original second color channel data and the third color channel data Interpolation processing is carried out respectively, obtains the RGB image of the full resolution.

11. according to the method described in claim 10, determining third it is characterized in that, described according to the raw image data The step of color channel data, comprising:

By the non-empty pixel in the all-pass channel data, the non-empty picture that is determined as in the third color channel data Vegetarian refreshments；

It determines the channel value of any non-empty pixel in the third color channel data, and then obtains third color channel Data；Wherein, the channel value of any non-empty pixel in the third color channel data, is according to the original all-pass The channel value of pixel in channel data with any non-empty pixel in same position, original first color are logical In track data with any non-empty pixel in the corresponding default field of pixel of same position each pixel In average channel value and second color channel data with any non-empty pixel same position pixel The average channel value of each pixel determines in corresponding default field.

12. according to the method for claim 11, which is characterized in that any non-cavity in the third color channel data The channel value of pixel is determined by following formula:

S₄=[C₂-(S₅+S₆-α)*K₁+β]*K₂；

In formula, S₄Indicate the channel value of any non-empty pixel in the third color channel data, C₂Indicate described original The channel value of pixel in all-pass channel data with any non-empty pixel in same position, S₅Indicate described original It is each in the corresponding default field of pixel of same position with any non-empty pixel in first color channel data The average channel value of a pixel, S₆It indicates in second color channel data with any non-empty pixel same The average channel value of each pixel in the corresponding default field of the pixel of position；α,β,K₁And K₂It is preset constant Value.

13. according to the method described in claim 10, it is characterized in that, described to original first color channel data, institute It states original second color channel data and the third color channel data carries out interpolation processing respectively, obtain the full resolution RGB image, comprising:

Interpolation processing is carried out to original first color channel data, is determined in original first color channel data Channel value at each cavity pixel, and then obtain the first color channel images of full resolution；

Interpolation processing is carried out to original second color channel data, is determined in original second color channel data Channel value at each cavity pixel, and then obtain the second color channel images of full resolution；

Interpolation processing is carried out to the third color channel data, determines each cavity in the third color channel data Channel value at pixel, and then obtain the third color channel image of full resolution；

14. according to the method for claim 13, which is characterized in that described to be carried out to original first color channel data Interpolation processing, comprising:

Alternatively, according to the original all-pass channel data, original second color channel data and the third color channel Data determine the 4th weight；According to the 4th weight and original first color channel data, to original first color Color channel data carries out interpolation processing；

Alternatively, according to the original all-pass channel data, original first color channel data and the third color channel Data determine the 5th weight；According to the 5th weight and original second color channel data, to original second color Color channel data carries out interpolation processing；

Alternatively, according to the original all-pass channel data, original first color channel data and original second color Channel data determines the 6th weight；It is logical to the third color according to the 6th weight and the third color channel data Track data carries out interpolation processing.

15. according to claim 1 to method described in any one of 14, which is characterized in that described by the all-pass channel image The step of being merged with the RGB image, obtain fused image, comprising:

Obtain the color information in the RGB image；

The color information, first luminance information and second luminance information are merged, after obtaining the fusion Image.

16. a kind of image processing apparatus, which is characterized in that described device includes:

Module is obtained, for obtaining the raw image data of image capture device acquisition；Wherein, the raw image data is at least Including original all-pass channel data, original first color channel data and original second color channel data, any channel data All have empty pixel, first color channel and second color channel are wherein two in R, G, B color channel Kind；

Generation module generates the all-pass channel image and full resolution of full resolution for being based on the raw image data RGB image；

Fusion Module obtains fused image for merging the all-pass channel image with the RGB image.

17. device according to claim 16, which is characterized in that the generation module, comprising:

All-pass channel image generates submodule, for carrying out interpolation processing to the original all-pass channel data, obtains described complete The all-pass channel image of resolution ratio.

18. device according to claim 17, which is characterized in that the all-pass channel image generates submodule, specific to use In:

19. device according to claim 17, which is characterized in that the all-pass channel image generates submodule, specific to use In:

20. device described in any one of 6 to 19 according to claim 1, which is characterized in that the generation module, comprising: first RGB image generates submodule, is used for:

21. device according to claim 20, which is characterized in that first RGB image generates submodule to the original Begin the first color channel data progress interpolation processing, obtains the first color channel images of full resolution, specifically:

22. device according to claim 20, which is characterized in that first RGB image generates submodule to the original Begin the second color channel data progress interpolation processing, obtains the second color channel images of full resolution, specifically:

23. device according to claim 20, which is characterized in that first RGB image generates submodule according to All-pass channel image, first color channel images and second color channel images, determine the third color of full resolution Color channel image, specifically:

24. device according to claim 23, which is characterized in that

The channel value of any pixel point in third color channel image is determined by following formula:

S₁=[C₁-(S₂+S₃-α)*K₁+β]*K₂；

25. device described in any one of 6 to 19 according to claim 1, which is characterized in that the generation module, comprising: second RGB image generates submodule, is used for:

26. device according to claim 25, which is characterized in that described second RGB image generates submodule according to institute Raw image data is stated, third color channel data is determined, specifically includes:

It determines the channel value of any non-empty pixel in the third color channel data, and then obtains the third color Channel data, wherein the channel value of any non-empty pixel in the third color channel data, is according to The channel value of pixel in original all-pass channel data with any non-empty pixel in same position, described original the It is each in the corresponding default field of pixel of same position with any non-empty pixel in one color channel data With any non-empty pixel in same position in the average channel value of pixel and second color channel data The corresponding default field of pixel in each pixel average channel value determine.

27. device according to claim 26, which is characterized in that

The channel value of any non-empty pixel in third color channel data is determined by following formula:

S₄=[C₂-(S₅+S₆-α)*K₁+β]*K₂；

28. device according to claim 25, which is characterized in that second RGB image generates submodule to the original Begin the first color channel data, original second color channel data and the third color channel data carries out interpolation respectively Processing, obtains the RGB image of the full resolution, specifically:

29. device according to claim 28, which is characterized in that second RGB image generates submodule to the original The first color channel data progress interpolation processing that begins specifically includes:

Interpolation interpolation processing is carried out to original first color channel data；Alternatively, according to the original all-pass channel data, Original second color channel data and the third color channel data, determine the 4th weight；According to the 4th weight With original first color channel data, interpolation processing is carried out to original first color channel data；

Second RGB image generates submodule and specifically includes to the original second color channel data progress interpolation processing:

Interpolation interpolation processing is carried out to default second color channel data；Alternatively, according to the original all-pass channel data, Original first color channel data and the third color channel data, determine the 5th weight；According to the 5th weight With original second color channel data, interpolation processing is carried out to original second color channel data；

Interpolation interpolation processing is carried out to the third color channel data；Alternatively, according to the original all-pass channel data, described Original first color channel data and original second color channel data, determine the 6th weight；According to the 6th weight With the third color channel data, interpolation processing is carried out to the third color channel data.

30. device described in any one of 6 to 29 according to claim 1, which is characterized in that the Fusion Module is specifically used for:

Obtain the color information in the RGB image；

31. a kind of image processing system characterized by comprising

Image capture device, including at least one imaging sensor, described image sensor include the color filter in all-pass channel Array, described image sensor, for acquiring raw image data, wherein the raw image data includes at least original complete Track data, original first color channel data and original second color channel data, any channel data all have cavity all Pixel, first color channel and second color channel are the two of them in R, G, B color channel；

Image processing equipment, for obtaining the raw image data of described image acquisition equipment acquisition；Based on the original image Data generate the all-pass channel image of full resolution and the RGB image of full resolution；By the all-pass channel image with it is described RGB image is merged, and fused image is obtained.

32. a kind of electronic equipment, which is characterized in that including processor, communication interface, memory and communication bus, wherein processing Device, communication interface, memory complete mutual communication by communication bus；

Memory, for storing computer program；

Processor when for executing the program stored on memory, realizes any method step in claim 1 to 15 Suddenly.

33. a kind of computer readable storage medium, which is characterized in that be stored with computer in the computer readable storage medium Program realizes any method and step in claim 1 to 15 when the computer program is executed by processor.