CN107786857A

CN107786857A - A kind of image restoring method and device

Info

Publication number: CN107786857A
Application number: CN201610720534.0A
Authority: CN
Inventors: 徐志高; 曾峰; 刘刚
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2016-08-25
Filing date: 2016-08-25
Publication date: 2018-03-09
Anticipated expiration: 2036-08-25
Also published as: CN107786857B

Abstract

The embodiment of the present application provides a kind of image restoring method and device.Methods described includes：Obtain the first pixel set of image to be restored, the pixel value of each pixel in first pixel set is the pixel value obtained by feux rouges R passages, green glow G passages, blue light channel B or infrared light IR passages；According to the pixel value of each IR passages and default first interpolation rule, determine respectively each R passages in first pixel set, G passages, channel B pixel value corresponding to IR components；Remove each R passages, G passages, channel B pixel value corresponding to IR components, first pixel set is updated to the second pixel set；R component or B component corresponding to the pixel value of each IR passages in second pixel set are determined, second pixel set is updated to the 3rd pixel set；According to the 3rd pixel set, by the image restoring to be restored.The present embodiment can reduce hardware cost.

Description

A kind of image restoring method and device

Technical field

The application is related to technical field of image processing, more particularly to a kind of image restoring method and device.

Background technology

As the continuous development of science and technology, the application of digital class image capture device are more and more extensive.Camera is that image is adopted Collect the essential device of equipment, and imaging sensor is the photo-sensitive cell in camera.

In order to solve the problems such as daytime, imaging needed optical filter to coordinate, RGB-IR imaging sensors arise at the historic moment.RGB-IR The difference of imaging sensor and conventional image sensor is that IR (Infra Red, infrared light) passage instead of R (Red, feux rouges) Passage or B (Blue, blue light) passage.The pixel data array schematic diagram that Fig. 1 a export for a kind of conventional image sensor, Fig. 1 b For the pixel data array schematic diagram of RGB-IR imaging sensors output.A square in figure represents a pixel.By Fig. 1 a Understand that the IR passages of RGB-IR imaging sensors instead of the pixel-array units of 2 pixel * 2 in conventional image sensor with Fig. 1 b In R passages or channel B, wherein, 2*2 array elements belong to bayer (Bayer) array.Bayer arrays are a kind of simulation human eyes To the sensitivity of color, using 1 red 2 green 1 blue pixel arrangement form.Also, bayer corresponding to RGB-IR imaging sensors Array is also to be alternately arranged array, i.e. RGGIR and BGGIR arrays are alternately present.

For the reason for consider low light effect and imaging sensor manufacturing process etc., and existing R, G (Green, it is green Light), channel B can also sense infrared light while visible ray (wavelength 400nm~780nm) is sensed.Equally, IR passages are sensing Also visible ray can be sensed while infrared light.Fig. 2 is spectral absorption curve corresponding to a kind of RGB-IR imaging sensors, wherein horizontal Axle represents lambda1-wavelength, and the longitudinal axis represents the relative response degree of channel B, G passages and R passages to lambda1-wavelength.From figure As can be seen that each passage has sensing to infrared light of the wavelength more than 780nm.

After imaging sensor collects pixel data array, because each pixel only includes the information of a passage, Therefore, pixel data now is referred to as bayer data, has class mosaic shape.If it is intended to obtain coloured image, it is necessary to Each pixel is reduced to include RGB triple channel information simultaneously, this process is referred to as image restoring, alternatively referred to as demosaicing.

It is mentioned above, the R, G, channel B in RGB-IR imaging sensors can in addition to the visible ray for sensing each passage Sense infrared light.After image restoring is carried out to the pixel data of this imaging sensor, obtained coloured image can be by The influence of infrared light and produce cross-color.

In order to solve the problems, such as the cross-color during this image restoring, usually configured in the prior art in camera Double switching device of optical fiber (IR-CUT).Two optical filters are included in IR-CUT, one is cutoff filter, and another is Full impregnated optical filter.In shooting image on daytime, illumination condition is good, can open cutoff filter, with filtering infrared light, keeps away Exempt from infrared light interference.When shooting at night, illumination condition is poor, can open full impregnated optical filter, using R, G, channel B to infrared light Response carry out light filling, obtain apparent image.But this method needs to configure extra IR-CUT, can increase hardware Cost.

The content of the invention

The purpose of the embodiment of the present application is the provision of a kind of image restoring method and device, to reduce hardware cost.

In order to achieve the above object, this application discloses a kind of image restoring method, methods described to include：

Obtain the first pixel set of image to be restored, the pixel value of each pixel in first pixel set is passes through The pixel value that feux rouges R passages, green glow G passages, blue light channel B or infrared light IR passages obtain；

According to the pixel value of each IR passages and default first interpolation rule, determine respectively each in first pixel set R passages, G passages, channel B pixel value corresponding to IR components；

Remove each R passages, G passages, channel B pixel value corresponding to IR components, first pixel set is updated to Second pixel set；

R component or B component corresponding to the pixel value of each IR passages in second pixel set are determined, by second picture Element set is updated to the 3rd pixel set；

According to the 3rd pixel set, by the image restoring to be restored.

Optionally, it is described according to the pixel value of each IR passages and default first interpolation rule, described first is determined respectively Each R passages in pixel set, G passages, channel B pixel value corresponding to IR components, including：

The first IR pixel data arrays corresponding to the pixel value of each IR passages are isolated from first pixel set；

According to the first IR pixel data arrays and first interpolation rule, first set of pixels is determined respectively Each R passages in conjunction, G passages, channel B pixel value corresponding to IR components.

Optionally, it is described regular according to the first IR pixel data arrays and first interpolation, institute is determined respectively State each R passages in the first pixel set, G passages, channel B pixel value corresponding to IR components, including：

For each R passages and each channel B, according to default first position relation, from the first IR pixel data arrays It is middle to determine that the pixel value for calculating each R passages corresponds to the first object pixel value of IR components, and calculate the pixel value pair of each channel B Answer the second target pixel value of IR components；

According to the first object pixel value, IR components corresponding to the pixel value of each R passages are calculated, according to second mesh Pixel value is marked, calculates IR components corresponding to the pixel value of each channel B；

According to IR components corresponding to each R passages and the pixel value of each channel B, the first IR pixel data arrays are updated For the 2nd IR pixel data arrays；

For each G passages, according to default second place relation, determine to calculate from the 2nd IR pixel data arrays The pixel value of each G passages corresponds to the 3rd target pixel value of IR components；

According to the 3rd target pixel value, IR components corresponding to the pixel value of each G passages are calculated.

Optionally, it is described remove each R passages, G passages, channel B pixel value corresponding to IR components, by first pixel Set is updated to the second pixel set, including：

Judge each R passages, G passages, channel B pixel value corresponding to IR components whether be not more than default first IR intensity Threshold value；

If it is, according to below equation, the pixel value of each R passages, G passages, channel B is removed corresponding IR points respectively Amount：

RGB_0=RGB_1-TH+abs (RGB_IR-TH)

Wherein, the RGB_0 be remove corresponding to R passages after IR components, G passages and channel B pixel value, it is described RGB_1 is the pixel value of each R passages, G passages and channel B, and the TH is the 2nd IR intensity thresholds, and the abs is to take absolute value Symbol, the RGB_IR are IR components corresponding to the pixel value of each R passages, G passages and channel B.

It is optionally, described to determine R component or B component corresponding to the pixel value of each IR passages in second pixel set, Including：

For each IR passages, according to default passage arranging rule, visible light component corresponding to each IR passages is determined, it is described It can be seen that light component is R component or B component；

For visible light component corresponding to each IR passages, according to default 3rd position relationship, from second set of pixels The 4th target pixel value of visible light component corresponding to the pixel value of each IR passages of calculating is determined in conjunction；

According to the 4th target pixel value, visible light component corresponding to the pixel value of each IR passages is calculated.

Optionally, it is described according to the 3rd pixel set, by the image restoring to be restored, including：

For each R passages in the 3rd pixel set and each channel B, according to default 4th position relationship, from institute State and determine to calculate G, the 5th target pixel value of B component corresponding to the pixel value of each R passages in the 3rd pixel set, and calculate 6th target pixel value of R, G component corresponding to the pixel value of each channel B；

According to the 5th target pixel value, G, B component corresponding to each R passages are calculated, according to the 6th object pixel Value, calculates R, G component corresponding to each channel B；

G, B component corresponding to each R passages, and R, G component corresponding to each channel B are updated to the 3rd set of pixels Close；

For each G passages in the 3rd pixel set after renewal, according to default 5th position relationship, after renewal Determine to calculate R, the 7th target pixel value of B component corresponding to the pixel value of each G passages in 3rd pixel set；

According to the 7th target pixel value, R, B component corresponding to the pixel value of the G passages are calculated.

In order to achieve the above object, this application discloses a kind of image restoring device, described device to include：

Pixel obtains module, each in first pixel set for obtaining the first pixel set of image to be restored The pixel value of pixel is the pixel value obtained by feux rouges R passages, green glow G passages, blue light channel B or infrared light IR passages；

First component determining module, it is true respectively for the pixel value according to each IR passages and default first interpolation rule Each R passages in fixed first pixel set, G passages, channel B pixel value corresponding to IR components；

First component remove module, for remove each R passages, G passages, channel B pixel value corresponding to IR components, by institute State the first pixel set and be updated to the second pixel set；

Second component determining module, for determining in second pixel set R corresponding to the pixel value of each IR passages points Amount or B component, the 3rd pixel set is updated to by second pixel set；

Image restoring module, for according to the 3rd pixel set, by the image restoring to be restored.

Optionally, the first component determining module, including：

IR separates submodule, for being isolated from first pixel set first corresponding to the pixel value of each IR passages IR pixel data arrays；

First component determination sub-module, for according to the first IR pixel data arrays and first interpolation rule Then, determine respectively each R passages in first pixel set, G passages, channel B pixel value corresponding to IR components.

Optionally, the first component determination sub-module, including：

First object pixel-value determining unit, for for each R passages and each channel B, being closed according to default first position System, determine that the pixel value for calculating each R passages corresponds to the first object pixel of IR components from the first IR pixel data arrays Value, and calculate the pixel value of each channel B and correspond to the second target pixel value of IR components；

First IR component calculation units, for according to the first object pixel value, the pixel value for calculating each R passages to be corresponding IR components, according to second target pixel value, calculate IR components corresponding to the pixel value of each channel B；

Data array updating block, for IR components corresponding to the pixel value according to each R passages and each channel B, by described in First IR pixel data arrays are updated to the 2nd IR pixel data arrays；

Second target pixel value determining unit, for for each G passages, according to default second place relation, from described Determine that the pixel value for calculating each G passages corresponds to the 3rd target pixel value of IR components in 2nd IR pixel data arrays；

2nd IR component calculation units, for according to the 3rd target pixel value, the pixel value for calculating each G passages to be corresponding IR components.

Optionally, first component removes module, including：

Judging submodule, for judge each R passages, G passages, channel B pixel value corresponding to IR components whether be not more than Default first IR intensity thresholds；

First component remove submodule, for when each R passages, G passages, channel B pixel value corresponding to IR components it is little When default first IR intensity thresholds, according to below equation, the pixel value of each R passages, G passages, channel B is removed pair respectively The IR components answered：

RGB_0=RGB_1-TH+abs (RGB_IR-TH)

Optionally, the second component determining module, including：

It can be seen that light component determination sub-module, for for each IR passages, according to default passage arranging rule, determining each IR Visible light component corresponding to passage, the visible light component are R component or B component；

First object pixel value determination sub-module, for for visible light component corresponding to each IR passages, according to default 3rd position relationship, determine to calculate the of visible light component corresponding to the pixel value of each IR passages from second pixel set Four target pixel values；

It can be seen that light component calculating sub module, for according to the 4th target pixel value, calculating the pixel value of each IR passages Corresponding visible light component.

Optionally, described image recovery module, including：

Second target pixel value determination sub-module, for leading to for each R passages in the 3rd pixel set and each B Road, according to default 4th position relationship, determine to calculate from the 3rd pixel set G corresponding to the pixel value of each R passages, 5th target pixel value of B component, and calculate the 6th target pixel value of R, G component corresponding to the pixel value of each channel B；

First component calculating sub module, for according to the 5th target pixel value, calculating G, B corresponding to each R passages points Amount, according to the 6th target pixel value, calculates R, G component corresponding to each channel B；

Pixel set updates submodule, and by G, B component corresponding to each R passages, and R, G component corresponding to each channel B are more New extremely the 3rd pixel set；

3rd target pixel value determination sub-module, for for each G passages in the 3rd pixel set after renewal, according to Default 5th position relationship, determine to calculate corresponding to the pixel value of each G passages R, B points from the 3rd pixel set after renewal 7th target pixel value of amount；

Second component calculating sub module, for according to the 7th target pixel value, calculating the pixel value pair of the G passages R, the B component answered.

As seen from the above technical solution, in the embodiment of the present application, first, for the first pixel of the image to be restored of acquisition Set, according to the first interpolation rule, determine respectively each R passages in the first pixel set, G passages, channel B pixel value corresponding to IR components.Then, remove each R passages, G passages, channel B pixel value corresponding to IR components, the first pixel set is updated to Second pixel set.Secondly, R component or B component corresponding to the pixel value of each IR passages in the second pixel set are determined, by second Pixel set is updated to the 3rd pixel set.Finally, according to the 3rd pixel set, by the image restoring to be restored.

That is, in the embodiment of the present application, according to the pixel value of each IR passages and the first interpolation rule, determine that each R leads to Road, G passages, channel B pixel value corresponding to IR components and remove, then according to removing each R, G after IR components, channel B Pixel value, determine the R or B component of IR passages, finally according to remove IR components after each pixel value, by image to be restored also It is former.It can be seen that using the embodiment of the present application, without filtering out IR components using hardware device, you can by the IR in each passage pixel value Component removes, and avoids the coloured image after reduction from producing cross-color because of the influence by infrared light, therefore can reduce hard Part cost.

Brief description of the drawings

, below will be to embodiment or existing in order to illustrate more clearly of the embodiment of the present application or technical scheme of the prior art There is the required accompanying drawing used in technology description to be briefly described.It should be evident that drawings in the following description are only this Some embodiments of application, for those of ordinary skill in the art, on the premise of not paying creative work, can be with Other accompanying drawings are obtained according to these accompanying drawings.

Fig. 1 a are a kind of schematic diagram of the array of image data of conventional image sensor output；

Fig. 1 b are a kind of schematic diagram of the array of image data of RGB-IR imaging sensors output；

Fig. 2 is spectral absorption curve corresponding to a kind of RGB-IR imaging sensors；

Fig. 3 is a kind of schematic flow sheet for the image restoring method that the embodiment of the present application provides；

Fig. 4 a are a kind of schematic diagram for first pixel set that RGB-IR imaging sensors obtain；

Fig. 4 b are a kind of schematic diagram for the first IR pixel data arrays isolated from Fig. 4 a；

Fig. 4 c are a kind of schematic diagram of the 2nd IR pixel data arrays after Fig. 4 b are updated；

A kind of schematic diagram for the second pixel set that Fig. 4 d are obtained after the IR components for each RGB channel of removal；

Fig. 4 e are a kind of schematic diagram that IR passages are replaced with to the 3rd pixel set obtained after visible light component；

Fig. 4 f are a kind of schematic diagram of the 3rd pixel set after renewal；

Fig. 4 g be complete image restoring after pixel set a kind of schematic diagram；

Fig. 5 is by a kind of schematic flow sheet of image restoring to be restored in Fig. 3 according to the 3rd pixel set；

Fig. 6 is a kind of structural representation for the image restoring device that the embodiment of the present application provides.

Embodiment

Below in conjunction with the accompanying drawing in the embodiment of the present application, the technical scheme in the embodiment of the present application is carried out clear, complete Whole description.Obviously, described embodiment is only the part of the embodiment of the application, rather than whole embodiments.Base Embodiment in the application, those of ordinary skill in the art are obtained all on the premise of creative work is not made Other embodiment, belong to the scope of the application protection.

The embodiment of the present application provides a kind of image restoring method and device, can reduce hardware cost.

Below by specific embodiment, the application is described in detail.

Fig. 3 is a kind of schematic flow sheet for the image restoring method that the embodiment of the present application provides, and can apply to electronics and sets Standby, the electronic equipment can include the mobile terminals such as video camera, smart mobile phone, tablet personal computer, and the non-shifting such as desktop computer Dynamic terminal.

Specifically, this method comprises the following steps：

Step S301, obtains the first pixel set of image to be restored, the pixel value of each pixel in the first pixel set For the pixel value obtained by feux rouges R passages, green glow G passages, blue light channel B or infrared light IR passages.

In embodiments of the present invention, electronic equipment can obtain the first pixel set of image to be restored.Specifically, electronics Equipment can obtain the first pixel set from image capture device.For example, image capture device can be included in electronic equipment, The elements such as imaging sensor can also be included in image capture device, so as to which electronic equipment can obtain image sensor element The first pixel set sent；Or electronic equipment can obtain the image sensor element transmission of external image collecting device The first pixel set.

In embodiments of the present invention, the imaging sensor in image capture device can be RGB-IR imaging sensors.Electricity Each R passages, G passages in the first pixel set that sub- equipment obtains, infrared light component is contained in the pixel value of channel B, i.e., IR components.

Fig. 1 b are a kind of pixel data array schematic diagram of RGB-IR imaging sensors output.A square in figure represents One pixel, and include an IR passage in each 2*2 array elements.From Fig. 1 b it can also be seen that due to each 2*2 Passage in array element is arranged and differed, and therefore, the square 4*4 array elements formed with each 4 pixels of ranks are minimum Cycling element, the arrangement of such passage is referred to as CFA (Color Filter Array, color filter array) 4*4 arrays.

Certainly, the BGGIR shown in Fig. 1 b and RGGIR is the specific array arrangement form of RGB-IR imaging sensors.Make For another specific embodiment, RGB-IR imaging sensors can also include the array arrangement forms such as GIRBG and GIRRG, The present embodiment is not specifically limited to this array arrangement form.

Fig. 4 a are the first pixel set schematic diagram that electronic equipment obtains from RGB-IR imaging sensors, wherein containing Totally 100 pixels, the pixel value of each pixel include the pixel value of R passages, G passages, channel B and IR passages to 10*10.In order that Statement becomes apparent from, and the application is that the pixel value of each passage in Fig. 4 a is assigned with one from 00~99 numbering.

Step S302, according to the pixel value of each IR passages and default first interpolation rule, the first set of pixels is determined respectively Each R passages in conjunction, G passages, channel B pixel value corresponding to IR components.

As a kind of specific embodiment, in the present embodiment, step S302, i.e., according to the pixel value of each IR passages and pre- If the first interpolation rule, determine respectively each R passages in first pixel set, G passages, channel B pixel value corresponding to IR components, it can include：

Step 1, the first IR pixel data arrays corresponding to the pixel value of each IR passages are isolated from the first pixel set.

Specifically, the first IR pixel data arrays can be obtained in the following ways：

R, G, the pixel value of channel B position are set to 0 in first pixel set, and the pixel value of IR passages position is protected Stay, obtain the first IR pixel data arrays.

By taking the first pixel set shown in Fig. 4 a as an example, the first IR pixel data arrays therefrom isolated in the manner described above As shown in Figure 4 b, the pixel value of IR passages is only included in Fig. 4 b.

Step 2, according to the first IR pixel data arrays and the first interpolation rule, determine respectively each in the first pixel set R passages, G passages, channel B pixel value corresponding to IR components.

As a kind of specific embodiment, above-mentioned steps 2, i.e., according to the first IR pixel data arrays and described First interpolation rule, determine respectively each R passages in first pixel set, G passages, channel B pixel value corresponding to IR point Amount, can include：

Step 2a, for each R passages and each channel B, according to default first position relation, from the first IR pixel data battle arrays Determine that the pixel value for calculating each R passages corresponds to the first object pixel value of IR components in row, and calculate the pixel value of each channel B Second target pixel value of corresponding IR components.

That is, in the present embodiment, the IR of the IR components and the pixel value of calculating channel B of the pixel value of R passages is calculated The process of component is similar, is required for according to first position relation, determines to be used to calculate from the first IR pixel data arrays The target pixel value of IR components.

In order that statement is easy, the IR components of the pixel value of channel B are represented with IR/B, the pixel of R passages is represented with IR/R The IR components of value.Follow-up X/Y form of presentation can be understood as representing the X-component of the pixel value of Y passages.

Specifically, it is determined that the first pixel set centre position IR/R when, can centered on the position of R passages, will The R passages upper left corner in first IR pixel data arrays, the lower left corner, the pixel value of IR passages in the upper right corner and the lower right corner are defined as One target pixel value.It is determined that the first pixel set centre position IR/B when, centered on the position of channel B, by the first IR The channel B upper left corner in pixel data array, the lower left corner, the pixel value of IR passages in the upper right corner and the lower right corner are defined as the second target Pixel value.

Illustrated below exemplified by determining the IR components of pixel value of channel B., can be with for example, when calculating IR/B44 From the first IR pixel data arrays shown in Fig. 4 b, determine that the IR in the B44 upper left corner, the lower left corner, the upper right corner and the lower right corner leads to The pixel value in road, i.e. IR33, IR53, IR35 and IR55 are the second target pixel value.

Step 2b, according to first object pixel value, IR components corresponding to the pixel value of each R passages are calculated, according to the second mesh Pixel value is marked, calculates IR components corresponding to the pixel value of each channel B.

As a kind of specific embodiment, IR/B and IR/R can be calculated in the following ways：

Obtain n_diff=abs (IR_{The upper left corner}–IR_{The lower right corner}) and p_diff=abs (IR_{The lower left corner}–IR_{The upper right corner})；

Work as n_diff>During p_diff, IR/B and IR/R are defined as (IR_{The lower left corner}+IR_{The upper right corner})/2；

Work as n_diff<During p_diff, IR/B and IR/R are defined as (IR_{The upper left corner}+IR_{The lower right corner})/2；

As n_diff=p_diff, IR/B and IR/R are defined as (IR_{The upper left corner}+IR_{The lower right corner}+IR_{The lower left corner}+IR_{The upper right corner})/4。

Wherein, abs is takes absolute value symbol, IR_{The upper left corner}、IR_{The lower right corner}、IR_{The lower left corner}And IR_{The upper right corner}Respectively the first IR pixel data battle arrays The upper left corner at each R passages and each channel B position in row, the lower right corner, the pixel value in the lower left corner and the upper right corner.

For example, according to Fig. 4 a and Fig. 4 b, the process for calculating IR/B44 is as follows：

N_diff=abs (IR33-IR55)；

P_diff=abs (IR53-IR35)；

If n_diff>P_diff, then IR/B44=(IR53+IR35)/2；

If n_diff<P_diff, then IR/B44=(IR33+IR55)/2；

If n_diff=p_diff, IR/B44=(IR33+IR55+IR53+IR35)/4.

Step 2c, according to IR components corresponding to each R passages and the pixel value of each channel B, by the first IR pixel data arrays It is updated to the 2nd IR pixel data arrays.

, can be by each R passages and each channel B it is determined that after IR components corresponding to each R passages and the pixel value of each channel B Pixel value corresponding to IR components be added to the first corresponding opening position of IR pixel data arrays, obtain the 2nd IR pixel data battle arrays Row.

For example, Fig. 4 b are the first IR pixel data arrays, Fig. 4 c are by corresponding to each R passages and the pixel value of each channel B IR components are updated to the 2nd IR pixel data arrays obtained after the first IR pixel data arrays shown in Fig. 4 b.That is, IR pixel datas shown in Fig. 4 c include the pixel value and each R, the IR components of channel B of IR passages.

Step 2d, for each G passages, according to default second place relation, determined from the 2nd IR pixel data arrays The pixel value for calculating each G passages corresponds to the 3rd target pixel value of IR components.

Specifically, it is determined that the first pixel set centre position IR/G when, can centered on the position of G passages, will G passages left side in 2nd IR pixel data arrays, right side, above and below IR pixel datas be defined as the 3rd object pixel Value, the 3rd target pixel value include the pixel value of IR passages and the IR components of R, G passage.

For example, when calculating IR/G45, G45 passages can be determined from the 2nd IR pixel data arrays shown in Fig. 4 c Left side, right side, above and below IR pixel values, i.e. IR44, IR46, IR35 and IR55 is the 3rd target pixel value.

Step 2e, according to the 3rd target pixel value, calculate IR components corresponding to the pixel value of each G passages.

As a kind of specific embodiment, IR/G can be calculated in the following ways, and IR/G represents the pixel value of G passages Corresponding IR components：

Obtain h_diff=abs (IR_{Left side}–IR_{Right side}) and v_diff=abs (IR_Top–IR_{Lower section})；

Work as h_diff>During v_diff, IR/G is defined as (IR_{Left side}+IR_{Right side})/2；

Work as h_diff<During v_diff, IR/G is defined as (IR_Top+IR_{Lower section})/2；

As h_diff=v_diff, IR/G is defined as (IR_{Left side}+IR_{Right side}+IR_Top+IR_{Lower section})/4。

Wherein, abs is takes absolute value symbol, IR_{Left side}、IR_{Right side}、IR_TopAnd IR_{Lower section}In respectively the 2nd IR pixel data arrays G passages pixel value position at left side, right side, above and below element.

For example, according to Fig. 4 a and Fig. 4 c, the process for calculating IR/G45 is as follows：

H_diff=abs (IR44-IR46)；

V_diff=abs (IR35-IR55)；

If h_diff>V_diff, then IR/G45=(IR44+IR46)/2；

If v_diff<H_diff, then IR/G45=(IR35+IR55)/2；

If h_diff=v_diff, IR/G45=(IR44+IR46+IR35+IR55)/4.

It should be noted that when calculating IR/R, IR/B and IR/G of the first pixel set marginal position, have as one kind The embodiment of body, can by the pixel value of IR passages in the 2*2 array elements where each R passages, channel B and G passages, it is determined that For the IR components of each R passages, channel B and G passages.

For example, in fig.4, B00, G01, G10, IR11 collectively constitute a 2*2 array element, and the array element position In the upper left corner of pixel set shown in Fig. 4 a, the position belongs to marginal position.Led to according to above-mentioned calculating each R passages, channel B and G The process of the IR components in road, the pixel value of IR11 passages can be defined as IR/B and IR/G.

Step S303, remove each R passages, G passages, channel B pixel value corresponding to IR components, by the first pixel set more New is the second pixel set.

It should be noted that step S303 can include numerous embodiments, directly each R passages, G passages, B can be led to The pixel value in road removes corresponding IR components.

But if the brightness of IR components has exceeded certain threshold value, illustrate to there may be overexposure phenomenon, at this moment will The pixel value of corresponding passage directly removes the loss that IR components will cause visible channel information.The brightness of this IR components surpasses Cross a range of it may be the case that as caused by including stronger light energy in the target image gathered.

In order to avoid channel information loses, the accuracy of image restoring result is improved, as a kind of specific embodiment, In the present embodiment, step S303, that is, remove each R passages, G passages, channel B pixel value corresponding to IR components, by the first picture Element set is updated to the second pixel set, can include：

Judge each R passages, G passages, channel B pixel value corresponding to IR components whether be not more than default first IR intensity Threshold value；If it is not, then do not process, if it is, according to below equation, by the pixel value of each R passages, G passages, channel B point Not Qu Chu corresponding to IR components：

RGB_0=RGB_1-TH+abs (RGB_IR-TH)

Wherein, RGB_0 be remove corresponding to R passages after IR components, G passages and channel B pixel value, RGB_1 is The pixel value of each R passages, G passages and channel B, TH are the 2nd IR intensity thresholds, and abs is the symbol that takes absolute value, and RGB_IR is each R IR components corresponding to the pixel value of passage, G passages and channel B.

Wherein, the first IR intensity thresholds can have certain relation with the 2nd IR intensity thresholds, for example, the first IR intensity Threshold value can be the multiple of the default value of the 2nd IR intensity thresholds, and predetermined number can take 2 or 3 etc numerical value.Above-mentioned Two IR intensity thresholds can be preset value, and e.g., for 12bit pixel data width, the 2nd IR intensity thresholds TH can be set to 512 or other values.The application is not specifically limited to the value of the first IR intensity thresholds and the 2nd IR intensity thresholds.

Removed for example, the pixel value of each R passages in the first pixel set shown in Fig. 4 a, G passages, channel B is corresponded to Corresponding IR components, the second pixel set as shown in figure 4d is obtained, in order that statement becomes apparent from, the application uses band in figure 4d The pixel value of apostrophe " ' " represents to eliminate R, G, the pixel value of channel B of infrared light component.

Step S304, R component or B component corresponding to the pixel value of each IR passages in the second pixel set are determined, by second Pixel set is updated to the 3rd pixel set.

It should be noted that the pixel value of the R, G, channel B in the second pixel set have eliminated corresponding IR points Amount.

Because in pixel data array corresponding to RGB-IR imaging sensors, IR passages instead of traditional image sensing R passages or channel B in device, therefore, in order to combine traditional images restoring method, it is necessary to obtain visible ray R, G of all positions, Channel B pixel value, that is, also need to determine R component or B component corresponding to the pixel value of the IR passages in the second pixel set.

As a kind of specific embodiment, in the present embodiment, step S304, that is, each IR in the second pixel set is determined R component or B component corresponding to the pixel value of passage, can include：

Step 1, for each IR passages, according to default passage arranging rule, visible ray corresponding to each IR passages point is determined Amount, the visible light component is R component or B component.

Wherein, passage arranging rule can be the regular array being alternately present according to the BGGIR shown in Fig. 1 b and RGGIR, Can also be the regular array being alternately present according to GIRBG and GIRRG, specific passage arranging rule can be according to actual conditions It is determined that.

Step 2, for visible light component corresponding to each IR passages, according to default 3rd position relationship, from the second pixel The 4th target pixel value of visible light component corresponding to the pixel value of each IR passages of calculating is determined in set.

Specifically, it is determined that the second pixel set centre position visible light component when, can using the position of IR passages as Center, by IR passages top in the second pixel set, lower section, left side, right side G passages pixel value, and the upper right corner, lower-left The R passages at angle or the pixel value of channel B, are defined as the 4th target pixel value.Wherein, the pixel value of G passages, R passages and channel B For the pixel value after the pixel value removal IR components of G passages, R passages and channel B.

Below to be illustrated exemplified by the R component (i.e. R/IR) for determining IR passages.It is understood that according to above-mentioned removal The R component that the 4th target pixel value after IR components determines is also not contain IR components.

For example, when calculating R/IR55, the 4th target pixel value can be determined from the second pixel set shown in Fig. 4 d For：IR55 tops, lower section, left side, right side G passages pixel value, i.e. G ' 45, G ' 65, G ' 54 and G ' 56, and IR55 upper rights Angle, the lower left corner R passages pixel value, i.e. R ' 46 and R ' 64.

Step 3, according to the 4th target pixel value, visible light component corresponding to the pixel value of each IR passages is calculated.

As a kind of specific embodiment, R/IR or B/IR can be calculated in the following ways：

Obtain h_h_laplace=abs (2*G_Top–G_{The top of top}–G_{Lower section}),

H_v_laplace=abs (2*G_Top–G_{The left side of top}–G_{The right side of top}),

V_h_laplace=abs (2*G_{Left side}–G_{The left side in left side}–G_{Right side}) and

V_v_laplace=abs (2*G_{Left side}–G_{The top in left side}–G_{The lower section in left side})；

For R/IR,

If h_h_laplace<H_v_laplace, then R/IR=(G_Top+G_{Lower section})*(R_{The upper right corner}+R_{The upper left corner})/[2*(G_{Left side}+ G_{Right side})]；

If v_h_laplace>V_v_laplace, then R/IR=(G_{Left side}+G_{Right side})*(R_{The upper right corner}+R_{The upper left corner})/[2*(G_Top+ G_{Lower section})]；

If above-mentioned two condition is all unsatisfactory for, R/IR=(R_{The upper right corner}+R_{The upper left corner})/2；

For B/IR,

If h_h_laplace<H_v_laplace, then B/IR=(G_Top+G_{Lower section})*(B_{The upper right corner}+B_{The upper left corner})/[2*(G_{Left side}+ G_{Right side})]；

If v_h_laplace>V_v_laplace, then B/IR=(G_{Left side}+G_{Right side})*(B_{The upper right corner}+B_{The upper left corner})/[2*(G_Top+ G_{Lower section})]；

If above-mentioned two condition is all unsatisfactory for, B/IR=(B_{The upper right corner}+B_{The upper left corner})/2。

Wherein, abs is takes absolute value symbol, G_Top、G_{Lower section}、G_{Left side}And G_{Right side}Respectively IR passages top, lower section, left side, the right side The pixel value of the G passages of side, R_{The upper right corner}And R_{The upper left corner}The respectively pixel value in the R passages upper right corner, the lower left corner, B_{The upper right corner}And B_{The upper left corner}Respectively The channel B upper right corner, the pixel value in the lower left corner, G_{The top of top}、G_{The left side of top}And G_{The right side of top}The respectively G of G passages_TopTop, left side and The pixel value on right side, G_{The top in left side}、G_{The left side in left side}And G_{The lower section in left side}The respectively G of G passages_{Left side}Top, the pixel value of left side and lower section.

For example, the second pixel set according to Fig. 4 d, the process for calculating R/IR55 are as follows：

The h_h_laplace=abs (- G ' 65 of 45-G ' of 2*G ' 25)；

The h_v_laplace=abs (- G ' 47 of 45-G ' of 2*G ' 43)；

The v_h_laplace=abs (- G ' 56 of 54-G ' of 2*G ' 52)；

The v_v_laplace=abs (- G ' 74 of 54-G ' of 2*G ' 34)；

If h_h_laplace<H_v_laplace, then R/IR55=(G ' 45+G ' 65) * (R ' 46+R ' 64)/[2* (G ' 54+ G′56)]；

If v_h_laplace>V_v_laplace, then R/IR55=(G ' 54+G ' 56) * (R ' 46+R ' 64)/[2* (G ' 45+ G′65)]；

If above-mentioned condition is unsatisfactory for, R/IR55=(R ' 46+R ' 64)/2.

It should be noted that when calculating R/IR and B/IR of the second pixel set marginal position, as a kind of specific Embodiment, it can determine as follows：

For the second pixel set leftmost side one row and the top a line 2*2 array elements in IR passages, R/IR and B/IR is identified as：The IR passages lower left corner and the upper right cornerPassage orThe average value of the pixel value of passage.For example, scheming In 4d, R/IR11 is (R ' 02+R ' 20)/2.

For the IR passages in the row 2*2 array elements of the second pixel set rightmost side one, R/IR and B/IR are identified as： The IR passages position lower left cornerPassage orThe pixel value of passage.For example, in figure 4d, B/IR39 is taken as B ' 48 picture Element value；

For the IR passages in second pixel set bottom a line 2*2 array elements, R/IR and B/IR are identified as： The IR passages position upper right cornerPassage orThe pixel value of passage.For example, in figure 4d, B/IR97 is taken as B ' 88 picture Element value；

For the IR passages in a 2*2 array element of the second pixel set last cell, R/IR and B/IR difference are true It is set to：The visible light component of IR passages in the 2*2 array elements in the IR passages position upper left corner.For example, in figure 4d, R/ IR99 is taken as R ' 77 pixel value.

It is determined that IR passages pixel value visible light component after, the second pixel set shown in Fig. 4 d can be updated For the 3rd pixel set shown in Fig. 4 e.Wherein, each IR passages have been replaced with R component and B component, and each pixel is not wrapped Component containing IR.In order that statement becomes apparent from, in figure 4e, R component and B component for replacing IR passage pixel values use band respectively The amount of apostrophe " ' " represents, for example, R ' 11 is exactly the R component pixel value of IR passages.

Step S305, according to the 3rd pixel set, by image restoring to be restored.

It should be noted that due in the 3rd pixel set each location of pixels remove infrared light component comprising one Visible channel information, in order to by image restoring to be restored corresponding to the 3rd pixel set, it is thus necessary to determine that in the 3rd pixel set Other two visible channel information of each location of pixels.

Specifically, according to the 3rd pixel set, by image restoring to be restored, numerous embodiments can be included, can also Image restoring to be restored, its specific implementation process the present embodiment are no longer described in detail using interpolation algorithm of the prior art.

As shown in the above, in the present embodiment, according to the pixel value of each IR passages and the first interpolation rule, each R is determined Passage, G passages, channel B pixel value corresponding to IR components and remove, then according to removing each R, G after IR components, channel B Pixel value, determine the R or B component of IR passages, finally according to each pixel value after IR components is removed, by image to be restored also It is former.It can be seen that using the present embodiment, without filtering out IR components using hardware device, you can by the IR components in each passage pixel value Remove, avoid reduction after coloured image cross-color is produced because of the influence by infrared light, therefore can reduce hardware into This.

In another embodiment herein, embodiment illustrated in fig. 3 can be improved.In order to more efficient, accurate Ground is by image restoring to be restored, step S305, i.e.,, can be with by the image restoring to be restored according to the 3rd pixel set Carried out according to schematic flow sheet shown in Fig. 5, it specifically includes following steps：

Step S501, for each R passages in the 3rd pixel set and each channel B, according to default 4th position relationship, Determine to calculate G, the 5th target pixel value of B component corresponding to the pixel value of each R passages from the 3rd pixel set, and calculate 6th target pixel value of R, G component corresponding to the pixel value of each channel B.

That is, in the present embodiment, the picture of G corresponding to the pixel value of each R passages, B component with calculating each channel B is calculated The process of R, G component is similar corresponding to plain value, is required for, according to the 4th position relationship, mesh being determined from the 3rd pixel set Mark pixel value.

Specifically, it is determined that the 3rd pixel set centre position G/R when, can be centered on the position of R passages, by R passages top in three pixel sets, lower section, left side, the pixel value of G passages on right side are defined as the 5th target pixel value；True , can be centered on the position of R passages, by the 3rd pixel set on R passages during the B/R in fixed 3rd pixel set centre position Side, lower section, left side, the pixel value of channel B on right side are defined as the 5th target pixel value.

It is determined that the 3rd pixel set centre position G/B when, can be centered on the position of channel B, by the 3rd pixel Channel B top in set, lower section, left side, the pixel value of G passages on right side are defined as the 6th target pixel value；It is determined that the 3rd During the R/B in pixel set centre position, can centered on the position of channel B, by the 3rd pixel set above channel B, under Side, left side, the pixel value of R passages on right side are defined as the 6th target pixel value.

For example, in the 3rd pixel set shown in Fig. 4 e, when calculating G/B44, it is determined that the 6th target pixel value be： B44 tops, lower section, left side, right side R passages pixel value, i.e. G ' 34, G ' 54, G ' 43 and G ' 45.

Step S502, according to the 5th target pixel value, G, B component corresponding to each R passages are calculated, according to the 6th object pixel Value, calculates R, G component corresponding to each channel B.

As a kind of specific embodiment, G/B, G/R or B/R, R/B can be calculated in the following ways：

Obtain h_h_laplace=abs (G_{The top of top}–G_{Lower section})+abs(2*G_Top–G_{The top of top}–G_{Lower section}),

H_v_laplace=abs (G_{The left side of top}–G_{The right side of top})+abs(2*G_Top–G_{The left side of top}–G_{The right side of top}),

V_h_laplace=abs (G_{The top in left side}–G_{The lower section in left side})+abs(2*G_{Left side}–G_{The top in left side}–G_{The lower section in left side}) and

V_v_laplace=abs (G_{The left side in left side}–G_{Right side})+abs(2*G_{Left side}–G_{The left side in left side}–G_{Right side})；

For G/B and G/R,

If h_h_laplace<H_v_laplace, then G/B and G/R is (G_Top+G_{Lower section})/2；

If v_h_laplace>V_v_laplace, then G/B and G/R is (G_{Left side}+G_{Right side})/2；

If above-mentioned condition is unsatisfactory for, G/B and G/R are (G_Top+G_{Lower section}+G_{Left side}+G_{Right side})/4；

For B/R,

If h_h_laplace<H_v_laplace, then B/R=(B_Top+B_{Lower section})/2；

If v_h_laplace>V_v_laplace, then B/R=(B_{Left side}+B_{Right side})/2；

If above-mentioned condition is unsatisfactory for, B/R=(B_Top+B_{Lower section}+B_{Left side}+B_{Right side})/4；

For R/B,

If h_h_laplace<H_v_laplace, then R/B=(R_Top+R_{Lower section})/2；

If v_h_laplace>V_v_laplace, then R/B=(R_{Left side}+R_{Right side})/2；

If above-mentioned condition is unsatisfactory for, R/B=(R_Top+R_{Lower section}+R_{Left side}+R_{Right side})/4。

Wherein, abs is takes absolute value symbol, G_Top、G_{Lower section}、G_{Left side}And G_{Right side}Respectively R passages or channel B top, lower section, a left side Side, right side G passages pixel value, B_Top、B_{Lower section}、B_{Left side}And B_{Right side}Respectively R passages top, lower section, left side, the channel B on right side Pixel value, R_Top、R_{Lower section}、R_{Left side}And R_{Right side}Respectively channel B top, lower section, left side, right side R passages pixel value. G_{The top of top}、G_{The left side of top}And G_{The right side of top}Respectively R passages or the G of channel B_TopTop, the pixel value on left side and right side, G_{The top in left side}、 G_{The left side in left side}And G_{The lower section in left side}Respectively R passages or the G of channel B_{Left side}Top, the pixel value of left side and lower section.

For example, the 3rd pixel set according to Fig. 4 e, the process for calculating G '/B ' 44 and R '/B ' 44 is as follows：

H_h_laplace=abs (- the G ' 54 of G ' 14)+abs (- G ' 54 of 34-G ' of 2*G ' 14),

H_v_laplace=abs abs (- the G ' 36 of G ' 32)+abs (- G ' 36 of 34-G ' of 2*G ' 32),

V_h_laplace=abs (- the G ' 63 of G ' 23)+abs (- G ' 63 of 43-G ' of 2*G ' 23) and

V_v_laplace=abs (- the G ' 45 of G ' 41)+abs (- G ' 45 of 43-G ' of 2*G ' 41)

For G '/B ' 44,

If h_h_laplace<H_v_laplace, then G '/B ' 44 is (G ' 34+G ' 54)/2；

If v_h_laplace>V_v_laplace, then G '/B ' 44 is (G ' 43+G ' 45)/2；

If above-mentioned condition is unsatisfactory for, G '/B ' 44 is (G ' 43+G ' 45+G ' 34+G ' 54)/4；

For R '/B ' 44,

If h_h_laplace<H_v_laplace, then R '/B ' 44=(R ' 34+R ' 54)/2；

If v_h_laplace>V_v_laplace, then R '/B ' 44=(R ' 43+R ' 45)/2；

If above-mentioned condition is unsatisfactory for, R '/B ' 44=(R ' 43+R ' 45+R ' 34+R ' 54)/4.

It should be noted that when calculating G/B, G/R or B/R, the R/B of the 3rd pixel set marginal position, as one kind Specific embodiment, it can determine as follows：

For the 2*2 array elements of the 3rd pixel set marginal position, G/B, G/R are identified as in the array element： The average value of the pixel value of two G passages in the array element.For example, in the 3rd pixel set shown in Fig. 4 e, G '/B ' 00 =G '/R ' 11=(G ' 01+G ' 10)/2.

For the 2*2 array elements of the 3rd pixel set marginal position, B/R is defined as in the array element：The array list The pixel value of G passages in member；R/B is defined as in the array element：The pixel value of R passages in the array element.

Step S503, by G, B component corresponding to each R passages, and R, G component are updated to the 3rd picture corresponding to each channel B Element set.

Fig. 4 f are a kind of schematic diagram of the 3rd pixel set after renewal.G corresponding to each R passages in figure, B component and R, G component have obtained corresponding to each channel B, in order that statement becomes apparent from, use the amount of the double apostrophes " 〞 " of band to represent in Fig. 4 f Through obtaining the channel B and R passages of other two components, such as.B 〞 00 represent the passage have been obtained for R 〞 00 and G 〞 00 this Two components.

Step S504, for each G passages in the 3rd pixel set after renewal, according to default 5th position relationship, Determine to calculate R, the 7th target pixel value of B component corresponding to the pixel value of each G passages from the 3rd pixel set after renewal.

Specifically, it is determined that renewal after the 3rd pixel set in centre position B/G when, can be with the position of G passages Centered on, the pixel value of channel B on G passages top in the 3rd pixel set after renewal, lower section, left side, right side is defined as 7th target pixel value.It is determined that renewal after the 3rd pixel set in centre position R/G when, can be with the position of G passages Centered on, the pixel value of channel B on R passages top in the 3rd pixel set after renewal, lower section, left side, right side is defined as 7th target pixel value.

For example, in Fig. 4 f, when calculating B/G ' 54, determine that the 7th target pixel value is：The tops of G ' 54, lower section, left side, The pixel value of the channel B on right side, i.e. B 〞 44, B 〞 64, B 〞 53 and B 〞 55.When calculating R/G ' 54, the 7th target pixel value is determined For：The tops of G ' 54, lower section, left side, right side R passages pixel value, i.e. R 〞 44, R 〞 64, R 〞 53 and R 〞 55.

Step S505, according to the 7th target pixel value, calculate R, B component corresponding to the pixel value of the G passages.

As a kind of specific embodiment, B/G and R/G can be calculated in the following ways：

Obtain h_h_diff=abs (G_{The top of top}–G_{The lower section of lower section})+abs(G–G_{The top of top}–G_{The lower section of lower section}),

H_v_diff=abs (G_{The left side in left side}–G_{The right side on right side}+abs(G–G_{The left side in left side}–G_{The right side on right side})；

For B/G,

If h_h_diff<H_v_diff, then B/G=(B_Top+B_{Lower section})/2；

If h_h_diff>H_v_diff, then B/G=(B_{Left side}+B_{Right side})/2；

If above-mentioned condition is unsatisfactory for, B/G=(B_Top+B_{Lower section}+B_{Left side}+B_{Right side})/4；

For R/G,

If h_h_diff<H_v_diff, then R/G=(R_Top+R_{Lower section})/2；

If h_h_diff>H_v_diff, then R/G=(R_{Left side}+R_{Right side})/2；

If above-mentioned condition is unsatisfactory for, R/G=(R_Top+R_{Lower section}+R_{Left side}+R_{Right side})/4。

Wherein, abs is takes absolute value symbol, B_Top、B_{Lower section}、B_{Left side}And B_{Right side}Respectively G passages top, lower section, left side, right side Channel B pixel value, R_Top、R_{Lower section}、R_{Left side}And R_{Right side}Respectively G passages top, lower section, left side, right side R passages pixel Value.G_{The top of top}、G_{The lower section of lower section}、G_{The left side in left side}And G_{The right side on right side}The lower section of top, lower section, the left side in left side, the right side respectively above G passages The G components on the right side of side.

For example, the 3rd pixel set after renewal is as shown in fig. 4f, the process for calculating B 〞/G ' 54 and R 〞/G ' 54 is as follows：

H_h_diff=abs (- the G ' 74 of G ' 34)+abs (- G ' 74 of 54-G ' of G ' 34) is obtained,

H_v_diff=abs (- the G ' 56 of G ' 52)+abs (- G ' 56 of 54-G ' of G ' 52)；

For B 〞/G ' 54,

If h_h_diff<H_v_diff, then B 〞/G ' 54=(B 〞 44+B 〞 64)/2；

If h_h_diff>H_v_diff, then B 〞/G ' 54=(B 〞 53+B 〞 55)/2；

If above-mentioned condition is unsatisfactory for, B 〞/G ' 54=(B 〞 44+B 〞 64+B 〞 53+B 〞 55)/4；

For R 〞/G ' 54,

If h_h_diff<H_v_diff, then R 〞/G ' 54=(R 〞 44+R 〞 64)/2；

If h_h_diff>H_v_diff, then R 〞/G ' 54=(R 〞 53+R 〞 55)/2；

If above-mentioned condition is unsatisfactory for, R 〞/G ' 54=(R 〞 44+R 〞 64+R 〞 53+R 〞 55)/4.

It should be noted that in B/G and R/G for calculating the 3rd pixel set marginal position after updating, as one kind Specific embodiment, it can determine as follows：

For the 2*2 array elements of the 3rd pixel set marginal position after renewal, by other B components in the array element Or R component is defined as B/G or R/G in the array element.For example, the 3rd pixel set after renewal is as shown in fig. 4f, by B 〞 00 Pixel value be defined as B 〞/G ' 01 and B 〞/G ' 10, R 〞 11 pixel value is defined as R 〞/G ' 01 and R 〞/G ' 10.

As an example, Fig. 4 g are the R component for calculating the G passages in the 3rd pixel set after being updated shown in Fig. 4 f With the pixel set after B component, each passage contains RGB component in the pixel set, is the colour for eliminating IR components Image, i.e. Fig. 4 g are to complete the image after image restoring.

Fig. 6 is a kind of structural representation for the image restoring device that the embodiment of the present application provides, and is implemented with method shown in Fig. 1 Example is corresponding, and described device includes：

Pixel obtains module 601, for obtaining the first pixel set of image to be restored, in first pixel set The pixel value of each pixel is the pixel value obtained by feux rouges R passages, green glow G passages, blue light channel B or infrared light IR passages；

First component determining module 602, for the pixel value according to each IR passages and default first interpolation rule, difference Determine each R passages in first pixel set, G passages, channel B pixel value corresponding to IR components；

First component remove module 603, for remove each R passages, G passages, channel B pixel value corresponding to IR components, First pixel set is updated to the second pixel set；

Second component determining module 604, for determining R corresponding to the pixel value of each IR passages in second pixel set Component or B component, second pixel set is updated to the 3rd pixel set；

Image restoring module 605, for according to the 3rd pixel set, by the image restoring to be restored.

As a kind of specific embodiment, in the embodiment shown in fig. 6, the first component determining module 602 can be with Including：

IR separates submodule (not shown), for isolating the pixel of each IR passages from first pixel set First IR pixel data arrays corresponding to value；

First component determination sub-module (not shown), for according to the first IR pixel data arrays and described First interpolation rule, determine respectively each R passages in first pixel set, G passages, channel B pixel value corresponding to IR point Amount.

As a kind of specific embodiment, in the embodiment shown in fig. 6, the first component determination sub-module can be with Including：

First object pixel-value determining unit (not shown), for for each R passages and each channel B, according to default First position relation, determine that the pixel value for calculating each R passages corresponds to IR components from the first IR pixel data arrays First object pixel value, and calculate the pixel value of each channel B and correspond to the second target pixel value of IR components；

First IR component calculation unit (not shown)s, for according to the first object pixel value, calculating each R passages Pixel value corresponding to IR components, according to second target pixel value, calculate IR components corresponding to the pixel value of each channel B；

Data array updating block (not shown), for IR corresponding to the pixel value according to each R passages and each channel B Component, the first IR pixel data arrays are updated to the 2nd IR pixel data arrays；

Second target pixel value determining unit (not shown), for for each G passages, according to the default second place Relation, determine that the pixel value for calculating each G passages corresponds to the 3rd target picture of IR components from the 2nd IR pixel data arrays Element value；

2nd IR component calculation unit (not shown)s, for according to the 3rd target pixel value, calculating each G passages Pixel value corresponding to IR components.

As a kind of specific embodiment, in the embodiment shown in fig. 6, first component removes module 603, can be with Including：

Judging submodule (not shown), for judge each R passages, G passages, channel B pixel value corresponding to IR point Whether amount is not more than default first IR intensity thresholds；

First component removes submodule (not shown), corresponding for the pixel value when each R passages, G passages, channel B IR components when being not more than default first IR intensity thresholds, according to below equation, by each R passages, G passages, channel B pixel It is worth IR components corresponding to removing respectively：

RGB_0=RGB_1-TH+abs (RGB_IR-TH)

As a kind of specific embodiment, in the embodiment shown in fig. 6, the second component determining module 604 can be with Including：

It can be seen that light component determination sub-module (not shown), for for each IR passages, being arranged according to default passage Rule, determines visible light component corresponding to each IR passages, and the visible light component is R component or B component；

First object pixel value determination sub-module (not shown), for for visible ray corresponding to each IR passages point Amount, according to default 3rd position relationship, determine to calculate corresponding to the pixel value of each IR passages from second pixel set It can be seen that the 4th target pixel value of light component；

It can be seen that light component calculating sub module (not shown), for according to the 4th target pixel value, calculating each IR Visible light component corresponding to the pixel value of passage.

As a kind of specific embodiment, in the embodiment shown in fig. 6, described image recovery module 605, including：

Second target pixel value determination sub-module (not shown), for for each R in the 3rd pixel set Passage and each channel B, according to default 4th position relationship, determine to calculate the picture of each R passages from the 3rd pixel set G, the 5th target pixel value of B component corresponding to element value, and calculate the 6th of R, G component corresponding to the pixel value of each channel B the Target pixel value；

First component calculating sub module (not shown), for according to the 5th target pixel value, calculating each R passages Corresponding G, B component, according to the 6th target pixel value, calculate R, G component corresponding to each channel B；

Pixel set updates submodule (not shown), and G, B component corresponding to each R passages, and each channel B is corresponding R, G component be updated to the 3rd pixel set；

3rd target pixel value determination sub-module (not shown), for in the 3rd pixel set after renewal Each G passages, according to default 5th position relationship, determine to calculate the pixel of each G passages from the 3rd pixel set after renewal 7th target pixel value of R, B component corresponding to value；

Second component calculating sub module (not shown), for according to the 7th target pixel value, calculating the G and leading to R, B component corresponding to the pixel value in road.

Because said apparatus embodiment is obtained based on embodiment of the method, there is identical technique effect with this method, Therefore the technique effect of device embodiment will not be repeated here.

For device embodiment, because it is substantially similar to embodiment of the method, so describe fairly simple, it is related Part illustrates referring to the part of embodiment of the method.

It should be noted that herein, such as first and second or the like relational terms are used merely to a reality Body or operation make a distinction with another entity or operation, and not necessarily require or imply and deposited between these entities or operation In any this actual relation or order.Moreover, term " comprising ", "comprising" or any other variant be intended to it is non- It is exclusive to include, so that process, method, article or equipment including a series of elements not only include those key elements, But also the other element including being not expressly set out, or also include solid by this process, method, article or equipment Some key elements.In the absence of more restrictions, the key element limited by sentence "including a ...", it is not excluded that including Other identical element also be present in the process of the key element, method, article or equipment.

It will appreciated by the skilled person that all or part of step in above-mentioned embodiment is can to pass through journey Come what is completed, described program can be stored in computer read/write memory medium the related hardware of sequence instruction.It is designated herein Storage medium, refer to ROM/RAM, magnetic disc, CD etc..

The preferred embodiment of the application is the foregoing is only, is not intended to limit the protection domain of the application.It is all Any modification, equivalent substitution and improvements done within spirit herein and principle etc., it is all contained in the protection domain of the application It is interior.

Claims

A kind of 1. image restoring method, it is characterised in that methods described includes：

Obtain the first pixel set of image to be restored, the pixel value of each pixel in first pixel set is passes through feux rouges The pixel value that R passages, green glow G passages, blue light channel B or infrared light IR passages obtain；

According to the pixel value of each IR passages and default first interpolation rule, determine that each R leads in first pixel set respectively Road, G passages, channel B pixel value corresponding to IR components；

Remove each R passages, G passages, channel B pixel value corresponding to IR components, first pixel set is updated to second Pixel set；

R component or B component corresponding to the pixel value of each IR passages in second pixel set are determined, by second set of pixels Conjunction is updated to the 3rd pixel set；

According to the 3rd pixel set, by the image restoring to be restored.
2. according to the method for claim 1, it is characterised in that described according to the pixel value of each IR passages and default first Interpolation rule, determine respectively each R passages in first pixel set, G passages, channel B pixel value corresponding to IR components, bag Include：

The first IR pixel data arrays corresponding to the pixel value of each IR passages are isolated from first pixel set；

According to the first IR pixel data arrays and first interpolation rule, determine respectively in first pixel set Each R passages, G passages, channel B pixel value corresponding to IR components.
3. according to the method for claim 2, it is characterised in that described according to the first IR pixel data arrays and institute State the first interpolation rule, determine respectively each R passages in first pixel set, G passages, channel B pixel value corresponding to IR Component, including：

It is true from the first IR pixel data arrays according to default first position relation for each R passages and each channel B The pixel value of each R passages of devise a stratagem calculation corresponds to the first object pixel value of IR components, and the pixel value of each channel B of calculating corresponds to IR Second target pixel value of component；

According to the first object pixel value, IR components corresponding to the pixel value of each R passages are calculated, according to second target picture Element value, calculates IR components corresponding to the pixel value of each channel B；

According to IR components corresponding to each R passages and the pixel value of each channel B, the first IR pixel data arrays are updated to Two IR pixel data arrays；

For each G passages, according to default second place relation, determine to calculate each G from the 2nd IR pixel data arrays The pixel value of passage corresponds to the 3rd target pixel value of IR components；

According to the 3rd target pixel value, IR components corresponding to the pixel value of each G passages are calculated.
4. according to the method for claim 1, it is characterised in that described to remove each R passages, G passages, the pixel value of channel B Corresponding IR components, first pixel set is updated to the second pixel set, including：

Judge each R passages, G passages, channel B pixel value corresponding to IR components whether be not more than default first IR intensity thresholds Value；

If it is, according to below equation, the pixel value of each R passages, G passages, channel B is removed into corresponding IR components respectively：

RGB_0=RGB_1-TH+abs (RGB_IR-TH)

Wherein, the RGB_0 be remove corresponding to R passages after IR components, G passages and channel B pixel value, the RGB_ 1 is the pixel value of each R passages, G passages and channel B, and the TH is the 2nd IR intensity thresholds, and the abs is the symbol that takes absolute value, The RGB_IR is IR components corresponding to the pixel value of each R passages, G passages and channel B.
5. according to the method for claim 1, it is characterised in that described to determine each IR passages in second pixel set R component corresponding to pixel value or B component, including：

For each IR passages, according to default passage arranging rule, visible light component corresponding to each IR passages is determined, it is described visible Light component is R component or B component；

For visible light component corresponding to each IR passages, according to default 3rd position relationship, from second pixel set It is determined that calculate the 4th target pixel value of visible light component corresponding to the pixel value of each IR passages；

According to the 4th target pixel value, visible light component corresponding to the pixel value of each IR passages is calculated.
6. according to the method for claim 1, it is characterised in that it is described according to the 3rd pixel set, treated described also Original image reduces, including：

For each R passages in the 3rd pixel set and each channel B, according to default 4th position relationship, from described Determine to calculate G, the 5th target pixel value of B component corresponding to the pixel value of each R passages in three pixel sets, and calculate each B 6th target pixel value of R, G component corresponding to the pixel value of passage；

According to the 5th target pixel value, G, B component corresponding to each R passages are calculated, according to the 6th target pixel value, meter Calculate R, G component corresponding to each channel B；

G, B component corresponding to each R passages, and R, G component corresponding to each channel B are updated to the 3rd pixel set；

For each G passages in the 3rd pixel set after renewal, according to default 5th position relationship, from the 3rd after renewal Determine to calculate R, the 7th target pixel value of B component corresponding to the pixel value of each G passages in pixel set；

According to the 7th target pixel value, R, B component corresponding to the pixel value of the G passages are calculated.
7. a kind of image restoring device, it is characterised in that described device includes：

Pixel obtains module, for obtaining the first pixel set of image to be restored, each pixel in first pixel set Pixel value be the pixel value that is obtained by feux rouges R passages, green glow G passages, blue light channel B or infrared light IR passages；

First component determining module, for the pixel value according to each IR passages and default first interpolation rule, institute is determined respectively State each R passages in the first pixel set, G passages, channel B pixel value corresponding to IR components；

First component removes module, for remove each R passages, G passages, channel B pixel value corresponding to IR components, by described the One pixel set is updated to the second pixel set；

Second component determining module, for determining R component or B corresponding to the pixel value of each IR passages in second pixel set Component, second pixel set is updated to the 3rd pixel set；

Image restoring module, for according to the 3rd pixel set, by the image restoring to be restored.
8. device according to claim 7, it is characterised in that the first component determining module, including：

IR separates submodule, for isolating the first IR pictures corresponding to the pixel value of each IR passages from first pixel set Plain data array；

First component determination sub-module, for according to the first IR pixel data arrays and first interpolation rule, dividing Do not determine each R passages in first pixel set, G passages, channel B pixel value corresponding to IR components.
9. device according to claim 8, it is characterised in that the first component determination sub-module, including：

First object pixel-value determining unit, for for each R passages and each channel B, according to default first position relation, from Determine that the pixel value for calculating each R passages corresponds to the first object pixel value of IR components in the first IR pixel data arrays, with And the pixel value of each channel B of calculating corresponds to the second target pixel value of IR components；

First IR component calculation units, for according to the first object pixel value, calculating IR corresponding to the pixel value of each R passages Component, according to second target pixel value, calculate IR components corresponding to the pixel value of each channel B；

Data array updating block, for IR components corresponding to the pixel value according to each R passages and each channel B, by described first IR pixel data arrays are updated to the 2nd IR pixel data arrays；

Second target pixel value determining unit, for for each G passages, according to default second place relation, from described second Determine that the pixel value for calculating each G passages corresponds to the 3rd target pixel value of IR components in IR pixel data arrays；

2nd IR component calculation units, for according to the 3rd target pixel value, calculating IR corresponding to the pixel value of each G passages Component.
10. device according to claim 7, it is characterised in that first component removes module, including：

Judging submodule, for judge each R passages, G passages, channel B pixel value corresponding to IR components whether no more than default The first IR intensity thresholds；

First component removes submodule, for when each R passages, G passages, channel B pixel value corresponding to IR components no more than pre- If the first IR intensity thresholds when, according to below equation, the pixel value of each R passages, G passages, channel B is removed respectively corresponding to IR components：

RGB_0=RGB_1-TH+abs (RGB_IR-TH)

Wherein, the RGB_0 be remove corresponding to R passages after IR components, G passages and channel B pixel value, the RGB_ 1 is the pixel value of each R passages, G passages and channel B, and the TH is the 2nd IR intensity thresholds, and the abs is the symbol that takes absolute value, The RGB_IR is IR components corresponding to the pixel value of each R passages, G passages and channel B.
11. device according to claim 7, it is characterised in that the second component determining module, including：

It can be seen that light component determination sub-module, for for each IR passages, according to default passage arranging rule, determining each IR passages Corresponding visible light component, the visible light component are R component or B component；

First object pixel value determination sub-module, for for visible light component corresponding to each IR passages, according to the default 3rd Position relationship, determine to calculate the 4th mesh of visible light component corresponding to the pixel value of each IR passages from second pixel set Mark pixel value；

It can be seen that light component calculating sub module, for according to the 4th target pixel value, the pixel value for calculating each IR passages to be corresponding Visible light component.
12. device according to claim 7, it is characterised in that described image recovery module, including：

Second target pixel value determination sub-module, for for each R passages in the 3rd pixel set and each channel B, root According to default 4th position relationship, determine to calculate G, B component corresponding to the pixel value of each R passages from the 3rd pixel set The 5th target pixel value, and calculate the 6th target pixel value of R, G component corresponding to the pixel value of each channel B；

First component calculating sub module, for according to the 5th target pixel value, calculating G, B component corresponding to each R passages, root According to the 6th target pixel value, R, G component corresponding to each channel B are calculated；

Pixel set updates submodule, and by G, B component corresponding to each R passages, and R, G component are updated to corresponding to each channel B 3rd pixel set；

3rd target pixel value determination sub-module, for for each G passages in the 3rd pixel set after renewal, according to default The 5th position relationship, from the 3rd pixel set after renewal determine to calculate R corresponding to the pixel value of each G passages, B component 7th target pixel value；

Second component calculating sub module, for according to the 7th target pixel value, corresponding to the pixel value for calculating the G passages R, B component.