CN107786857B

CN107786857B - A kind of image restoring method and device

Info

Publication number: CN107786857B
Application number: CN201610720534.0A
Authority: CN
Inventors: 徐志高; 曾峰; 刘刚
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2016-08-25
Filing date: 2016-08-25
Publication date: 2019-07-19
Anticipated expiration: 2036-08-25
Also published as: CN107786857A

Abstract

The embodiment of the present application provides a kind of image restoring method and device.The described method includes: obtaining the first pixel set of image to be restored, the pixel value of each pixel in first pixel set is the pixel value obtained by the channel feux rouges R, the channel green light G, blue light channel B or the channel infrared light IR；According to the pixel value in each channel IR and preset first interpolation rule, determine respectively each channel R in first pixel set, the channel G, channel B the corresponding IR component of pixel value；Remove each channel R, the channel G, channel B the corresponding IR component of pixel value, first pixel set is updated to the second pixel set；The corresponding R component of pixel value or B component for determining each channel IR in second pixel set, are updated to third pixel set for second pixel set；According to the third pixel set, by the image restoring to be restored.The present embodiment can reduce hardware cost.

Description

A kind of image restoring method and device

Technical field

This application involves technical field of image processing, in particular to a kind of image restoring method and device.

Background technique

With the continuous development of science and technology, digital class image capture device using more and more extensive.Camera is that image is adopted Collect the essential device of equipment, and imaging sensor is the photosensitive element in camera.

In order to solve the problems such as daytime, imaging needed optical filter to cooperate, RGB-IR imaging sensor comes into being.RGB-IR The difference of imaging sensor and conventional image sensor is that (Infra Red, the infrared light) channel IR is instead of R (Red, feux rouges) Channel or B (Blue, blue light) channel.Fig. 1 a is a kind of pixel data array schematic diagram of conventional image sensor output, Fig. 1 b For the pixel data array schematic diagram of RGB-IR imaging sensor output.A square in figure represents a pixel.By Fig. 1 a With Fig. 1 b it is found that the channel IR of RGB-IR imaging sensor is instead of 2 pixel *, 2 pixel-array unit in conventional image sensor In the channel R or channel B, wherein 2*2 array element belongs to bayer (Bayer) array.Bayer array is a kind of simulation human eye To the sensitivity of color, using 1 red 2 green 1 blue pixel arrangement form.Also, the corresponding bayer of RGB-IR imaging sensor Array is also to be alternately arranged array, i.e. RGGIR and BGGIR array is alternately present.

For the reason of consider low light effect and imaging sensor manufacturing process etc., and existing R, G (Green, it is green Light), channel B can also incude infrared light while incuding visible light (wavelength 400nm~780nm).Equally, the channel IR is incuding Also visible light can be incuded while infrared light.Fig. 2 is a kind of corresponding spectral absorption curve of RGB-IR imaging sensor, wherein horizontal Axis indicates that lambda1-wavelength, the longitudinal axis indicate channel B, the channel G and the channel R to the relative response degree of lambda1-wavelength.From figure As can be seen that infrared light of each channel to wavelength greater than 780nm has induction.

After imaging sensor collects pixel data array, since each pixel only includes the information in a channel, Therefore, pixel data at this time is referred to as bayer data, has class mosaic shape.If it is intended to obtaining color image, need Each pixel is reduced to simultaneously comprising RGB triple channel information, this process is known as image restoring, alternatively referred to as demosaicing.

It is mentioned above, the R, G, channel B in RGB-IR imaging sensor can in addition to the visible light for incuding each channel Incude infrared light.After the pixel data to this imaging sensor carries out image restoring, obtained color image be will receive The influence of infrared light and generate cross-color.

In order to solve the problems, such as the cross-color during this image restoring, usually configured in camera in the prior art Double switching device of optical fiber (IR-CUT).It include two optical filters in IR-CUT, one is cutoff filter, the other is Full impregnated optical filter.When shooting image daytime, illumination condition is good, can open cutoff filter, with filtering infrared light, keeps away Exempt from infrared light interference.In night shooting, illumination condition is poor, full impregnated optical filter can be opened, using R, G, channel B to infrared light Response carry out light filling, obtain clearer image.But this method needs to configure additional IR-CUT, will increase hardware Cost.

Summary of the invention

The embodiment of the present application has been designed to provide a kind of image restoring method and device, to reduce hardware cost.

In order to achieve the above object, this application discloses a kind of image restoring methods, which comprises

The first pixel set of image to be restored is obtained, the pixel value of each pixel in first pixel set is to pass through The pixel value that the channel feux rouges R, the channel green light G, blue light channel B or the channel infrared light IR obtain；

According to the pixel value in each channel IR and preset first interpolation rule, determine respectively each in first pixel set The channel R, the channel G, channel B the corresponding IR component of pixel value；

Remove each channel R, the channel G, channel B the corresponding IR component of pixel value, first pixel set is updated to Second pixel set；

The corresponding R component of pixel value or B component for determining each channel IR in second pixel set, by second picture Element set is updated to third pixel set；

According to the third pixel set, by the image restoring to be restored.

Optionally, described according to the pixel value in each channel IR and preset first interpolation rule, described first is determined respectively Each channel R in pixel set, the channel G, channel B the corresponding IR component of pixel value, comprising:

The corresponding first IR pixel data array of pixel value in each channel IR is isolated from first pixel set；

According to the first IR pixel data array and first interpolation rule, first set of pixels is determined respectively Each channel R in conjunction, the channel G, channel B the corresponding IR component of pixel value.

Optionally, described regular according to the first IR pixel data array and first interpolation, institute is determined respectively State each channel R in the first pixel set, the channel G, channel B the corresponding IR component of pixel value, comprising:

For each channel R and each channel B, according to preset first position relationship, from the first IR pixel data array The middle determining pixel value for calculating each channel R corresponds to the first object pixel value of IR component, and calculates the pixel value pair of each channel B Answer the second target pixel value of IR component；

According to the first object pixel value, the corresponding IR component of pixel value in each channel R is calculated, according to second mesh Pixel value is marked, the corresponding IR component of pixel value of each channel B is calculated；

According to the corresponding IR component of the pixel value in each channel R and each channel B, the first IR pixel data array is updated For the 2nd IR pixel data array；

For each channel G, according to preset second position relationship, determines and calculate from the 2nd IR pixel data array The pixel value in each channel G corresponds to the third target pixel value of IR component；

According to the third target pixel value, the corresponding IR component of pixel value in each channel G is calculated.

Optionally, each channel R of the removal, the channel G, channel B the corresponding IR component of pixel value, by first pixel Set is updated to the second pixel set, comprising:

Judge each channel R, the channel G, channel B the corresponding IR component of pixel value whether be not more than preset first IR intensity Threshold value；

If it is, according to the following formula, the pixel value of each channel R, the channel G, channel B is removed corresponding IR points respectively Amount:

RGB_0=RGB_1-TH+abs (RGB_IR-TH)

Wherein, the RGB_0 is the pixel value in the channel R after the corresponding IR component of removal, the channel G and channel B, described RGB_1 is the pixel value in each channel R, the channel G and channel B, and the TH is the 2nd IR intensity threshold, and the abs is to take absolute value Symbol, the RGB_IR are the corresponding IR component of pixel value in each channel R, the channel G and channel B.

Optionally, in the determination second pixel set each channel IR the corresponding R component of pixel value or B component, Include:

The corresponding visible light component in each channel IR is determined according to preset channel arranging rule for each channel IR, it is described It can be seen that light component is R component or B component；

For the corresponding visible light component in each channel IR, according to preset the third place relationship, from second set of pixels The 4th target pixel value for calculating the corresponding visible light component of pixel value in each channel IR is determined in conjunction；

According to the 4th target pixel value, the corresponding visible light component of pixel value in each channel IR is calculated.

Optionally, described according to the third pixel set, by the image restoring to be restored, comprising:

For in the third pixel set each channel R and each channel B, according to preset 4th positional relationship, from institute The 5th target pixel value for determining the corresponding G of pixel value, B component that calculate each channel R in third pixel set is stated, and is calculated 6th target pixel value of corresponding R, G component of the pixel value of each channel B；

According to the 5th target pixel value, the corresponding G in each channel R, B component are calculated, according to the 6th object pixel Value, calculates corresponding R, G component of each channel B；

The corresponding G in each channel R, B component and corresponding R, G component of each channel B are updated to the third set of pixels It closes；

For each channel G in updated third pixel set, according to preset 5th positional relationship, from updated The 7th target pixel value of the corresponding R of pixel value, B component that calculate each channel G are determined in third pixel set；

According to the 7th target pixel value, the corresponding R of pixel value, the B component in the channel G are calculated.

In order to achieve the above object, this application discloses a kind of image restoring device, described device includes:

Pixel obtains module, each in first pixel set for obtaining the first pixel set of image to be restored The pixel value of pixel is the pixel value obtained by the channel feux rouges R, the channel green light G, blue light channel B or the channel infrared light IR；

First component determining module, for according to the pixel value in each channel IR and preset first interpolation rule, difference to be true Each channel R in fixed first pixel set, the channel G, channel B the corresponding IR component of pixel value；

First component remove module, for remove each channel R, the channel G, channel B the corresponding IR component of pixel value, by institute It states the first pixel set and is updated to the second pixel set；

Second component determining module, the pixel value for determining each channel IR in second pixel set are R points corresponding Amount or B component, are updated to third pixel set for second pixel set；

Image restoring module is used for according to the third pixel set, by the image restoring to be restored.

Optionally, the first component determining module, comprising:

IR separates submodule, for isolating the pixel value corresponding first in each channel IR from first pixel set IR pixel data array；

First component determines submodule, for being advised according to the first IR pixel data array and first interpolation Then, determine respectively each channel R in first pixel set, the channel G, channel B the corresponding IR component of pixel value.

Optionally, first component determines submodule, comprising:

First object pixel-value determining unit is closed for being directed to each channel R and each channel B according to preset first position System determines that the pixel value for calculating each channel R corresponds to the first object pixel of IR component from the first IR pixel data array Value, and calculate the pixel value of each channel B and correspond to the second target pixel value of IR component；

First IR component calculation unit, for according to the first object pixel value, the pixel value for calculating each channel R to be corresponding IR component the corresponding IR component of pixel value of each channel B is calculated according to second target pixel value；

Data array updating unit will be described for the corresponding IR component of pixel value according to each channel R and each channel B First IR pixel data array is updated to the 2nd IR pixel data array；

Second target pixel value determination unit, for being directed to each channel G, according to preset second position relationship, from described Determine that the pixel value for calculating each channel G corresponds to the third target pixel value of IR component in 2nd IR pixel data array；

2nd IR component calculation unit, for according to the third target pixel value, the pixel value for calculating each channel G to be corresponding IR component.

Optionally, first component removes module, comprising:

Judging submodule, for judge each channel R, the channel G, channel B the corresponding IR component of pixel value whether be not more than Preset first IR intensity threshold；

First component removes submodule, for when each channel R, the channel G, the corresponding IR component of pixel value of channel B it is little When preset first IR intensity threshold, according to the following formula, the pixel value of each channel R, the channel G, channel B is removed pair respectively The IR component answered:

RGB_0=RGB_1-TH+abs (RGB_IR-TH)

Optionally, the second component determining module, comprising:

It can be seen that light component determines submodule, for determining each IR according to preset channel arranging rule for each channel IR The corresponding visible light component in channel, the visible light component are R component or B component；

First object pixel value determines submodule, for being directed to the corresponding visible light component in each channel IR, according to preset The third place relationship determines the of the corresponding visible light component of pixel value for calculating each channel IR from second pixel set Four target pixel values；

It can be seen that light component computational submodule, for calculating the pixel value in each channel IR according to the 4th target pixel value Corresponding visible light component.

Optionally, described image recovery module, comprising:

Second target pixel value determines submodule, and each channel R and each B for being directed in the third pixel set are logical Road, according to preset 4th positional relationship, determined from the third pixel set calculate each channel R the corresponding G of pixel value, 5th target pixel value of B component, and calculate the 6th target pixel value of corresponding R, G component of pixel value of each channel B；

First component computational submodule, for it is G, B point corresponding to calculate each channel R according to the 5th target pixel value Amount, according to the 6th target pixel value, calculates corresponding R, G component of each channel B；

Pixel set updates submodule, more by the corresponding G in each channel R, B component and corresponding R, G component of each channel B The new extremely third pixel set；

Third target pixel value determines submodule, each channel G for being directed in updated third pixel set, according to Preset 5th positional relationship determines that the pixel value for calculating each channel G is R, B point corresponding from updated third pixel set 7th target pixel value of amount；

Second component computational submodule, for calculating the pixel value pair in the channel G according to the 7th target pixel value R, the B component answered.

As seen from the above technical solution, in the embodiment of the present application, firstly, for the first pixel of the image to be restored obtained Set determines that each channel R, the channel G, the pixel value of channel B are corresponding in the first pixel set according to the first interpolation rule respectively IR component.Then, remove each channel R, the channel G, channel B the corresponding IR component of pixel value, the first pixel set is updated to Second pixel set.Secondly, the corresponding R component of pixel value or B component in each channel IR in the second pixel set are determined, by second Pixel set is updated to third pixel set.Finally, according to third pixel set, by the image restoring to be restored.

That is, according to the pixel value in each channel IR and the first interpolation rule, determining that each R is logical in the embodiment of the present application Road, the channel G, channel B pixel value corresponding IR component and remove, then according to each R, G after removal IR component, channel B Pixel value determines the R or B component in the channel IR, finally according to each pixel value after removal IR component, also by image to be restored It is former.As it can be seen that IR component is filtered out without hardware device using the embodiment of the present application, it can be by the IR in each channel pixel value Component removal, the color image after avoiding reduction generate cross-color due to the influence by infrared light, therefore can reduce hard Part cost.

Detailed description of the invention

In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below There is attached drawing needed in technical description to be briefly described.It should be evident that the accompanying drawings in the following description is only this Some embodiments of application for those of ordinary skill in the art without creative efforts, can be with It obtains other drawings based on these drawings.

Fig. 1 a is a kind of schematic diagram of the array of image data of conventional image sensor output；

Fig. 1 b is a kind of schematic diagram of the array of image data of RGB-IR imaging sensor output；

Fig. 2 is a kind of corresponding spectral absorption curve of RGB-IR imaging sensor；

Fig. 3 is a kind of flow diagram of image restoring method provided by the embodiments of the present application；

Fig. 4 a is the schematic diagram for the first pixel set of one kind that RGB-IR imaging sensor obtains；

Fig. 4 b is a kind of schematic diagram for the first IR pixel data array isolated from Fig. 4 a；

Fig. 4 c is by a kind of schematic diagram of the updated 2nd IR pixel data array of Fig. 4 b；

Fig. 4 d is a kind of schematic diagram of the second pixel set obtained after the IR component for removing each RGB channel；

Fig. 4 e is a kind of schematic diagram that the channel IR is replaced with to the third pixel set obtained after visible light component；

Fig. 4 f is a kind of schematic diagram of updated third pixel set；

Fig. 4 g is a kind of schematic diagram for completing the pixel set after image restoring；

Fig. 5 is according to third pixel set in Fig. 3 by a kind of flow diagram of image restoring to be restored；

Fig. 6 is a kind of structural schematic diagram of image restoring device provided by the embodiments of the present application.

Specific embodiment

Below in conjunction with the attached drawing in the embodiment of the present application, technical solutions in the embodiments of the present application carries out clear, complete Whole description.Obviously, described embodiment is only a part of the embodiment of the application, instead of all the embodiments.Base Embodiment in the application, those of ordinary skill in the art are obtained all without making creative work Other embodiments shall fall in the protection scope of this application.

The embodiment of the present application provides a kind of image restoring method and device, can reduce hardware cost.

Below by specific embodiment, the application is described in detail.

Fig. 3 is a kind of flow diagram of image restoring method provided by the embodiments of the present application, can be applied to electronics and sets Standby, which may include the non-shifting such as the mobile terminals such as video camera, smart phone, tablet computer and desktop computer Dynamic terminal.

Specifically, this method comprises the following steps:

Step S301 obtains the first pixel set of image to be restored, the pixel value of each pixel in the first pixel set For the pixel value obtained by the channel feux rouges R, the channel green light G, blue light channel B or the channel infrared light IR.

In embodiments of the present invention, the first pixel set of the available image to be restored of electronic equipment.Specifically, electronics Equipment can obtain the first pixel set from image capture device.For example, may include image capture device in electronic equipment, It can also include the elements such as imaging sensor in image capture device, thus, the available image sensor element of electronic equipment The first pixel set sent；Alternatively, the image sensor element of the available external image acquisition equipment of electronic equipment is sent The first pixel set.

In embodiments of the present invention, the imaging sensor in image capture device can be RGB-IR imaging sensor.Electricity Each channel R in the first pixel set that sub- equipment obtains, the channel G, channel B pixel value in contain infrared light component, i.e., IR component.

Fig. 1 b is a kind of pixel data array schematic diagram of RGB-IR imaging sensor output.A square in figure represents One pixel, and include a channel IR in each 2*2 array element.From Fig. 1 b it can also be seen that due to each 2*2 Channel arrangement in array element is not identical, is minimum with the rectangular 4*4 array element that each 4 pixels of ranks form therefore Cycling element, the arrangement of such channel is known as CFA (Color Filter Array, color filter array) 4*4 array.

Certainly, BGGIR and RGGIR shown in Fig. 1 b is the specific array arrangement form of RGB-IR imaging sensor.Make For another specific embodiment, RGB-IR imaging sensor can also include the array arrangements forms such as GIRBG and GIRRG, The present embodiment is not specifically limited this array arrangement form.

Fig. 4 a is the first pixel set schematic diagram that electronic equipment is obtained from RGB-IR imaging sensor, wherein containing Totally 100 pixels, the pixel value of each pixel include the pixel value in the channel R, the channel G, channel B and the channel IR to 10*10.In order to make Statement becomes apparent from, and the application is that the pixel value in each channel in Fig. 4 a is assigned with one from 00~99 number.

Step S302 determines the first set of pixels according to the pixel value in each channel IR and preset first interpolation rule respectively Each channel R in conjunction, the channel G, channel B the corresponding IR component of pixel value.

As a kind of specific embodiment, in the present embodiment, step S302, i.e., according to the pixel value in each channel IR and pre- If the first interpolation rule, determine that each channel R, the channel G, the pixel value of channel B are corresponding in first pixel set respectively IR component may include:

Step 1, the corresponding first IR pixel data array of pixel value in each channel IR is isolated from the first pixel set.

Specifically, the first IR pixel data array can be obtained in the following ways:

R, G in first pixel set, channel B position pixel value set the pixel value of the channel 0, IR position and protect It stays, obtains the first IR pixel data array.

For the first pixel set shown in Fig. 4 a, the first IR pixel data array therefrom isolated in the manner described above It as shown in Figure 4 b, only include the pixel value in the channel IR in Fig. 4 b.

Step 2, it according to the first IR pixel data array and the first interpolation rule, determines respectively each in the first pixel set The channel R, the channel G, channel B the corresponding IR component of pixel value.

As a kind of specific embodiment, above-mentioned steps 2, i.e., according to the first IR pixel data array and described First interpolation rule determines that each channel R in first pixel set, the channel G, the pixel value of channel B are IR points corresponding respectively It measures, may include:

Step 2a, for each channel R and each channel B, according to preset first position relationship, from the first IR pixel data battle array It determines that the pixel value for calculating each channel R corresponds to the first object pixel value of IR component in column, and calculates the pixel value of each channel B Second target pixel value of corresponding IR component.

That is, calculating the IR component of the pixel value in the channel R in the present embodiment and calculating the IR of the pixel value of channel B The process of component be it is similar, require according to first position relationship, from the first IR pixel data array determine for calculating The target pixel value of IR component.

In order to keep statement easy, the IR component of the pixel value of channel B is indicated with IR/B, the pixel in the channel R is indicated with IR/R The IR component of value.The form of presentation of subsequent X/Y can be understood as indicating the X-component of the pixel value in the channel Y.

Specifically, when determining the IR/R in the first pixel set middle position, it can be centered on the position in the channel R, it will The channel the R upper left corner in first IR pixel data array, the lower left corner, the upper right corner and the lower right corner the pixel value in the channel IR be determined as One target pixel value.When determining the IR/B in the first pixel set middle position, centered on the position of channel B, by the first IR The channel B upper left corner in pixel data array, the lower left corner, the upper right corner and the lower right corner the pixel value in the channel IR be determined as the second target Pixel value.

It is illustrated for determining the IR component of pixel value of channel B below.It, can be with for example, when calculating IR/B44 From the first IR pixel data array shown in Fig. 4 b, determine that the IR in the upper left corner of B44, the lower left corner, the upper right corner and the lower right corner is logical The pixel value in road, i.e. IR33, IR53, IR35 and IR55 are the second target pixel value.

Step 2b calculates the corresponding IR component of pixel value in each channel R, according to the second mesh according to first object pixel value Pixel value is marked, the corresponding IR component of pixel value of each channel B is calculated.

As a kind of specific embodiment, IR/B and IR/R can be calculated in the following ways:

Obtain n_diff=abs (IR_{The upper left corner}–IR_{The lower right corner}) and p_diff=abs (IR_{The lower left corner}–IR_{The upper right corner})；

As n_diff > p_diff, IR/B and IR/R are determined as (IR_{The lower left corner}+IR_{The upper right corner})/2；

As n_diff < p_diff, IR/B and IR/R are determined as (IR_{The upper left corner}+IR_{The lower right corner})/2；

As n_diff=p_diff, IR/B and IR/R are determined as (IR_{The upper left corner}+IR_{The lower right corner}+IR_{The lower left corner}+IR_{The upper right corner})/4。

Wherein, abs is the symbol that takes absolute value, IR_{The upper left corner}、IR_{The lower right corner}、IR_{The lower left corner}And IR_{The upper right corner}Respectively the first IR pixel data battle array The pixel value in each channel R and each channel B position upper left corner, the lower right corner, the lower left corner and the upper right corner in column.

For example, the process for calculating IR/B44 is as follows according to Fig. 4 a and Fig. 4 b:

N_diff=abs (IR33-IR55)；

P_diff=abs (IR53-IR35)；

If n_diff > p_diff, IR/B44=(IR53+IR35)/2；

If n_diff < p_diff, IR/B44=(IR33+IR55)/2；

If n_diff=p_diff, IR/B44=(IR33+IR55+IR53+IR35)/4.

Step 2c, according to the corresponding IR component of the pixel value in each channel R and each channel B, by the first IR pixel data array It is updated to the 2nd IR pixel data array.

It, can be by each channel R and each channel B after determining the corresponding IR component of pixel value in each channel R and each channel B The corresponding IR component of pixel value be added at the first corresponding position of IR pixel data array, obtain the 2nd IR pixel data battle array Column.

For example, Fig. 4 b is the first IR pixel data array, Fig. 4 c is that the pixel value in each channel R and each channel B is corresponding IR component is updated to the 2nd IR pixel data array obtained after the first IR pixel data array shown in Fig. 4 b.That is, IR pixel data shown in Fig. 4 c includes the pixel value in the channel IR and the IR component of each R, channel B.

Step 2d, according to preset second position relationship, is determined for each channel G from the 2nd IR pixel data array The pixel value for calculating each channel G corresponds to the third target pixel value of IR component.

Specifically, when determining the IR/G in the first pixel set middle position, it can be centered on the position in the channel G, it will The channel G left side in 2nd IR pixel data array, right side, above and below IR pixel data be determined as third object pixel Value, the third target pixel value include the pixel value in the channel IR and the IR component in the channel R, G.

For example, can determine the channel G45 from the 2nd IR pixel data array shown in Fig. 4 c when calculating IR/G45 Left side, right side, above and below IR pixel value, i.e., IR44, IR46, IR35 and IR55 be third target pixel value.

Step 2e calculates the corresponding IR component of pixel value in each channel G according to the third target pixel value.

As a kind of specific embodiment, IR/G can be calculated in the following ways, and IR/G indicates the pixel value in the channel G Corresponding IR component:

Obtain h_diff=abs (IR_{Left side}–IR_{Right side}) and v_diff=abs (IR_Top–IR_{Lower section})；

As h_diff > v_diff, IR/G is determined as (IR_{Left side}+IR_{Right side})/2；

As h_diff < v_diff, IR/G is determined as (IR_Top+IR_{Lower section})/2；

As h_diff=v_diff, IR/G is determined as (IR_{Left side}+IR_{Right side}+IR_Top+IR_{Lower section})/4。

Wherein, abs is the symbol that takes absolute value, IR_{Left side}、IR_{Right side}、IR_TopAnd IR_{Lower section}In respectively the 2nd IR pixel data array The pixel value position left side in the channel G, right side, above and below element.

For example, the process for calculating IR/G45 is as follows according to Fig. 4 a and Fig. 4 c:

H_diff=abs (IR44-IR46)；

V_diff=abs (IR35-IR55)；

If h_diff > v_diff, IR/G45=(IR44+IR46)/2；

If v_diff < h_diff, IR/G45=(IR35+IR55)/2；

If h_diff=v_diff, IR/G45=(IR44+IR46+IR35+IR55)/4.

It should be noted that when calculating IR/R, IR/B and IR/G of the first pixel set marginal position, as a kind of tool The embodiment of body can determine the pixel value in the channel IR in the 2*2 array element where each channel R, channel B and the channel G For the IR component in each channel R, channel B and the channel G.

For example, in fig.4, B00, G01, G10, IR11 collectively constitute a 2*2 array element, and the array element position The upper left corner of the pixel set shown in Fig. 4 a, the position belong to marginal position.It is logical according to each channel R of above-mentioned calculating, channel B and G The pixel value in the channel IR11 can be determined as IR/B and IR/G by the process of the IR component in road.

Step S303, remove each channel R, the channel G, channel B the corresponding IR component of pixel value, more by the first pixel set It is newly the second pixel set.

It should be noted that step S303 may include numerous embodiments, directly each channel R, the channel G, B can be led to The pixel value in road removes corresponding IR component.

But if the brightness of IR component has been more than certain threshold value, illustrates that there may be overexposure phenomenons, at this moment will The pixel value in corresponding channel directly removes the loss that IR component will cause visible channel information.The brightness of this IR component is super It crosses a certain range of it may be the case that as including caused by stronger light energy in target image collected.

In order to avoid channel information loss, the accuracy of image restoring result is improved, as a kind of specific embodiment, In the present embodiment, step S303, that is, remove each channel R, the channel G, channel B the corresponding IR component of pixel value, by the first picture Element set is updated to the second pixel set, may include:

Judge each channel R, the channel G, channel B the corresponding IR component of pixel value whether be not more than preset first IR intensity Threshold value；If it is not, then be not processed, if it is, according to the following formula, by the pixel value of each channel R, the channel G, channel B point Corresponding IR component is not removed:

RGB_0=RGB_1-TH+abs (RGB_IR-TH)

Wherein, RGB_0 is the pixel value in the channel R after the corresponding IR component of removal, the channel G and channel B, and RGB_1 is The pixel value in each channel R, the channel G and channel B, TH are the 2nd IR intensity threshold, and abs is the symbol that takes absolute value, and RGB_IR is each R The corresponding IR component of the pixel value in channel, the channel G and channel B.

Wherein, the first IR intensity threshold can there are certain relationships with the 2nd IR intensity threshold, for example, the first IR intensity Threshold value can be the multiple of the default value of the 2nd IR intensity threshold, and preset quantity can take 2 or 3 etc numerical value.Above-mentioned Two IR intensity thresholds can be preset value, and e.g., for the pixel data width of 12bit, the 2nd IR intensity threshold TH be can be set to 512 or other values.The application is not specifically limited the value of the first IR intensity threshold and the 2nd IR intensity threshold.

For example, by the corresponding removal of pixel value in each channel R, the channel G, channel B in the first pixel set shown in Fig. 4 a Corresponding IR component obtains the second pixel set as shown in figure 4d, and in order to become apparent from statement, the application uses band in figure 4d The pixel value of apostrophe " ' " indicates to eliminate the pixel value of R, G of infrared light component, channel B.

Step S304 determines the corresponding R component of pixel value or B component in each channel IR in the second pixel set, by second Pixel set is updated to third pixel set.

It should be noted that the pixel value of R, G, channel B in the second pixel set have had been removed corresponding IR points Amount.

Since in the corresponding pixel data array of RGB-IR imaging sensor, the channel IR is instead of traditional image sensing The channel R or channel B in device, therefore, in order to combine traditional images restoring method, need to obtain all positions visible light R, G, Channel B pixel value, i.e., also it needs to be determined that the corresponding R component of pixel value or B component in the channel IR in the second pixel set.

As a kind of specific embodiment, in the present embodiment, step S304 determines each IR in the second pixel set The corresponding R component of the pixel value in channel or B component may include:

Step 1, the corresponding visible light point in each channel IR is determined according to preset channel arranging rule for each channel IR Amount, the visible light component are R component or B component.

Wherein, channel arranging rule can be the regular array being alternately present according to BGGIR shown in Fig. 1 b and RGGIR, It is also possible to the regular array being alternately present according to GIRBG and GIRRG, specific channel arranging rule can be according to the actual situation It determines.

Step 2, for the corresponding visible light component in each channel IR, according to preset the third place relationship, from the second pixel The 4th target pixel value for calculating the corresponding visible light component of pixel value in each channel IR is determined in set.

Specifically, can be with the position in the channel IR when determining the visible light component in the second pixel set middle position Center, by the channel IR top in the second pixel set, lower section, left side, right side the channel G pixel value and the upper right corner, lower-left The channel R at angle or the pixel value of channel B, are determined as the 4th target pixel value.Wherein, the pixel value in the channel G, the channel R and channel B For the pixel value after the pixel value removal IR component in the channel G, the channel R and channel B.

Below to be illustrated for the R component (i.e. R/IR) for determining the channel IR.It is understood that according to above-mentioned removal The R component that the 4th target pixel value after IR component determines is also without containing IR component.

For example, can determine the 4th target pixel value from the second pixel set shown in Fig. 4 d when calculating R/IR55 Are as follows: the top IR55, lower section, left side, right side the channel G pixel value, i.e. G ' 45, G ' 65, G ' 54 and G ' 56 and IR55 upper right Angle, the lower left corner the channel R pixel value, i.e. R ' 46 and R ' 64.

Step 3, according to the 4th target pixel value, the corresponding visible light component of pixel value in each channel IR is calculated.

As a kind of specific embodiment, R/IR or B/IR can be calculated in the following ways:

Obtain h_h_laplace=abs (2*G_Top–G_{The top of top}–G_{Lower section}),

H_v_laplace=abs (2*G_Top–G_{The left side of top}–G_{The right side of top}),

V_h_laplace=abs (2*G_{Left side}–G_{The left side in left side}–G_{Right side}) and

V_v_laplace=abs (2*G_{Left side}–G_{The top in left side}–G_{The lower section in left side})；

For R/IR,

If h_h_laplace < h_v_laplace, R/IR=(G_Top+G_{Lower section})*(R_{The upper right corner}+R_{The upper left corner})/[2*(G_{Left side}+ G_{Right side})]；

If v_h_laplace > v_v_laplace, R/IR=(G_{Left side}+G_{Right side})*(R_{The upper right corner}+R_{The upper left corner})/[2*(G_Top+ G_{Lower section})]；

If above-mentioned two condition is all unsatisfactory for, R/IR=(R_{The upper right corner}+R_{The upper left corner})/2；

For B/IR,

If h_h_laplace < h_v_laplace, B/IR=(G_Top+G_{Lower section})*(B_{The upper right corner}+B_{The upper left corner})/[2*(G_{Left side}+ G_{Right side})]；

If v_h_laplace > v_v_laplace, B/IR=(G_{Left side}+G_{Right side})*(B_{The upper right corner}+B_{The upper left corner})/[2*(G_Top+ G_{Lower section})]；

If above-mentioned two condition is all unsatisfactory for, B/IR=(B_{The upper right corner}+B_{The upper left corner})/2。

Wherein, abs is the symbol that takes absolute value, G_Top、G_{Lower section}、G_{Left side}And G_{Right side}The respectively channel IR top, lower section, left side, the right side The pixel value in the channel G of side, R_{The upper right corner}And R_{The upper left corner}The respectively pixel value in the channel the R upper right corner, the lower left corner, B_{The upper right corner}And B_{The upper left corner}Respectively The pixel value in the channel B upper right corner, the lower left corner, G_{The top of top}、G_{The left side of top}And G_{The right side of top}The respectively G in the channel G_TopTop, left side and The pixel value on right side, G_{The top in left side}、G_{The left side in left side}And G_{The lower section in left side}The respectively G in the channel G_{Left side}Top, left side and lower section pixel value.

For example, the second pixel set according to shown in Fig. 4 d, the process for calculating R/IR55 are as follows:

H_h_laplace=abs (45-G ' of 2*G ', 25-G ' 65)；

H_v_laplace=abs (45-G ' of 2*G ', 43-G ' 47)；

V_h_laplace=abs (54-G ' of 2*G ', 52-G ' 56)；

V_v_laplace=abs (54-G ' of 2*G ', 34-G ' 74)；

H_h_laplace < h_v_laplace, R/IR55=if (G ' 45+G ' 65) * (R ' 46+R ' 64)/[2* (G ' 54+ G′56)]；

V_h_laplace > v_v_laplace, R/IR55=if (G ' 54+G ' 56) * (R ' 46+R ' 64)/[2* (G ' 45+ G′65)]；

If above-mentioned condition is not satisfied, R/IR55=(R ' 46+R ' 64)/2.

It should be noted that when calculating R/IR and B/IR of the second pixel set marginal position, as a kind of specific Embodiment can determine as follows:

For the second pixel set leftmost side one column and the top a line 2*2 array element in the channel IR, R/IR and B/IR is identified as: the channel the IR lower left corner and the upper right cornerChannel orThe average value of the pixel value in channel.For example, scheming In 4d, R/IR11 is (R ' 02+R ' 20)/2.

For the channel IR in the column 2*2 array element of the second pixel set rightmost side one, R/IR and B/IR are identified as: The channel the IR position lower left cornerChannel orThe pixel value in channel.For example, in figure 4d, B/IR39 is taken as the picture of B ' 48 Element value；

For the channel IR in second pixel set bottom a line 2*2 array element, R/IR and B/IR are identified as: The channel the IR position upper right cornerChannel orThe pixel value in channel.For example, in figure 4d, B/IR97 is taken as the picture of B ' 88 Element value；

For the channel IR in a 2*2 array element of the second pixel set last cell, R/IR and B/IR difference are true It is set to: the visible light component in the channel IR in the 2*2 array element in the channel the IR position upper left corner.For example, in figure 4d, R/ IR99 is taken as the pixel value of R ' 77.

After determining the visible light component of the pixel value in the channel IR, the second pixel set shown in Fig. 4 d can be updated For third pixel set shown in Fig. 4 e.Wherein, each channel IR has been replaced with R component and B component, and each pixel is not wrapped Component containing IR.In order to become apparent from statement, in figure 4e, R component and B component for replacing the channel IR pixel value use band respectively The amount of apostrophe " ' " indicates, for example, R ' 11 is exactly the R component pixel value in the channel IR.

Step S305, according to third pixel set, by image restoring to be restored.

It should be noted that due in third pixel set each location of pixels include the infrared light component of removal Visible channel information, in order to by the corresponding image restoring to be restored of third pixel set, it is thus necessary to determine that in third pixel set Other two visible channel information of each location of pixels.

It, can also be with specifically, by image restoring to be restored, may include numerous embodiments according to third pixel set Using interpolation algorithm in the prior art by image restoring to be restored, specific implementation process the present embodiment is no longer described in detail.

As shown in the above, in the present embodiment, according to the pixel value in each channel IR and the first interpolation rule, each R is determined Channel, the channel G, channel B the corresponding IR component of pixel value and remove, then according to removal IR component after each R, G, channel B Pixel value, determine the R or B component in the channel IR, finally according to removal IR component after each pixel value, also by image to be restored It is former.As it can be seen that IR component is filtered out without hardware device using the present embodiment, it can be by the IR component in each channel pixel value Removal, avoid reduction after color image cross-color is generated due to the influence by infrared light, therefore can reduce hardware at This.

In another embodiment herein, embodiment illustrated in fig. 3 can be improved.In order to more efficient, accurate Ground is by image restoring to be restored, step S305, i.e.,, can be with by the image restoring to be restored according to the third pixel set It is carried out according to flow diagram shown in Fig. 5, specifically includes the following steps:

Step S501, in third pixel set each channel R and each channel B, according to preset 4th positional relationship, The 5th target pixel value of the corresponding G of pixel value, B component that calculate each channel R are determined from third pixel set, and are calculated 6th target pixel value of corresponding R, G component of the pixel value of each channel B.

That is, calculating the corresponding G of pixel value, B component and the picture for calculating each channel B in each channel R in the present embodiment The process of corresponding R, G component of plain value be it is similar, require according to the 4th positional relationship, the determination mesh from third pixel set Mark pixel value.

Specifically, when determining the G/R in third pixel set middle position, it can be centered on the position in the channel R, by The channel R top in three pixel sets, lower section, left side, right side the pixel value in the channel G be determined as the 5th target pixel value；True , can be centered on the position in the channel R when determining the B/R in third pixel set middle position, it will be in third pixel set on the channel R The pixel value of the channel B of side, lower section, left side, right side is determined as the 5th target pixel value.

It, can be centered on the position of channel B, by third pixel when determining the G/B in third pixel set middle position Channel B top in set, lower section, left side, right side the pixel value in the channel G be determined as the 6th target pixel value；Determining third When the R/B in pixel set middle position, can centered on the position of channel B, by third pixel set above channel B, under Side, left side, right side the pixel value in the channel R be determined as the 6th target pixel value.

For example, in the third pixel set shown in Fig. 4 e, when calculating G/B44, the 6th determining target pixel value are as follows: The top B44, lower section, left side, right side the channel R pixel value, i.e. G ' 34, G ' 54, G ' 43 and G ' 45.

Step S502 calculates the corresponding G in each channel R, B component, according to the 6th object pixel according to the 5th target pixel value Value, calculates corresponding R, G component of each channel B.

As a kind of specific embodiment, G/B, G/R or B/R, R/B can be calculated in the following ways:

Obtain h_h_laplace=abs (G_{The top of top}–G_{Lower section})+abs(2*G_Top–G_{The top of top}–G_{Lower section}),

H_v_laplace=abs (G_{The left side of top}–G_{The right side of top})+abs(2*G_Top–G_{The left side of top}–G_{The right side of top}),

V_h_laplace=abs (G_{The top in left side}–G_{The lower section in left side})+abs(2*G_{Left side}–G_{The top in left side}–G_{The lower section in left side}) and

V_v_laplace=abs (G_{The left side in left side}–G_{Right side})+abs(2*G_{Left side}–G_{The left side in left side}–G_{Right side})；

For G/B and G/R,

If h_h_laplace < h_v_laplace, G/B and G/R are (G_Top+G_{Lower section})/2；

If v_h_laplace > v_v_laplace, G/B and G/R are (G_{Left side}+G_{Right side})/2；

If above-mentioned condition is not satisfied, G/B and G/R are (G_Top+G_{Lower section}+G_{Left side}+G_{Right side})/4；

For B/R,

If h_h_laplace < h_v_laplace, B/R=(B_Top+B_{Lower section})/2；

If v_h_laplace > v_v_laplace, B/R=(B_{Left side}+B_{Right side})/2；

If above-mentioned condition is not satisfied, B/R=(B_Top+B_{Lower section}+B_{Left side}+B_{Right side})/4；

For R/B,

If h_h_laplace < h_v_laplace, R/B=(R_Top+R_{Lower section})/2；

If v_h_laplace > v_v_laplace, R/B=(R_{Left side}+R_{Right side})/2；

If above-mentioned condition is not satisfied, R/B=(R_Top+R_{Lower section}+R_{Left side}+R_{Right side})/4。

Wherein, abs is the symbol that takes absolute value, G_Top、G_{Lower section}、G_{Left side}And G_{Right side}The respectively channel R or channel B top, lower section, a left side Side, right side the channel G pixel value, B_Top、B_{Lower section}、B_{Left side}And B_{Right side}The respectively channel B of the channel R top, lower section, left side, right side Pixel value, R_Top、R_{Lower section}、R_{Left side}And R_{Right side}Respectively channel B top, lower section, left side, right side the channel R pixel value. G_{The top of top}、G_{The left side of top}And G_{The right side of top}The respectively G of the channel R or channel B_TopTop, left and right side pixel value, G_{The top in left side}、 G_{The left side in left side}And G_{The lower section in left side}The respectively G of the channel R or channel B_{Left side}Top, left side and lower section pixel value.

For example, the third pixel set according to shown in Fig. 4 e, the process for calculating G '/B ' 44 and R '/B ' 44 is as follows:

H_h_laplace=abs (14-G ' 54 of G ')+abs (34-G ' of 2*G ', 14-G ' 54),

H_v_laplace=abs abs (32-G ' 36 of G ')+abs (34-G ' of 2*G ', 32-G ' 36),

V_h_laplace=abs (23-G ' 63 of G ')+abs (43-G ' of 2*G ', 23-G ' 63) and

V_v_laplace=abs (41-G ' 45 of G ')+abs (43-G ' of 2*G ', 41-G ' 45)

For G '/B ' 44,

If h_h_laplace < h_v_laplace, G '/B ' 44 are (G ' 34+G ' 54)/2；

If v_h_laplace > v_v_laplace, G '/B ' 44 are (G ' 43+G ' 45)/2；

If above-mentioned condition is not satisfied, G '/B ' 44 is (G ' 43+G ' 45+G ' 34+G ' 54)/4；

For R '/B ' 44,

If h_h_laplace < h_v_laplace, R '/B ' 44=(R ' 34+R ' 54)/2；

If v_h_laplace > v_v_laplace, R '/B ' 44=(R ' 43+R ' 45)/2；

If above-mentioned condition is not satisfied, R '/B ' 44=(R ' 43+R ' 45+R ' 34+R ' 54)/4.

It should be noted that in G/B, G/R or B/R, the R/B for calculating third pixel set marginal position, as one kind Specific embodiment can determine as follows:

For the 2*2 array element of third pixel set marginal position, G/B, G/R are identified as in the array element: The average value of the pixel value in two channels G in the array element.For example, in the third pixel set shown in Fig. 4 e, G '/B ' 00 =G '/R ' 11=(G ' 01+G ' 10)/2.

For the 2*2 array element of third pixel set marginal position, B/R is determined in the array element are as follows: the array list The pixel value in the channel G in member；R/B is determined in the array element are as follows: the pixel value in the channel R in the array element.

The corresponding G in each channel R, B component and corresponding R, G component of each channel B are updated to third picture by step S503 Element set.

Fig. 4 f is a kind of schematic diagram of updated third pixel set.The corresponding G in each channel R, B component in figure and Corresponding R, G component of each channel B has obtained, and in order to become apparent from statement, is indicated in Fig. 4 f with the amount of the double apostrophes " 〞 " of band The channel B and the channel R of other two components are obtained, such as.B 〞 00 indicate the channel have been obtained for R 〞 00 and G 〞 00 this Two components.

Step S504, for each channel G in updated third pixel set, according to preset 5th positional relationship, The 7th target pixel value of the corresponding R of pixel value, B component that calculate each channel G are determined from updated third pixel set.

Specifically, in determining updated third pixel set when the B/G in middle position, it can be with the position in the channel G Centered on, the pixel value of the channel B of the channel G top, lower section, left side, right side in updated third pixel set is determined as 7th target pixel value.It, can be with the position in the channel G in determining updated third pixel set when the R/G in middle position Centered on, the pixel value of the channel B of the channel R top, lower section, left side, right side in updated third pixel set is determined as 7th target pixel value.

For example, when calculating 54 B/G ', determining the 7th target pixel value in Fig. 4 f are as follows: 54 top G ', lower section, left side, The pixel value of the channel B on right side, i.e. B 〞 44, B 〞 64, B 〞 53 and B 〞 55.When calculating 54 R/G ', the 7th target pixel value is determined Are as follows: 54 top G ', lower section, left side, right side the channel R pixel value, i.e. R 〞 44, R 〞 64, R 〞 53 and R 〞 55.

Step S505 calculates the corresponding R of pixel value, the B component in the channel G according to the 7th target pixel value.

As a kind of specific embodiment, B/G and R/G can be calculated in the following ways:

Obtain h_h_diff=abs (G_{The top of top}–G_{The lower section of lower section})+abs(G–G_{The top of top}–G_{The lower section of lower section}),

H_v_diff=abs (G_{The left side in left side}–G_{The right side on right side}+abs(G–G_{The left side in left side}–G_{The right side on right side})；

For B/G,

If h_h_diff < h_v_diff, B/G=(B_Top+B_{Lower section})/2；

If h_h_diff > h_v_diff, B/G=(B_{Left side}+B_{Right side})/2；

If above-mentioned condition is not satisfied, B/G=(B_Top+B_{Lower section}+B_{Left side}+B_{Right side})/4；

For R/G,

If h_h_diff < h_v_diff, R/G=(R_Top+R_{Lower section})/2；

If h_h_diff > h_v_diff, R/G=(R_{Left side}+R_{Right side})/2；

If above-mentioned condition is not satisfied, R/G=(R_Top+R_{Lower section}+R_{Left side}+R_{Right side})/4。

Wherein, abs is the symbol that takes absolute value, B_Top、B_{Lower section}、B_{Left side}And B_{Right side}The respectively channel G top, lower section, left side, right side Channel B pixel value, R_Top、R_{Lower section}、R_{Left side}And R_{Right side}Respectively the channel G top, lower section, left side, right side the channel R pixel Value.G_{The top of top}、G_{The lower section of lower section}、G_{The left side in left side}And G_{The right side on right side}Top, the lower section of lower section, the left side in left side, the right side respectively above the channel G The G component on the right side of side.

For example, updated third pixel set is as shown in fig. 4f, the process for calculating B 〞/G ' 54 and R 〞/G ' 54 is as follows:

H_h_diff=abs (34-G ' 74 of G ')+abs (54-G ' of G ', 34-G ' 74) is obtained,

H_v_diff=abs (52-G ' 56 of G ')+abs (54-G ' of G ', 52-G ' 56)；

For B 〞/G ' 54,

If h_h_diff < h_v_diff, B 〞/G ' 54=(B 〞 44+B 〞 64)/2；

If h_h_diff > h_v_diff, B 〞/G ' 54=(B 〞 53+B 〞 55)/2；

If above-mentioned condition is not satisfied, B 〞/G ' 54=(B 〞 44+B 〞 64+B 〞 53+B 〞 55)/4；

For R 〞/G ' 54,

If h_h_diff < h_v_diff, R 〞/G ' 54=(R 〞 44+R 〞 64)/2；

If h_h_diff > h_v_diff, R 〞/G ' 54=(R 〞 53+R 〞 55)/2；

If above-mentioned condition is not satisfied, R 〞/G ' 54=(R 〞 44+R 〞 64+R 〞 53+R 〞 55)/4.

It should be noted that when calculating B/G and R/G of updated third pixel set marginal position, as one kind Specific embodiment can determine as follows:

For the 2*2 array element of updated third pixel set marginal position, by other B components in the array element Or R component is determined as B/G or R/G in the array element.For example, updated third pixel set is as shown in fig. 4f, by B 〞 00 Pixel value be determined as B 〞/G ' 01 and B 〞/G ' 10, the pixel value of R 〞 11 is determined as R 〞/G ' 01 and R 〞/G ' 10.

As an example, Fig. 4 g is the R component for calculating the channel G in updated third pixel set shown in Fig. 4 f With the pixel set after B component, it is the colour for eliminating IR component that each channel, which contains RGB component, in the pixel set Image, i.e. Fig. 4 g are the images after completing image restoring.

Fig. 6 is a kind of structural schematic diagram of image restoring device provided by the embodiments of the present application, is implemented with method shown in Fig. 1 Example is corresponding, and described device includes:

Pixel obtains module 601, for obtaining the first pixel set of image to be restored, in first pixel set The pixel value of each pixel is the pixel value obtained by the channel feux rouges R, the channel green light G, blue light channel B or the channel infrared light IR；

First component determining module 602, for according to the pixel value in each channel IR and preset first interpolation rule, difference Determine each channel R in first pixel set, the channel G, channel B the corresponding IR component of pixel value；

First component remove module 603, for remove each channel R, the channel G, channel B the corresponding IR component of pixel value, First pixel set is updated to the second pixel set；

Second component determining module 604, for determining the corresponding R of pixel value in each channel IR in second pixel set Second pixel set is updated to third pixel set by component or B component；

Image restoring module 605 is used for according to the third pixel set, by the image restoring to be restored.

As a kind of specific embodiment, in the embodiment shown in fig. 6, the first component determining module 602 can be with Include:

IR separates submodule (not shown), for isolating the pixel in each channel IR from first pixel set It is worth corresponding first IR pixel data array；

First component determines submodule (not shown), for according to the first IR pixel data array and described First interpolation rule determines that each channel R in first pixel set, the channel G, the pixel value of channel B are IR points corresponding respectively Amount.

As a kind of specific embodiment, in the embodiment shown in fig. 6, first component determines submodule, can be with Include:

First object pixel-value determining unit (not shown), for being directed to each channel R and each channel B, according to default First position relationship, determine that the pixel value for calculating each channel R corresponds to IR component from the first IR pixel data array First object pixel value, and calculate the pixel value of each channel B and correspond to the second target pixel value of IR component；

First IR component calculation unit (not shown), for calculating each channel R according to the first object pixel value The corresponding IR component of pixel value the corresponding IR component of pixel value of each channel B is calculated according to second target pixel value；

Data array updating unit (not shown), for the corresponding IR of pixel value according to each channel R and each channel B The first IR pixel data array is updated to the 2nd IR pixel data array by component；

Second target pixel value determination unit (not shown), for being directed to each channel G, according to the preset second position Relationship determines that the pixel value for calculating each channel G corresponds to the third target picture of IR component from the 2nd IR pixel data array Element value；

2nd IR component calculation unit (not shown), for calculating each channel G according to the third target pixel value The corresponding IR component of pixel value.

As a kind of specific embodiment, in the embodiment shown in fig. 6, first component removes module 603, can be with Include:

Judging submodule (not shown), for judging that each channel R, the channel G, the pixel value of channel B are IR points corresponding Whether amount is not more than preset first IR intensity threshold；

First component removes submodule (not shown), for corresponding to when the pixel value of each channel R, the channel G, channel B IR component be not more than preset first IR intensity threshold when, according to the following formula, by each channel R, the channel G, channel B pixel Value removes corresponding IR component respectively:

RGB_0=RGB_1-TH+abs (RGB_IR-TH)

As a kind of specific embodiment, in the embodiment shown in fig. 6, the second component determining module 604 can be with Include:

It can be seen that light component determines submodule (not shown), for being directed to each channel IR, arranged according to preset channel Rule determines that the corresponding visible light component in each channel IR, the visible light component are R component or B component；

First object pixel value determines submodule (not shown), for for the corresponding visible light point in each channel IR Amount determines that the pixel value for calculating each channel IR is corresponding from second pixel set according to preset the third place relationship It can be seen that the 4th target pixel value of light component；

It can be seen that light component computational submodule (not shown), for calculating each IR according to the 4th target pixel value The corresponding visible light component of the pixel value in channel.

As a kind of specific embodiment, in the embodiment shown in fig. 6, described image recovery module 605, comprising:

Second target pixel value determines submodule (not shown), for for each R in the third pixel set Channel and each channel B determine the picture for calculating each channel R according to preset 4th positional relationship from the third pixel set Element is worth the 5th target pixel value of corresponding G, B component, and calculate each channel B corresponding R, G component of pixel value the 6th Target pixel value；

First component computational submodule (not shown), for calculating each channel R according to the 5th target pixel value Corresponding G, B component calculate corresponding R, G component of each channel B according to the 6th target pixel value；

Pixel set updates submodule (not shown), and the corresponding G in each channel R, B component and each channel B is corresponding R, G component be updated to the third pixel set；

Third target pixel value determines submodule (not shown), for in updated third pixel set Each channel G determines the pixel for calculating each channel G according to preset 5th positional relationship from updated third pixel set It is worth the 7th target pixel value of corresponding R, B component；

Second component computational submodule (not shown), for it is logical to calculate the G according to the 7th target pixel value The corresponding R of the pixel value in road, B component.

Since above-mentioned apparatus embodiment is obtained based on embodiment of the method, and this method technical effect having the same, Therefore details are not described herein for the technical effect of Installation practice.

For device embodiment, since it is substantially similar to the method embodiment, related so describing fairly simple Place illustrates referring to the part of embodiment of the method.

It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or any other variant be intended to it is non- It is exclusive to include, so that the process, method, article or equipment for including a series of elements not only includes those elements, It but also including other elements that are not explicitly listed, or further include solid by this process, method, article or equipment Some elements.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including There is also other identical elements in the process, method, article or equipment of the element.

It will appreciated by the skilled person that all or part of the steps in above embodiment is can to pass through journey What sequence instructed relevant hardware to complete, the program can store in computer-readable storage medium.It is designated herein Storage medium refers to ROM/RAM, magnetic disk, CD etc..

The foregoing is merely the preferred embodiments of the application, are not intended to limit the protection scope of the application.It is all Any modification, equivalent substitution, improvement and etc. done within spirit herein and principle are all contained in the protection scope of the application It is interior.

Claims

1. a kind of image restoring method, which is characterized in that the described method includes:

The first pixel set of image to be restored is obtained, the pixel value of each pixel in first pixel set is to pass through feux rouges The pixel value that the channel R, the channel green light G, blue light channel B or the channel infrared light IR obtain；Wherein, the R in the first pixel set is logical Road, the channel G, channel B pixel value in include IR component；

According to the pixel value in each channel IR and preset first interpolation rule, determine that each R is logical in first pixel set respectively Road, the channel G, channel B the corresponding IR component of pixel value；

The corresponding R component of pixel value or B component for determining each channel IR in second pixel set, by second set of pixels Conjunction is updated to third pixel set；

According to the third pixel set, by the image restoring to be restored.

2. the method according to claim 1, wherein described according to the pixel value in each channel IR and preset first Interpolation rule, determine respectively each channel R in first pixel set, the channel G, channel B the corresponding IR component of pixel value, packet It includes:

According to the first IR pixel data array and first interpolation rule, determined in first pixel set respectively Each channel R, the channel G, channel B the corresponding IR component of pixel value.

3. according to the method described in claim 2, it is characterized in that, described according to the first IR pixel data array and institute State the first interpolation rule, determine respectively each channel R in first pixel set, the channel G, channel B the corresponding IR of pixel value Component, comprising:

For each channel R and each channel B, according to preset first position relationship, from the first IR pixel data array really The pixel value that devise a stratagem calculates each channel R corresponds to the first object pixel value of IR component, and the pixel value of each channel B of calculating corresponds to IR Second target pixel value of component；

According to the first object pixel value, the corresponding IR component of pixel value in each channel R is calculated, according to second target picture Element value, calculates the corresponding IR component of pixel value of each channel B；

According to the corresponding IR component of the pixel value in each channel R and each channel B, the first IR pixel data array is updated to Two IR pixel data arrays；

For each channel G, according to preset second position relationship, is determined from the 2nd IR pixel data array and calculate each G The pixel value in channel corresponds to the third target pixel value of IR component；

4. the method according to claim 1, wherein each channel R of the removal, the channel G, channel B pixel value First pixel set is updated to the second pixel set by corresponding IR component, comprising:

If it is, according to the following formula, the pixel value of each channel R, the channel G, channel B is removed corresponding IR component respectively:

RGB_0=RGB_1-TH+abs (RGB_IR-TH)

Wherein, the RGB_0 is the pixel value in the channel R after the corresponding IR component of removal, the channel G and channel B, the RGB_ 1 is the pixel value in each channel R, the channel G and channel B, and the TH is the 2nd IR intensity threshold, and the abs is the symbol that takes absolute value, The RGB_IR is the corresponding IR component of pixel value in each channel R, the channel G and channel B.

5. the method according to claim 1, wherein each channel IR in the determination second pixel set The corresponding R component of pixel value or B component, comprising:

The corresponding visible light component in each channel IR is determined according to preset channel arranging rule for each channel IR, it is described visible Light component is R component or B component；

For the corresponding visible light component in each channel IR, according to preset the third place relationship, from second pixel set Determine the 4th target pixel value for calculating the corresponding visible light component of pixel value in each channel IR；

6. general is described to also the method according to claim 1, wherein described according to the third pixel set Original image reduction, comprising:

For in the third pixel set each channel R and each channel B, according to preset 4th positional relationship, from described The 5th target pixel value of the corresponding G of pixel value, B component that calculate each channel R are determined in three pixel sets, and calculate each B 6th target pixel value of corresponding R, G component of the pixel value in channel；

According to the 5th target pixel value, the corresponding G in each channel R, B component are calculated, according to the 6th target pixel value, meter Calculate corresponding R, G component of each channel B；

The corresponding G in each channel R, B component and corresponding R, G component of each channel B are updated to the third pixel set；

For each channel G in updated third pixel set, according to preset 5th positional relationship, from updated third The 7th target pixel value of the corresponding R of pixel value, B component that calculate each channel G are determined in pixel set；

7. a kind of image restoring device, which is characterized in that described device includes:

Pixel obtains module, each pixel for obtaining the first pixel set of image to be restored, in first pixel set Pixel value be the pixel value obtained by the channel feux rouges R, the channel green light G, blue light channel B or the channel infrared light IR；Wherein, The channel R, the channel G in one pixel set include IR component in the pixel value of channel B；

First component determining module, for determining institute respectively according to the pixel value in each channel IR and preset first interpolation rule State each channel R in the first pixel set, the channel G, channel B the corresponding IR component of pixel value；

First component removes module, for remove each channel R, the channel G, channel B the corresponding IR component of pixel value, by described the One pixel set is updated to the second pixel set；

Second component determining module, for determining the corresponding R component of pixel value or B in each channel IR in second pixel set Second pixel set is updated to third pixel set by component；

8. device according to claim 7, which is characterized in that the first component determining module, comprising:

IR separates submodule, the corresponding first IR picture of pixel value for isolating each channel IR from first pixel set Plain data array；

First component determines submodule, for dividing according to the first IR pixel data array and first interpolation rule Do not determine each channel R in first pixel set, the channel G, channel B the corresponding IR component of pixel value.

9. device according to claim 8, which is characterized in that first component determines submodule, comprising:

First object pixel-value determining unit, for being directed to each channel R and each channel B, according to preset first position relationship, from Determine that the pixel value for calculating each channel R corresponds to the first object pixel value of IR component in the first IR pixel data array, with And the pixel value of each channel B of calculating corresponds to the second target pixel value of IR component；

First IR component calculation unit, for calculating the corresponding IR of pixel value in each channel R according to the first object pixel value Component calculates the corresponding IR component of pixel value of each channel B according to second target pixel value；

Data array updating unit, for the corresponding IR component of pixel value according to each channel R and each channel B, by described first IR pixel data array is updated to the 2nd IR pixel data array；

Second target pixel value determination unit, for being directed to each channel G, according to preset second position relationship, from described second Determine that the pixel value for calculating each channel G corresponds to the third target pixel value of IR component in IR pixel data array；

2nd IR component calculation unit, for calculating the corresponding IR of pixel value in each channel G according to the third target pixel value Component.

10. device according to claim 7, which is characterized in that first component removes module, comprising:

Judging submodule, for judge each channel R, the channel G, channel B the corresponding IR component of pixel value whether be not more than it is default The first IR intensity threshold；

First component removes submodule, for being not more than in advance when the corresponding IR component of pixel value of each channel R, the channel G, channel B If the first IR intensity threshold when, according to the following formula, by each channel R, the channel G, channel B pixel value remove respectively it is corresponding IR component:

RGB_0=RGB_1-TH+abs (RGB_IR-TH)

11. device according to claim 7, which is characterized in that the second component determining module, comprising:

It can be seen that light component determines submodule, for determining each channel IR according to preset channel arranging rule for each channel IR Corresponding visible light component, the visible light component are R component or B component；

First object pixel value determines submodule, for being directed to the corresponding visible light component in each channel IR, according to preset third Positional relationship determines the 4th mesh for calculating the corresponding visible light component of pixel value in each channel IR from second pixel set Mark pixel value；

It can be seen that light component computational submodule, for according to the 4th target pixel value, the pixel value for calculating each channel IR to be corresponding Visible light component.

12. device according to claim 7, which is characterized in that described image recovery module, comprising:

Second target pixel value determines submodule, each channel R and each channel B for being directed in the third pixel set, root According to preset 4th positional relationship, the corresponding G of pixel value, the B component for calculating each channel R are determined from the third pixel set The 5th target pixel value, and calculate the 6th target pixel value of corresponding R, G component of pixel value of each channel B；

First component computational submodule, for calculating the corresponding G in each channel R, B component, root according to the 5th target pixel value According to the 6th target pixel value, corresponding R, G component of each channel B is calculated；

Pixel set updates submodule, and the corresponding G in each channel R, B component and corresponding R, G component of each channel B are updated to The third pixel set；

Third target pixel value determines submodule, each channel G for being directed in updated third pixel set, according to default The 5th positional relationship, determine the corresponding R's of pixel value, B component for calculating each channel G from updated third pixel set 7th target pixel value；

Second component computational submodule, for according to the 7th target pixel value, the pixel value for calculating the channel G to be corresponding R, B component.