CN109074637A - For generating the method and system of output image from multiple corresponding input picture channels - Google Patents

Abstract

Describe the method and system for generating output image from multiple (N number of) corresponding input picture channels.Determine the Jacobian matrix in the multiple corresponding input picture channel.Calculate the main feature vector of the apposition of the Jacobian matrix.Setting symbol associated with the main feature vector, thus leads to positive scalar value by the input picture channel pixel of the main eigenvector projection.Output image is generated, the output image is every pixel projection of the input channel on the direction of the main feature vector.

Description

For generating the method and system of output image from multiple corresponding input picture channels

Technical field

The present invention relates to for from such as multiple spectrum and/or multiple sensors image data multichannel image Data generate the method and system of output image.

Background technique

There are many applications, wherein multiple images or image channel are fused together, are individually summarized with being formed (summary) gray scale or colored output.These include calculating photography (such as RGB-NIR), multispectral photography, diffusion tensor (medicine) and remote sensing.

Many different equipment capture image, then show image on monitor or other display equipment.Finally, greatly It is most all to be explained by human viewer or only appreciate.In some cases, going to visual image from the image of capture is very It is simple: only to need to carry out color correction using the image of RGB color cameras capture, to show visually close to original scene Image.But when, for example, when capturing image except visible electromagnetic spectrum, or work as and capture more than three channel (also referred to as Dimension) when, situation is really not so simple.

In many imaging applications, the number of channels of capture is more than what human viewer was observed that.Although the mankind Vision system can visualize three kinds of color dimensions, but the content that many image capture systems capture is significantly more than this: multispectral Section and Hyperspectral imager can capture 200 or more Color Channels, including the image captured in infrared and ultraviolet range.

A kind of method for making the information visualization in multispectral section or high spectrum image is that simply display is included in visible light Signal section in spectrum；In other words, the color image that displaying duplication will be seen by human viewer.In this way Problem is that the information from other mode such as infrared ray and ultraviolet light will be lost.Alternatively, being more generally two kinds The color of the different still metamerisms of spectrum will be displayed as identical.

Another method is in the information for mix the information from all channels and make reflection component image The pseudo color image of appearance.Although this method will retain some information from all different modalities, each object is distributed to Color may be significantly different with true colors.

A kind of method for attempting to retain in the output image and transmit the information from information source is referred to as image co-registration.? In image co-registration, image detail present in N number of input picture or channel is combined into an output image.Image interfusion method Including the method based on wavelet decomposition, Laplace tower transformation (Laplacian pyramid) and neural network.

Image gradient is a kind of nature for showing image detail information and multiduty mode, and has been used as several images The basis of integration technology.The powerful method for summarizing the gradient information on N number of input picture channel is referred to as Di Zenzo structure It measures (2 × 2 inner products for being defined as the image Jacobian of N × 2).Method based on structure tensor has very in computer vision It applies, including image segmentation and image co-registration more.

In general, image co-registration is carried out in derivative domain.Herein, it was found that a new compound fusion derivative, it It can best consider the details on all images, then obtained gradient fields (gradient field) are accumulated again Divide (reintegrated).

This be Socolinsky and Wolff used in US 6,539,126 method (hereinafter referred to as " SW ", and Entire contents are incorporated herein by reference).It finds one group of 1 dimension equivalent gradient using Di Zenzo structure tensor (equivalent gradient), in terms of the orientation and size of this group of equivalent gradient, this group of equivalent gradient with from multichannel figure As derived tensor in least square meaning as close possible to.Di Zenzo structure tensor (Z) is also referred to as first fundamental form, Its inner product for being defined as Jacobian: Z=J^TJ。

Equivalent gradient is defined by the most important characteristic value and associated feature vector of structure tensor.Exported gradient Symbol be also indefinite (this is the weakness of SW method), and heuristic definition must be carried out.Unfortunately, this method In export gradient fields it is usually non-integrable.Integral is attempted in least square meaning, seeks a single channel image z The solution of (x, y), derivative is as close possible to equivalent gradient.Therefore, again integration step would generally magically change go out new details (this be thin Section does not appear in any input picture or image channel), including haloing, it is bent pseudomorphism and large scale puppet gradient.

Since gradient fields again Integral Problem (can not field of integration) are substantially ill posed, so derivative field technique will be total It is that magically change goes out the details being not present in original image in fused image.

The technology for applying additional constraint to Integral Problem again recently can be alleviated sometimes but cannot eliminate these pseudomorphisms.

In other methods, fused image is post-processed, so that connected component (is defined as having identical input The region of the input multispectral image of vector value) it must image pixel intensities having the same.Unfortunately, this additional step meeting Generate unnatural profile and edge effect.

Summary of the invention

According to an aspect of the invention, there is provided a kind of be used for from multiple, N number of, corresponding input picture channel generation The method for exporting image, which comprises

Determine the Jacobian matrix in the multiple corresponding input picture channel；

Calculate the main feature vector of the apposition of Jacobian matrix；

Symbol associated with main feature vector is set, is thus led by the input picture channel pixel of main eigenvector projection Cause positive scalar value；And

Generate every pixel projection of the output image as the input channel on principal eigenvector direction.

Calculating step, it is also preferable to include following steps:

For each element of the Jacobian matrix for non-zero, sparse N vector projection figure is generated from the Jacobian matrix Picture；And

For the element of the Jacobian matrix for zero, by the sparse N vector projection image filling be zero it is described it is refined can Than the element of matrix.

Filling may include being inserted by the way that the vector of each neutral element to be defined as to the average value of local neighborhood.Average value It may be edge sensitive.Filling may include carrying out bilateral filtering to sparse N vector projection image.Two-sided filter preferably wraps Include intersection two-sided filter.Filling step may include smoothing N vector projection image.Filling step may include interpolation N to Measure projected image.Filling step may include that edge sensitive diffusion is executed to N vector projection image.

Filter step may include independently being filtered to each channel of N vector projection image.

Method may further include scales each vector after filling, with unit length.

Method may further include the spread vector after filling, and each component of a vector is moved away average value one The angle of a fixed multiple.

Method may further include following steps:

The determination is executed to the input picture channel for reducing sampling and calculates step, and to calculated main feature vector Sampling is amplified, for using in generation step.

Each unique input picture vector can map directly to single projection vector.

Mapping between the main feature vector of unique input picture vector sum may be implemented as look-up table.

Input picture can have N number of channel and export image with M channel, and main feature vector includes that will input figure The Jacobian of the N of picture × 2 is mapped to every pixel M × N matrix transformation that target M × 2 exports Jacobian.

Method, which may further include, carries out input picture channel by respective M × N transformation in input picture channel The step of every pixel transform.

The input picture Jacobian of N × 2 can be mapped as M × 2 and strengthen Jacobi's pairing determinant by M × N transformation.

Calculate step can with the following steps are included:

The sparse image of N × 2 is inserted by the element for the Jacobian matrix for being zero to generate sparse M × N changing image.

Method may further include following steps:

The determination is executed to the input picture channel for reducing sampling and calculates step, and to calculated M × N convert into Row amplification sampling, for being used in generation step.

Each unique input picture vector can map directly to single M × N transformation vector.Unique input picture vector Mapping between M × N transformation may be implemented as look-up table.

According to another aspect of the present invention, a kind of be used for from multiple, N number of, corresponding input picture channel generation is provided The system for exporting image, the system comprises:

It is arranged to access the input in N number of input picture channel；

It is configured to execute the processor of the computer program code for executing image processing module, comprising:

It is configured to determine the computer program code of the Jacobian matrix in the multiple corresponding input picture channel；

It is configured to calculate the computer program code of the main feature vector of the apposition of Jacobian matrix；

It is configured to setting symbol associated with the main feature vector, thus by the main eigenvector projection Input picture channel pixel generates the computer program code of positive scalar value；With,

It is configured to generate output image as every pixel projection of the input channel on principal eigenvector direction Computer program code.

To the computer program code that calculates can with the following steps are included:

Be configured to each element for the Jacobian matrix for non-zero from the Jacobian matrix generate sparse N to Measure the computer program code of projected image；With,

It is configured to insert the element of the Jacobian matrix for zero computer program of sparse N vector projection image Code.

The computer program code for being configured to filling may include being configured to smoothing N vector projection image Computer program code.

The computer program code for being configured to filling may include the meter for being configured to interpolation N vector projection image Calculation machine program code.

The computer program code for being configured to filling may include being configured to execute side to N vector projection image The computer program code of edge sensitivity diffusion.

Filter can be arranged to independently be filtered each channel of N vector projection image.

Processor can be configured to execute computer program code, each vector to be scaled after filling, with With unit length.

Processor can be configured to execution computer program code will be each with the spread vector after filling Component of a vector moves away the angle of average value one fixed multiple.

Processor can be configured to execute computer program code, to obtain the input channel for reducing sampling, to contracting The input picture channel of small sampling executes the determination and calculates step, and amplifies and take to calculated main feature vector Sample, for being used in generation step.

System can also include the look-up table mapped between unique main feature vector of input picture vector sum, the system quilt It is arranged to access look-up table, to determine the main feature vector for generating output image.

Input picture can have N number of channel and export image with M channel, and main feature vector includes that will input figure The Jacobian of the N of picture × 2 is mapped to every pixel M × N matrix transformation that target M × 2 exports Jacobian.

Processor can be further configured to execute computer program code, to pass through the respective of input picture channel M × N transformation every pixel transform is carried out to input picture channel.

The input picture Jacobian of N × 2 can be mapped as M × 2 and strengthen Jacobi's pairing determinant by M × N transformation.

Processor can be configured to execute computer program code, to pass through the member for the Jacobian matrix for being zero Element inserts the sparse image of N × 2 to generate sparse M × N changing image.

Processor can be further configured to execute computer program code, with logical to the input picture for reducing sampling Road executes the determination and calculating, and amplifies sampling to calculated M × N transformation, for generating output image.

Each unique input picture vector can map directly to single M × N transformation vector.

System can also include the look-up table mapped between unique input picture vector sum M × N transformation, which is set It sets to access look-up table, to determine M × N transformation for generating output image.

In an embodiment of the present invention, from N channel image data (channel be image component, layer or channel or individually Image) generate and export the corresponding output image data of image.It is different from the art methods of such as those described above, this A little embodiments attempt the details for avoiding introducing magically change by avoiding again integration step and other pseudomorphisms.

In an embodiment of the present invention, output image is generated, in the output image, x and y derivative generates and such as above-mentioned SW The identical equivalent gradient field of the method for method.In the process, the fusion/constitutional diagram with sought derivative structure is obtained Picture, without integrating again.

The embodiment of the present invention executes every pixel projection (linear combination) of input channel, to generate output image.Output Image does not need to carry out differential (differentiated), but if differential is wanted to export image, will generate and class discussed above As equivalent gradient.In an embodiment of the present invention, projecting direction is the main feature vector of the apposition of Jacobian.Projection It executes in image space, causes to export scalar image, rather than in prior art approaches, art methods It is run in gradient field, and provides output gradient, these output gradients tend not to integrate again in the case where no pseudomorphism.

In a preferred embodiment, it discloses for handling the image with sparse derivative information.In a preferred embodiment, exist Before projection input picture channel is to generate output image, spread between similar image-region using joint two-sided filter Projection coefficient.Global projected image can also be found, wherein each unique multichannel input vector be mapped to individually project to Amount.That is, projected image is the look-up table from input picture.

In a preferred embodiment, every channel projection can be exported, to create RGB color (or in general channel M) Output.

Detailed description of the invention

The embodiment of the present invention is only described by way of example now with reference to attached drawing, in the accompanying drawings:

Fig. 1 is the flow chart for generating the method for output image from multiple corresponding input picture channels；

Fig. 2 is the flow chart of the various aspects of the sample implementation of the method for Fig. 1；

Fig. 3 is the flow chart according to the method for another embodiment；With,

Fig. 4 is according to an embodiment of the present invention is used for from multiple, N number of, corresponding input picture channel generation output image System schematic diagram.

Specific embodiment

In the following description, I (x, y) is used to indicate the xth of n × m vector-valued image, y pixel.Each pixel has N number of Plane.For example, if I (x, y) is about red, green and blue (RGB) Color Space Definitions color image, the pixel It will be RGB vector: [R G B].If image is also included as that the plane of delineation of NIR (near-infrared) or NIR image are schemed with RGB As associated, then each pixel will be 4 vectors: [R G B NIR].

It should be understood that each plane can be the channel of single image, or can come from separate sources about identical The data of the image of object.

In order to understand the derivative structure of image, image is in the x and y direction in each of N number of plane of delineation plane Carry out differential.This provides N × 2 (for x the and y derivative of each plane of delineation in N number of plane of delineation) and this is summarised in N In × 2 Jacobian matrix J:

In SW method described above, finds closest to the single of all derivatives in all planes of delineation and equivalent lead Number:

In SW method, the size and orientation for exporting gradient are known, but its symbol is heuristic definition.Partly It is that the pseudomorphism discussed above seen in SW method is related with the export heuristic setting of symbol of gradient.SW method is usual Magically change goes out new details (it is not appeared in any input picture or image channel), including haloing, is bent pseudomorphism and large scale Pseudo- gradient.

Fig. 1 is the flow chart for generating the method for output image from multiple corresponding input picture channels.

In step 10, Jacobian matrix (J) is determined for multiple corresponding input picture channels.It is listed at (1) above One example of this matrix.

In step 20, the main feature vector of the apposition of J is calculated.

In step 30, the symbol of main feature vector is determined.Symbol is preferably determined as so that by eigenvector projection Input picture channel pixel should lead to positive scalar value.The symbol of projection vector is correspondingly configured.

In step 40, output image is generated as every pixel projection of the input channel on the direction of main feature vector.

It has been determined that the unit length feature vector (here shown as v) of the column space of J has various useful characteristics:

i.v^tThe gradient that J (by v multiplied by J (1 × N vector is multiplied by the Jacobian of N × 2)) is provided is equal to be generated by (2) Until unknown symbols gradient (can as described below as handled).

Ii. because attribute (i) is linear operation, differential is also linear operation, so order of operation can exchange, it may be assumed that

On the left side of (3), we carry out differential, and before this, we make a new scalar image as original graph The linear combination of picture, wherein the component of v defines the weight of every combination of channels.It, can after and then giving v at a pixel Image (such as output image can be blending image) is exported derived from N number of input channel (as I (x, y) directly to calculate Original image linear combination) and the needs that do not integrate again.

Iii. it can show that the value for exporting the pixel of image must all positive values due to being intended to make to export image.This A input is as follows:

i.v^t I(x, y) < 0 is thenv←-v (4)

A problem using SW method is that the symbol of equivalent gradient vector is unknown.It has been proposed that setting symbol, with Brightness step (R+G+G)/3 or optimization symbol are matched, to maximize the integrability of desired gradient fields.Every kind of method all needs It further calculates, is not always suitable.On the contrary, it is different from SW method, in an embodiment of the present invention, equivalent derivative vector Symbol (arrow to the left in (4) indicate distribution) can be assigned in a manner of good principle.

Fig. 2 is the flow chart of the various aspects of the sample implementation of the method for Fig. 1.

It is defeated on the direction of the main feature vector of the apposition of Jacobian J as above by reference to as being discussed Fig. 1 The every pixel projection (linear combination) for entering channel I (x) generates sought combination scalar image O (x):

Discussed above is Jacobians.There are a variety of methods can achieve main feature vector U^x, it is shown below A kind of preferred embodiment of bright mode.

Main feature vector U^xIt is U^xThe first column vector:

U is then a part (subscript of the singular value decomposition of Jacobian J^xIndicate the picture position x, y):

J=USV^T (6)

U, S and V is the matrix of N × 2,2 × 2 and 2 × 2 respectively.U and V be it is nonnormal, S is diagonal matrix (its diagonal line point Amount >=0).

SW equivalent gradient (until unknown symbols) is by the main feature vector of unit length of the Z of dominant eigenvalue scaling.This is SV^TFirst row.Pass through U^xTransposition before multiply (7) and return and the identical equivalent gradient that is found by SW method.

In other words, U^xIt is Jacobian and structure tensor Z (Z=VS²V^T) subduplicate inverse product, by This show that the inverse square root of Z is VS-¹。

Structure tensor be it is positive, semidefinite, and therefore, characteristic value will be it is true, it is positive.It is continuous in bottom channel And characteristic value be different in image, the main feature vector of apposition also will constantly change, and can be from calculated above Out.

In the image of region or structure tensor with consistent characteristic value (such as corner) with zero derivative, at one There may be variation (discontinuity) biggish compared with another place picture position on the projecting direction of picture position discovery, this can It can be problematic when determining principal eigenvector.Fig. 2 is that processing image channel data are real suitable for the present invention to ensure Apply the flow chart of the preferred method of example.

In step 100, projected image P (x, y) is initialized as zero in each pixel position.

In step 110, it is based on U^xAnd S^xFollowing filling P (x, y):

IfAndThen P (x, y)=U^x

Every place all assumes to meet two threshold conditions and there are non-zero Jacobians, and two characteristic values are enough , then there is sparse N vector projection in different (that is, the image at every place has non-zero derivative, and not in the corner seldom occurred) Image P^s(x, y) (subscript " s " indicates that vector is sparse).

In order to ensure there is the final projected image-P of a specific projection vector in P (x, y)-each spatial position^s(x, Y) it is received in.Specifically, the N vector at each position (x, y) is the average value of its local neighborhood, and wherein average value is also pair Edge sensitive.This is completed in step 120, and wherein P (x, y) is by bilateral filtering, preferably by the simple intersection pair of application Side filter.

P (x, y)=BilatFilt (I (x, y), P^s(x, y), σ_d, σ_r)

θ₁And θ₂It is the system parameter that can be changed according to embodiment.In one embodiment, they are freely set For 0.01 (assuming that image value is in [0,1]).

Preferably, two-sided filter is with the intersection two-sided filter by the original image I range item limited.Filtering Preferably using with standard deviation_dAnd by σ_rThe Gaussian spatial of standard deviation in the range of parametrization is fuzzy to often leading to What road independently executed.In σ_r=σ_drWhen=0, generation is not spread.With σ_d→ ∞ and σ_d→ ∞, diffusion become global average, And projection is intended to the global weighted sum of input channel.If σ_d→ ∞ and σ_r=0, then each difference of the value in image Vector will be associated with identical projection vector, therefore bilateral filtering step defines the surjection that can be used as look-up tables'implementation (surjective mapping)。

In addition to these border conditions, the standard deviation in two-sided filter should be selected, to provide sought diffusion, but Should select with ensure space be it is sufficiently large, to avoid spatial artifacts.

In one embodiment, σ_dAnd σ_rIt is separately arranged as min (X；Y) * 4 and ((max (I)-min (I))/4)).One In a embodiment, these values are found by rule of thumb.

In step 130, P (x, y) is adjusted, so that each projecting direction is unit vector.

Optional step 140 can also be applied.In this step, spread function is applied to P (x, y), to improve projection Image.Specifically, in one example, each projecting direction is moved away average value one fixed multiple by spread function (diffusing step pulls in the opposite direction and projecting direction is caused more to connect with the projecting direction found in step 110 angle Nearly average value).

The exact spread function that will be applied will be different because of embodiment, and additionally depend on discussed domain.

Under default situations, extension be by calculate diffusion before and after deviation average average angle deviation come into Capable.Diffusion vector is by single factor k (k >=1) scaling afterwards, so that average angle deviation is identical as before diffusing step.If Spread function generate negative value, then the value be limited (clipped) be 0.Zoom factor k can be wanted according to each embodiment It asks and changes.For example, k can be 2 in time-lapse photography image, to stretch projected image.In multifocal application, the value of K can With larger (such as 5 or 8).

In the embodiment of fig. 2, it is interpolated or spreads across the specific projection vector of image.In a preferred embodiment, this To intersect two-sided filter by the way that application is simple and realize, the intersection two-sided filter have been found to standard gaussian or in Value filter provides superior as a result, because it is using including that picture structure in input picture channel carrys out guiding projection vector Diffusion.

There are also other ways can provide " filling " perspective view, including anisotropy parameter, connected component label (to input In identical connected component execute same projection (being similar to) or more strongly execute space constraint than bilateral filtering).

Final projected image can be further constrained, therefore it is the function for inputting multichannel image.That is, Projected image can be the look-up table from input multichannel image.

After executing step 100-130 (and optionally there are also 140), the result is that limit every pixel of projecting direction N number of value projects N vector I (x) along the projecting direction, to generate scalar output image.

Fig. 3 is the flow chart according to the method for another embodiment.

In this embodiment, carrying out diminution sampling to input picture in step 200 (or is alternatively Ke Yiti For or obtain reduce the input picture of sampled version) and only for thumbnail image calculate P in step 210 (P can be with Mode identical with the mode illustrated referring for example to Fig. 2 is calculated).Then, it is taken in a step 220 using the bilateral amplification of joint Sample finds full resolution projected image, then in step 230, is used to generate non-diminution for the full resolution projected image and takes Every pixel projection of the input channel of sample.

Again, final perspective view can be the look-up table (LUT) of input multichannel image.It can be calculated on thumbnail LUT。

The thumbnail calculating also have the advantage that, it may be assumed that projected image can be calculated with sliced fashion, i.e., this method from It does not need to calculate full resolution projected image.

In the example RGB-NIR image pair of 682 × 1024 resolution ratio, it is logical which is independent R, G and B to fusion Road, totally 3 fusion steps need 54.93 seconds under full resolution, and use the MATLAB implementation of embodiment to 68 × 102 2.82 seconds are needed when reducing the thumbnail calculating of sampling.The increase of this speed can't significantly affect result images --- and it is corresponding Full resolution and the average SSIM (structural similarity index) reduced between sampling result are 0.9991 in image channel.General feelings Under condition, it has been found possible to image is substantially reduced be sampled as 10K pixel thumbnail (alternatively, as in this example, It is slightly or even smaller), and there is good result.Although almost always if narrowing down to about VGA resolution, The result calculated on thumbnail will be with result calculated on full resolution image close to identical.

Fig. 4 is for from multiple, N number of, corresponding input picture channel 401-404 to generate the system 400 of output image Schematic diagram.As described above, channel can be independent image, the image feeding from camera, the feeding of single or associated picture Component, single or component of related image file etc..In the shown embodiment, (such as camera can be with for camera 401 and 402 RGB, and one is infrared) and data source 403/404 be illustrated as providing image channel.For example, data source can be point The image file of layer, every layer from the image file is played the role of individual channel 403,404.It should be understood that many groups It closes and displacement is possible, and the not homologous quantity of image channel is inexhaustible.

System 400 includes being arranged to access the input 410 in N number of input picture channel.This may be to be connected to data feedback It send, file I/O equipment or system or the interface or bus of some other input.

System 400 further includes processor 420 and the operation of system 400 and executes the calculating for executing image processing module Any necessary memory or other components needed for machine program code, comprising:

It is configured to determine the computer program code of the Jacobian matrix in the multiple corresponding input picture channel；

It is configured to calculate the computer program code of the main feature vector of the apposition of Jacobian matrix；

It is configured to the computer program code of setting symbol associated with the main feature vector, thus by described The input picture channel pixel of main eigenvector projection generates positive scalar value；With,

It is configured to generate the computer program code of output image, the output image is as input channel in main feature Every pixel projection on the direction of vector.

Output image 430 for example can be output to memory, data storage, via network via I/O equipment or system It is output to user interface or is output to picture reproducer, such as printer or other equipment, for generating hard copy.Output figure Input as can also be used as other systems.

The extension of the above method.

Assuming that N, x and y derivative are mapped as single equivalent gradient instead of using SW method, then some other function is usedf。 Vector function f () returns to 1 × 2 vector, i.e., the estimation derivative of every pixel x and y.It, can as an example of this function To modify SW method, so that when exporting their equivalent gradient, the weight of biggish every channel derivative is greater than small.Each At pixel, discovery projection vector v meets:

v ^tJ=+-f(J) (7)

Equation (7) owes fixed.Have very muchvThis equation can be met.But this can by determine least-norm solution come It solves:

v=JcWher is whereinf(J)[J^t]^-1 (8)

WhereincIt is 2- vector.That is,vIn the column space of J.Alternatively, in pixelvPlace, it can be found that its most (with Least square meaning) meet at all pixels in given pixel and associated neighborhoodsv ^tJ=+-f(J)。

As embodiment discussed above, initial projections vector-valued image is initially sparse and should be processed, with Projection vector is anywhere limited by edge sensitive DIFFUSION TREATMENT.

Herein, eachv(x, y) has unit length not important.On the contrary, if given final projection is formed as dilute The weighted array of the original projection in projected image is dredged, then the summation of weight is 1.

(10) the right understands are as follows: final projected image is by the inverse of weight (for limiting final projected imagev (x, y)) it scales.

In WO2011/023969, copy is incorporated herein by reference, and N component image is fused into M component pairing figure As (wherein usual M < < N).Another example is 4 channel RGB-NIR images are mapped to 3 dimensions fusion color image.

In disclosed method and system, by the source N × 2 Jacobian J^sIt is transformed to 3 × 2 (for colored situation) Strengthen Jacobian J^A。J^AIn each of 3 derivative planes all integrate again, to provide final image.Again When integrating (usually non-integrable field), the pseudomorphism integrated again is often generated.

In an embodiment of the present invention, every pixel can solve 3 × N linear transformation T, so that:

(10)

TJ^S=J^A

Also due to differential is linear, the fused 3-D image at given pixel can be calculated as TI(x, y), because If J is precisely calculated in we for this transformed 3 channel image of our differential^A.As described above, there are many Ts to expire Foot (11).Least-norm solution can be used uniquely to limit T.Alternatively, can by find most meet given pixel position and The single T of the pixel of its neighborhood finds T with least square meaning.

Therefore, there are non-zero Jacobian J^sImage-region in, J^AAnd T^s(x, y) can be computed (such as Subscript T^sChanging image of drawing attention to is initially sparse).This obtains final non-sparse T by spreading initial sparse mapping ensemble (x, y) (in each position, we have 3 × N transformation T).Using similar diffusion process described in a section as above, final output Blending image is equal to T (x, y)I(x,y)。

Again, T (x, y) can be the function of input picture (input of each multichannel be mapped to single transformation, and this is reflected It penetrates and can be used as look-up table to realize).

The image interfusion method of other algorithms, the Eynard that various experiments have been carried out to compare the above method et al. The frequency spectrum edge (SE) of (based on M is found using graph Laplacian to N channel color mapping) and Connah et al. Method (it is integrated again based on structure tensor and based on the gradient tabled look-up).As a result appended opinion is set forth in the form of attachment 1 Wen Zhong, and be incorporated herein by reference." 2015 year IEEE world calculating of the paper publishing 7 to 13 December in 2015 On the 334-342 pages of the ICCV15 procceedings of machine visual conference (ICCV) ", and it is incorporated herein by reference.

The embodiment of the present invention is shown in Fig. 1 of attachment 1 compared with prior method, wherein there are two uniform whites Image removes the upper left corner and lower left corner a quarter respectively.Wavelet transform (DWT) figure is generated using the method based on small echo Picture, this method merge the coefficient of two images under different proportion.We are run using CM (selection maximum value) selection method The DWT image co-registration embodiment of standard, this method is very simple, and is one to behave oneself best in the comparison.

Input picture very little, therefore only 7 grades of wavelet decompositions.In 1c and 1d, it is shown that use 4 He of Daubechies The output of 1.3 small echo of Biorthogonal.Obviously, wavelet method and SW method (1e) are unsuitable for for this image The example of fusion.However, the result of the embodiment of the present invention (1f) makes image co-registration without pseudomorphism.Green line shown in 1h Strength distribution curve is shown in 1f has desired equal Brightness Whites value, and SW strength distribution curve 1g show it is a large amount of unreal The Strength Changes dissolved.

In 1 Fig. 2 of attachment, it is shown that for test colour blindness stone Yuan Shi (Ishihara) colour vision test board colour to ash Spend image co-registration example.The output of SW method is shown in Fig. 2 f of attachment 1.SW method fails herein, because image is It is made of the colored circles in white background.Because all edges are all isolated in this way, equivalent gradient field is accurate Color gradient is characterized, and is integrable, and the output in Fig. 2 f of attachment 1 does not integrate pseudomorphism.However, after fusion Image can not capture the actual look and feeling of input.On the contrary, the image generated by the embodiment of the present invention in Fig. 2 e (intermediate steps are shown in Fig. 2 b-2d of attachment 1) is shown initial projections direction and is spread, this pair using bilateral filtering step Side filter step considers that the projecting direction calculated at pixel together with other image-regions.

For example, final gray scale output can be used for the image optimization of colour anomaly viewer.Image in Fig. 2 e of attachment 1 It can be used as the luminance channel replacement in the LUV color space of Protanope analog image, by the original RGB image (figure of attachment 1 It is in 3a), sightless color change is mapped to colour anomaly observer and can experience for colour anomaly observer The luminance channel details arrived.It in this particular example, the use of reduce sampling rate is 0.5, and k extensograph parameter is 2. The result for the system that Eynard et al. suggests also has reached expected result as comparing presentation-both of which, still Eynard et al. is because their fusion changes the color value of output and generates higher levels of difference, and by reality of the invention The output image for applying example generation only influences brightness.

The quality that the gray scale generated from RGB image exports can be measured by various measurements.The measurement ratio of Kuhn et al. Compared with the Gray homogeneity between pixel in the color distance and output gray level image between pixel in original RGB image.Attachment 1 Table 1 show when applied to RGB image and CIE L luminance channel, Eynard et al. from Cadik data set result and When the result of the embodiment of the present invention, the comparison of the result of the measurement.It should be understood that the result of the embodiment of the present invention is in many In the case of be superior.

The image captured for remote sensing application usually crosses over visible light and infrared wavelength spectrum.From Landsat 5 Thematic Mapper (TM) obtains data, can see example in Fig. 6 of attachment 1.There are 7 capture image channel (3 In visible spectrum and 4 infrared images).From 0.45-0.51 μm (blue), 0.52-0.60 μm (green) and 0.63-0.69 μm (red) captures three visual pictures, is used separately as the channel B, G and R of input RGB image.In Fig. 6 a of attachment 1, display Input RGB image from Landsat image set, and infrared band 5 and 7 is shown in Fig. 6 b and 6c of attachment 1 comprising The additional detail being not present in RGB wave band.All 4 infrared channels, due to space reasons, are only shown herein for merging 2.4 infrared channels successively use the embodiment of the present invention in Fig. 6 f of the SW method and attachment 1 in Fig. 6 d of attachment 1 with R, G and channel B fusion, then exporting RGB channel has high and low quantile, matches with input RGB channel.In attachment 1 Show in Fig. 6 e frequency spectrum edge method as a result, its directly fusion RGB image and all 7 multi-band images.

For the application, the 0.5 k extensograph parameter for reducing sampling rate and 2 is used.Resulting image ratio SW method It is more detail.

In Fig. 3 of attachment 1, traditional RGB image (3a) is merged with near-infrared (NIR) image (3b).According to the present invention The processing of embodiment is merged in the channel R with NIR by application 3 times-, and the channel G is merged with NIR, channel B is merged with NIR.So Afterwards, post-processing is executed, wherein image is stretched, so that their 0.05 and 0.95 quantile is identical as original RGB image.Finally Image be shown in Fig. 3 e of attachment 1.In order to compare, it is shown that the output of frequency spectrum edge, i.e. Fig. 3 c and Eynard of attachment 1 Et al. output, i.e. the 3d of attachment 1.In same image sequence, amplification details illustration is shown in the 3f of attachment 1.POP method The more NIR details of output picture catching ratio SE result, while generating natural colors more more than the result of Eynard et al., The result of Eynard et al. has green colour cast and lacks color contrast.POP goes out good color contrast as the result is shown, natural Degree and details.For the application, the 0.1 k extensograph parameter for reducing sampling rate and 1 is used.

Multifocal image co-registration is another potential application, usually using the gray level image with different focus setting It is studied.The multifocal image co-registration of standard, which is related to fusion, has different two gray scale input pictures for focusing setting.Every It opens in input picture, there are about the images of half to be in focusing state, therefore can be generated on each point by combining them The image of focusing.

The table 2 of attachment 1, which is shown, to be measured in the multifocal point image of several standards using standard picture fusion mass to upper sheet The performance of inventive embodiments (POP image interfusion method) in the task compares.Q_XY/FMeasurement is based on gradient similarity, Q (X；Y； F) measurement is based on structural similarity image measurement (SSIM),Measurement is based on mutual information.As a result with various comparable sides Method is compared-in most cases, it is got the upper hand by the final image that the embodiment of the present invention generates.

Full light photography (plenoptic photography) provides the various focusing options again of color image, allow from Single exposure creation has the image of the different depth of field.The embodiment of the present invention can be used for the image co-registration for focusing these differences At the single image focused completely.Due to knowing, only one of which image is focused in each pixel, it is possible to be answered for this It is finely adjusted with to embodiment.In one example, the biggish k in application extension function scales item, and uses 0.5 Reduce sampling rate.It can be created that in this way and clearly export image, all focus on each pixel.

Fig. 7 of attachment 1 shows an image, wherein creating four different figures focused again from single exposure Picture.Using one embodiment of the present of invention, the images that difference focuses be fused at each single focusedimage-compare and Speech, the result of the method for Eynard et al. does not show perfect details in all parts of image, and has unnatural Colouring information.

Time-lapse photography is related to capturing the image of Same Scene in different time.In the case where gray level image, it can be used The embodiment of the present invention merges these., can be respectively to R for RGB image, the lamination of G and channel B is merged. This fusion results creates an output image, it combines details most significant in all delay images.For the application, Use the 0.5 k extensograph parameter for reducing sampling rate and 2.Fig. 8 of attachment 1 is shown from a series of of different piece round the clock The result of the method for delay image (from Eynard et al.) and the result and Eynard et al. of POP fusion.Only make at night The details that can be just seen with artificial light sources in result only daytime visible details combine, but the embodiment of the present invention As a result more natural colors are produced.

It should be understood that certain embodiments of the present invention can be used as resident code (example in firmware as discussed below Such as, software algorithm or program) and/or computer usable medium with control logic introduce, the control logic can have Have and is executed in the computer system of computer processor.Such computer system generally includes memory storage, the memory Storage is configured to be provided according to execution come the output of the execution of the code of self-configuring processors.Code can be used as firmware or Software is configured, and can be organized into one group of module, such as discrete code module, function call, the invocation of procedure Or the object in Object-Oriented Programming environment.If realized using module, code may include the list of coordination with one another operation A module or multiple modules.

Alternative embodiment of the invention be construed as include referred to herein as or instruction part, element and feature, it Either individually or collectively by two or more parts, elements or features any or all of combination in a manner of, and wherein Specific integer is mentioned above has known equivalent, these known equivalent quilts in the art related to the present inventionly It is considered as and is incorporated herein, as individually proposes.

Although it have been described that illustrated embodiment of the invention, it is to be understood that not departing from by the attached claims In the case where the present invention that statement and its equivalent in book limit, those skilled in the art can various changes can be made, replacement And change.

Attachment 1

POP image co-registration-derivative area image fusion without integrating again

Alex E.Hayes and Graham D.Finlayson

(A.E.Hayes and G.D.Finlayson work in computational science system of University of East Anglia of Britain, e-mail: alex.hayes@uea.ac.uk,g.findayson@uea.ac.uk)

It makes a summary-has many application programs, plurality of image is fused, individually summarize gray scale or colored output to be formed, Including calculating photography (such as RGB-NIR), diffusion tensor (medical treatment) and remote sensing.In general, and intuitively, image melts Conjunction is carried out in derivative domain.Here, a new compound fusion derivative is found, is most preferably considered thin on all images Then section integrates generated gradient fields again.However, the usual magically change of integration step goes out new details (it is not appeared in again In any input picture wave band), including halation and bending pseudomorphism.In this paper, we are and avoiding integration step again Avoid the details of these magically changes.

Our work is directly established in the work of Socolinsky and Wolff, they tie from every pixel Di Zenzo Structure tensor exports their equivalent gradient field, which is defined as the inner product of image Jacobian.We show X the and y derivative of the projection of original image generates similarly in the main feature vector of the apposition (POP) of Jacobian Equivalent gradient field.In the case where doing so, we have been derived from blending image, with the derivative structure sought by us.Certainly, should Projection will be only just meaningful in the case where Jacobian has non-zero derivative, therefore calculate fusion figure at us We spread two kinds of method of diffusion of projecting direction-proposition and are compared as before.Generated POP blending image has maximum Fusion details, but avoid the pseudomorphism of magically change.Experiment shows the target image fusion measurement using the prior art, our side Method shows transcendent image co-registration performance.

Index term-image co-registration, gradient integrate again, derivative domain, and colour arrives gray scale, RGB-NIR.

1. introducing

Image co-registration has application, including multispectral photography [1], imaging of medical [2], remote sensing [3] in many Problem Areas [4] are photographed with calculating.In image co-registration, the image detail that we seek to will be present in N number of input picture is combined into one Export image.Image gradient is the natural generic way [5] for showing image detail information, and has been used as a variety of images The basis of integration technology, including [6] and [7].Other image interfusion methods include the method [8] based on wavelet decomposition, La Pula This tower-shaped transformation [9] and neural network [10].

It is DiZenzo structure tensor [11] [12] by the powerful way that the gradient information on N number of input picture channel summarizes (its 2 × 2 inner product for being defined as the image Jacobian of N × 2).Method based on structure tensor has in computer vision Have and much apply [13], be included in picture breakdown [14], and relevant to this paper is for image co-registration [15].

The image interfusion method (SW) with significant impact of Socolinsky and Wolff is found using structure tensor 1 dimension set of gradient is imitated, which must be similar in terms of their orientation and size with least square meaning as far as possible Tensor [16] derived from multichannel image.They show equivalent gradient by the most significant characteristic value and correlation of structure tensor The characteristic value of connection limits.Unfortunately, the exported gradient fields of Socolinsky and Wolff are usually non-integrable.Because (no Integrable field) gradient fields again Integral Problem be it is intrinsic ill posed, so derivative field technique will be always in blending image Magically change goes out the details being not present in original image.

Modern technologies are applied with additional restraint to Integral Problem again, these technologies can reduce sometimes but cannot remove Pseudomorphism [17], [18], [19], [20] and [21].In another work [22], fused image is post-processed, so that Connected component (region of its input multispectral image for being defined as identical input vector value) must be having the same Image pixel intensities.Unfortunately, this additional step can produce unnatural profile and edge effect.

In this paper, we have developed a kind of derivative area image fusion method, which obviate to the needs integrated again, And therefore, we avoid integrates pseudomorphism again.Our method starts from calculating the outer of the Jacobian matrix of image derivative Product (rather than definition structure inner product of tensors).We demonstrate original multi-channel image apposition (POP) tensor main feature Projection on the direction of vector leads to the same equivalent gradient field limited in the method for Socolinsky and Wolff.Certainly, this is first Beginning projected image is not all limited in everywhere very well, such as it only can have the place of non-zero derivative in Jacobian It is non-zero, and therefore we are spread available POP projecting direction using bilateral filter.POP blending image is projection Every pixel dot product of image and multichannel preimage.Generated POP blending image has maximum fusion details, but keeps away completely The pseudomorphism of magically change is exempted from.

The comparison of POP image co-registration front method therewith is shown in FIG. 1, wherein there is two consistent white images, they It is respectively, is removed upper left and lower-left a quarter.Using the method based on small echo, by the coefficient of two images with different ratios Example merges, and generates wavelet transform (DWT) image.We have run standard DWT using CM (selection maximum value) selection method Image co-registration embodiment, this method are simple, and are one of optimal representation in comparison [8].Input picture is smaller, because There is only 7 grades of wavelet decompositions for this.In 1c and 1d, we show both use more Bei Xi (Daubechies) 4 and biorthogonal (Biorthogonal) output of 1.3 small echos, Optimum wavelet type are found in [8].It is evident that wavelet method and The method (1e) of Socolinsky and Wolff all cannot be used in the image co-registration example.However, POP image interfusion method (1f) (being discussed in detail in Part III) success in blending image, without pseudomorphism.The intensity distribution of green line in 1f is bent Line (showing in 1h) has desired equal Brightness Whites value, and the strength distribution curve 1g of Socolinsky and Wolff is shown The Strength Changes that sizable magically change comes out.

Fig. 1: image co-registration example: (a) and (b) is by method (c) and (d) fusion based on small echo, generates serious image Pseudomorphism.The method (e) based on gradient of Socolinsky and Wolff works preferably, but intensity gradient is magically change (g), It is all not present in the input image.POP method (f) captures all input details, and without pseudomorphism or the details of magically change.

Second part discusses the background of our method.Our POP image interfusion method proposes in Part III. In Part IV, propose experiment (including compared with other methods).It summarizes in Part V to paper.

2. background

Multichannel image is expressed as by let us(x is that 2 dimension images are sat Mark, I (x) is N vector value).The Jacobian of image I is defined as:

Di Zenzo structure tensor [11] (it is known as first fundamental form in Differential Geometry) is defined as refined Than the inner product of determinant:

Z=J^TJ (2)

If c=[α β]^TIndicate unit length vector, then the squared magnitude of multichannel gradient can be written as: | | Jc | |₂= c^TZc.That is, structure tensor deftly summarizes the derivative structure after the combination of multichannel image.

The singular value decomposition (SVD) of J discloses image interfusion method for understanding Socolinsky and Wolff and also The our own POP Image Fusion proposed in next part is all useful structure.

J=USV^T (3)

In equation (3), U, V and S are N × N and 2 × 2 orthonormal matrixs and the diagonal matrix of N × 2 respectively.In SVD In decomposition (it is unique), monodrome is the component of diagonal matrix S, and in order from being up to minimum.I-th of single-value representation For S_ii, and the i-th column of U and V are expressed as U_iAnd V_i。

We can use SVD to calculate the Eigenvalues Decomposition of structure tensor Z:

Z=VS²V^T (4)

The most significant characteristic value of Z isCorresponding feature vector is V₁.This feature vector defines in the plane of delineation Greatest gradient comparison direction, and S₁₁It is the size of the gradient.

In the method [16] of Socolinsky and Wolff, 2 vector S₁₁V₁It is the basis of their equivalent gradient, that is, leads Gradient fields out generate the structure tensor of every pixel, the closest structure tensor defined from multichannel image of these structure tensors (equation 2).The gradient fields of every pixel are written as:

In equation 5, subscript^xIt is also represented by the picture position x, y.We (rather than write S using this name₁₁(x)V₁(x) Literary style) keep the equation more succinct.Correspondingly, J^x、Z^x、U^x、S^xAnd V^xIndicate every pixel Jacobian, Di Zenzo The every pixel SVD of tensor sum is decomposed.

In this stage, the G (x) in equation 5 is fuzzy on its symbol.Socolinsky and Wolff is by symbol It is set as matching brightness step (that is, gradient orients V₁In (R+G+B)/3).Symbol is also optimised, to maximize export gradient The integrability [19] of field.Once we secure symbol, then we write

Generally speaking, gradient fields are exportedIt is non-integrable (spiral of field is not that every place is all 0).Therefore, Socolinsky and Wolff solves output image O (x) by solving Poisson formula with least square meaning.

WhereinIndicate the diverging of gradient fields.Unfortunately, since gradient fields are non-integrable, so O must It is fixed that there is the details (gradient) not appeared in multichannel input I.For example, in Fig. 1, it is seen that " the bending in ' SW ' Pseudomorphism " does not appear in the plane of delineation of any one fusion.The pseudomorphism of this magically change is collectively referred to as at high contrast fringes " halation ".

In [1], as long as always discuss expectation equivalent gradient integrable on different scales, it integrates again Image should be just the global map of input.In effect, integration step is reduced again, to find the global map of original image (look-up table) has the derivative [23] close to the equivalent gradient of Socolinsky and Wolff.Integral principle is logical again for look-up table The often astonishing good image co-registration of transmitting (it looks like Socolinsky and Wolff image, but does not have pseudomorphism).So And output image is that the constraint of the simple overall situation function exported can produce blending image sometimes, is not shown very well more Details in each wave band of channel image.

2.1SVD, PCA and feature vector analysis

Finally, we comment on, Z^x's(feature vector associated with maximum eigenvalue) is exactly J^xRow space Main feature vector.It is vector direction, along the direction, J^xRow projection have it is maximum variation (feature vector analysis with Principal component analysis is identical, and wherein average value is not that [24] are subtracted from data before calculating maximum change direction).All numbers It can be analyzed on their row space and column space according to matrix.VectorIt is vector direction, along the direction, J^xColumn Projection, i.e. column space main feature vector have maximum variation.Vector U^xIt is only merely U^xFirst row.

3.POP image co-registration

It is from all available gradient informations of multichannel image that gradient is exported in the method for Socolinsky and Wolff The fusion mathematically established very well.That is, Socolinsky and Wolff can produce poor appearance results, this be because The discomfort integrated again for (can not field of integration) gradient fields is fixed.

The basic premise of our method is that we can carry out image co-registration, without integrating again.But we Seek the projection (linear combination) for the every pixel for only finding input channel, so that if our differential export projected image, I The equivalent gradient that will generate us and seek.As a result, not only we can take this projecting method, but also projecting direction is refined Than the main feature vector of the apposition of determinant.

POP Image Fusion Theory: in the projection of the first eigenvector of the apposition of single discrete location x Jacobian The scalar of formation (is expressed as ) there is property (wherein S^x=-1 or 1), it is assumed that function I_kIt (x) is continuous.

Prove: since differential and adduction are linear operators, and as we assume that following function be it is continuous,

Remember U^xIt is a part (see equation 3) of the singular value decomposition of Jacobian, and therefore, U^xAnd V^xAt this It is orthonormal matrix in decomposition, and S^xIt is diagonal matrix, is directly followed by

Certainly, as when we export G (x) from inner product tensor analysis, we have unknown symbol, symbol is fuzzy here Property still has.We are by S^xIt is set as -1 or 1, so that=S^xG(x)。

Although symbol selection in proving to by the export gradient map of Socolinsky and the method for Wolff, We do not need to set symbol in this way.In fact, we finally want the image that fusion has positive image value, we Do not use Socolinsky and Wolff[16] heuristic.But we select symbol, so that projected image is positive (the required property of any blending image):

s^x=sign (U^x.I(x)) (11)

Equation 11 always solved in a manner of limiting very well symbol ambiguity (and in this way compared to Socolinsky and Wolff is an important advance).

POP principle of image fusion is directed to single image point, and assumes that following multichannel image is continuous.We are uncommon Prestige understand we whether can on all picture positions aware application POP principle of image fusion, and even in following figure When as being not continuous.

Firstly, we comment on, we can be by U^xIt is written as:

U^x=J^xV^x[S^x]^-1 (12)

That is, U^xIt is the subduplicate product of Jacobian and Di ZenZo structure tensor.Because result tensor is just half It is fixed, thus characteristic value always real number and be positive, and assume that following multichannel image is continuous and feature Value is different, then U^x(the main feature vectors of outer product matrices) will also can consecutive variations.However, being zero or structure tensor in derivative It, can a picture position compared with another picture position in the image-region of characteristic value with coincidence (such as corner) To find, there are the large change on projecting direction (discontinuities).Then, and then we must interpolation or diffusion projection vector, It is allowed to limit very well on the image.We can realize this point in many ways, in the embodiment that we default, pass through This point is completed using simple intersection bilateral filter.It additionally post-processes described in the bilateral filtering and one group 3.1 After step, we have the N number of value of every pixel, limit projecting direction, and along the direction, we project N vector I (x), with Produce scalar output image.

Let us indicatesAs projected image.In POP image co-registration, scalar Output image O (x) is calculated as simple every pixel dot product.

3.1 find the algorithm of projected image

Initialization(each location of pixels is initialised to 0 projection).

1) for all picture position x, Jacobian J is calculated^x。

If 2)AndThen(at this Stage,It is sparse).

3)

4)

5)

Implementation detail

Projection vector is spread in function diffuse () expression, and to insert the value lacked, wherein marginal information is not important 's.In the embodiment that we default, this has used intersection bilateral filter, and range item is defined by original image I.Filtering pair It independently carries out, is obscured wherein Gaussian spatial is utilized, with standard deviation in every channel_d, and the standard deviation in range Parameter is σ_r.In σ_d=σ_rWhen=0, do not spread.With σ_d→ ∞ and σ_r→ ∞, diffusion becomes global average, and projects It is intended to the global weighted sum of input channel.If σ_d→ ∞ and σ_r=0, then the vector of the different value of each of image will It is associated with identical projection vector, therefore bilateral filtering step defines the surjection [23] that can be used as look-up tables'implementation.Except these Outside border condition, the standard deviation of two-sided filter should be selected, to furnish us with sought diffusion, but we are also required to ensure Space be it is sufficiently large, to avoid spatial artifacts.In our experiment, σ_dAnd σ_rIt is set as min (X, Y) × 4 and ((max (I)-min(I))/4))。

After bilateral filtering,It is fine and close, but each projecting direction is not unit vector.This in step 4 more It mends.Finally, our application extension function spread (), leave average value one fixed multiple with each mobile projecting direction Angle (diffusing step pulls in the opposite direction, and leads to projecting direction and the throwing found in step 2 in the algorithm It compares closer to average value in shadow direction).Default is that we only calculate before and after diffusion and the average angle deviation of average value. We spread vector after scaling by single factor k (k >=1), so that being identical before average angle deviation and diffusing step 's.If spread function establishes negative value, we are by clipping to 0.This zoom factor k can become according to the needs of each application Change.

Global variable

Instead of the every pixel projection in the part of the foregoing description, POP principle of image fusion can be used to implement global image fusion Scheme.We are thrown input picture Jacobian J by the standardized first eigenvector U of symbol on each pixel Shadow.

Which establishes goal gradient set G, without intrinsic symbol problem in structure tensor method before.From this In a little gradients, we form target Laplace transform (▽ O) from their second dervative, we can solve Poisson from here Equation, to find output image, but we use the integration method again [23] based on LUT from Finlayson et al.. In order to accomplish this point, we are had found from the polynomial function (multinomial function) of the Laplace transform of input channel to target The least square regression of Laplace transform.

This weight set is equivalent to look-up table, i.e. surjection, and can be applied to the polynomial function of input picture, with Generate output fusion results (O=poly (I) × Z).

This has the advantage of guarantee no pseudomorphism, and dramatically increase efficiency of algorithm.

3.2 Rapid Implementation modes

For acceleration technique, input picture can reduce sampling, and can calculate only for thumbnailIn this feelings Under condition, method be can choose to amplify caused by samplingValue, to provide throwing in each pixel of full size image plane Shadow.

The intersection bilateral filter used in the situation of full resolution projected image becomes combining bilateral amplification sampling, and And in thumbnail projected image, to amplify sampling to it, using corresponding input picture channel as full resolution Navigational figure [25].

The global variable of POP is also using the small versions of input picture come work.Projection vector, goal gradient and La Pu Lars transformation calculates under small scale first.Weight set Z is calculated, and is applied to the multinomial of full resolution input picture Formula function, to generate fusion results.

Under this thumbnail embodiment, POP method becomes extremely quickly.Using from wherein the one of Kodak's data acquisition system The colour of image arrives the example of gradation conversion, 3 to the 1 channel fusion problem on the image of 768 × 512 pixels, global drawn game Portion's variable spends 5.13 seconds and 5.16 seconds respectively under full resolution.If we use a quarter resolution ratio (each size On be all thumbnail 1/2), then this drops to 1.31 seconds and 1.41 seconds, and reduces sampling level (in each size 1/16 Be 1/4) under, drop to 0.35 second and 0.49 second.Even smaller size of thumbnail can be used, and respective performances increase, and High quality output image is kept simultaneously.

We comment on, and the calculating of this thumbnail also has the advantage that projected image can be calculated with fragment, i.e., we are from being not required to Calculate full resolution projected image.

4. experiment

Our method is compared by we with two kinds of prior art algorithms, both prior art algorithms are: The image interfusion method of Eynard et al., based on use figure Laplace transform, with find M to N channel color mapped, And frequency spectrum edge (SE) method [1] of Connah et al., it is accumulated again based on structure tensor together with the gradient based on look-up table Divide [23].

4.1RGB-NIR image co-registration

In Fig. 3, it is intended that merge traditional RGB image (3a) with near-infrared (NIR) image (3b).We apply POP image co-registration 3 times, i.e., we merge the channel R and NIR, the fusion channel G and NIR, and merge channel B and NIR.We into Row post-processing, wherein we stretch image, so that their 0.05 and 0.95 quantile is identical as original RGB image.POP image Fusion results are shown in Fig. 3 e.In order to be compared, we show both the output of frequency spectrum edge, i.e. Fig. 3 c's and Eynard et al. Output, i.e. 3d.Under identical image sequence, we show the details illustration of amplification in 3f.The output image ratio of POP method SE result captures more NIR details, and generates more natural colours, Eynard etc. than the result of Eynard et al. simultaneously The result of people has green colour cast and lacks color contrast.POP good color contrast as the result is shown, naturalness and thin Section.For this application, it is 1 that we, which are 0.1, k extensograph parameter using diminution sampling rate,.

Fig. 2: we show both stone Yuan Shi (Ishihara) colour vision test boards in (a).It is derived first in POP image co-registration Beginning projected image shows in (b)-pays attention to how sharp and sparse the image is, and shows in (c) bilateral filtering and just Image (step 3 and 4) after state.Application extension function provides final projection in (d).(a) with every pixel of (d) Product is shown in (e).In order to be compared, the output of the algorithm of our the display Socolinsky and Wolff in (f).

4.2 is colored to gradation conversion

In order to generate optimization performance to gradation conversion for colored, using the k extensograph parameter for 2, this is at this Projection vector in business with high separation is important, because 3 input dimensions must be compressed to 1 output dimension.For Another optimization of the task is that the tone in CIE LUV color space is used to take as intersecting bilateral filtering/bilateral amplification The navigational figure (this helps to ensure to have the image-region of different tones to have different projections) of sample, and be therefore easier With different output gray level values.

Table 1 shows all image colors of Kodak's data acquisition system to the comparison [27] of gradation conversion performance.POP method with The result [28] of CIE L (brightness) and Eynard et al. is compared.Used measurement is that the root of Kuhn et al. averagely adds Square (RWMS) error metrics [29] are weighed, colored difference and output gray level figure between the pixel in input RGB image are compared Strength difference as in.POP method is behaved oneself best in multiple test images.

4.3 multifocal image co-registrations

Multifocal image co-registration is another potential application, has used the gray level image pair set with different focal lengths It is studied [10] [14].The multifocal image co-registration of standard, which is related to fusion, has two gray scales of different focal length setting defeated Enter image.In each input picture, approximately half image is in focusing, therefore by two images of combination, can produce In each point all in the image of focusing.

Table 2 shows the POP image interfusion method in this task in the multifocal point image of multiple standards to upper performance Compare, is measured using standard picture fusion mass.Q_XY/FMeasurement is based on gradient similarity [30], and Q (X, Y, F) measurement is based on knot Structure similar degree image measurement (SSIM) [31] [32], andMeasurement is based on mutual information [33].As a result with week and king side Method is compared, and this method is based on the fusion [34] of multiple dimensioned weighted gradient (MWFG), and merges and compared with the DWT of standard Compared with being selected using Daubechies small echo and CM (selection maximum value) coefficient, on POP result all accounts in most of situation Wind.

Table 1: colour arrives gray scale quantitative comparison.It is averaged for the method for CIE L (brightness), Eynard et al. and POP RWMS error metric value (up to 3s.f., other than being more necessary information).All values are all × 10^-3。

Fig. 4 shows input picture and " Pepsi " image pair as a result, existing in DWT result around the visible of letter Pseudomorphism, and other two results do not have visible pseudomorphism.For this application, we are 0.5 using sampling rate is reduced, and K extensograph parameter is 2.5.

Full light photography provides the various options of focusing again of color image, allows that there is the image of the different depth of field to expose from single [35] are generated in light.The image that POP method can be used to merge these different focus becomes the single image generally in focusing. Due to be aware of in image only have one on each pixel in focusing, so our method can for this application into Row fine tuning.Here, we scale item using biggish k in spread function, and are we used sampling rate is reduced 0.5.This allows to generate the output image being perfectly clear, on each pixel all in focusing.

Fig. 7 shows image (from Huang et al. [35]), wherein four different focus images again are produced from single exposure It is raw.POP method be used to merge these different focus image become on each point all in the single image of focusing, compare and Speech, the result of the method for Eynard et al. does not show perfect details in all parts of image, and has unnatural coloured silk Color information.

Fig. 3: RGB-NIR image co-registration: " Water47 " [26] compare-original image and near-infrared input picture, frequency spectrum side Edge, Eynard et al. and POP result (details, upper left: RGB, upper right: SE, lower-left: Eynard et al., bottom right: POP).POP knot Fruit has excellent contrast and details compared with other methods.SE the result is that naturally, and be added to additional detail, and The result of Eynard et al. effectively shifts NIR details, but has green tint colour cast color.

Fig. 4: multifocal fusion: two gray scale input pictures with different focuses, DWT, MWGF[34] and POP melt Close result.

Table 2: multifocal fusion: the table of measurement results.

Fusion is exposed 4.4 more

More exposure fusion (MEF) fusions are avoided to the simple of high dynamic range (HDR) imaging and the substitution that can practice By from the input picture set with different exposures directly to output blending image come the step of generating HDR image.This It is perfect record that method, which assumes all input pictures all, and [36] are widely used in consumer photography.

The comparison of MEF algorithm [37] proposes that MEF fusion is used as weighted average problem.

Wherein O is blending image, and N is the quantity of more exposure input pictures, I_nIt (x) is brightness (or other coefficient values), and W_n (x) it is weight in x-th of pixel in n-th of exposure image.Weight factor W_n(x) can be spatial variations or complete Office.

It is subjective relatively in, based on evaluating 8 MEF algorithms by their average viewpoint score (MOS), grade from 1 to 10, The algorithm to behave oneself best is the algorithm [38] of Mertens et al..This multiple dimensioned Laplacian pyramid decomposition based on input picture, Wherein the coefficient of each image is weighted by the combination of contrast, color saturation and good exposure, is then integrated again, with Generate blending image.Method of the optimal algorithm of the second of Lee et al. based on Mertens, but increase details raising.

POP principle can be used to calculate the every pixel weight in part, or global weight is to the polynomial function of input channel.I Shown in Fig. 5 expose input picture (due to space, the 4th most dark image is not shown), and use The global variable of POP is to " cave " image sequence as a result, it is compared with two methods to behave oneself best from [37]. Notably, the optimal representation partial approach of very close Mertens of global POP result et al., but show even The dark corner in more caves.The result of Lee et al. is it will be readily apparent that details improves, and therefore image is highly write It may be why it is not preferred reason in fact.

4.5 remote sensing

The image captured for remote sensing application is generally across visible and infrared wavelength spectrum.We use from Landsat No. 5 automatic control mappers (TM) [40].No. Landsat5 TM captures 7 image channels, 3 in visible spectrum, 4 It is infrared image.From 0.45-0.51 μm (blue), 0.52-0.60 μm (green) and 0.63-0.69 μm (red) capture three can See image, we are used separately as the channel B, G and R of input RGB image.In Fig. 6 a, we show both come from Landsat The input RGB image of image set, and infrared band 5 and 7 is shown in Fig. 6 b and 6c comprising it is not deposited in RGB wave band Additional detail.All 4 infrared channels only show 2 for merging, but due to space reasons herein.4 infrared channels It is successively merged using the POP method in the method and Fig. 6 f of Socolinsky and Wolff in Fig. 6 d with R, G and channel B, then Exporting RGB channel has high and low quantile, matches with input RGB channel.In Fig. 6 e, we show both frequency spectrum edge sides Method [1] as a result, its directly merge RGB image and all 7 multi-band images.For this application, we use 0.5 contracting Small sampling rate, k extensograph parameter are 2.

SE and POP method all generates the significantly more details result compared with SW method.It is considered that the result of POP method Slightly be preferable over SE's as a result, because its details it is sharper keen and cleaner.

Fig. 6: remote sensing image fusion-Landsat No. 5 [40]: original RGB image, wave band 1-3 (a), infrared band 5 (b) Additional detail is captured with 7 (c), is merged by SW (d), SE (e) and POP (f) method with RGB.

Fig. 7: multifocal away from fusion: to capture four colored inputs with different focal point under single exposure using full light camera The fusion results of image and Eynard et al. and POP method.POP result brings the details on image with natural color into Sharper keen focal length.

4.6 merge time-lapse photography

Time-lapse photography is related to the lower image [28] for capturing same scene in different times.In the case where gray level image, The fusion of standard POP method can be used in these images, and for RGB image, the lamination of R, G and channel B is separately merged.This Fusion results generate output image, combine the protrusion details of all delay images.For this application, we use 0.5 Diminution sampling rate, k extensograph parameter be 2.

Fig. 8 shows a series of delay images of the different piece from daytime and evening (from Eynard et al. [28]), the result of the method for POP fusion and Eynard et al..It is only visible under artificial light at night in two results Details all with only daytime visible details combine, but POP result produces much more natural colour.It has to be noticed that Here that the result of the Eynard occurred et al. is different from proposing in their paper in color as a result, we oneself to defeated Enter the code that image has run their offers, code produces this result.It is considered that POP result is in either case all It is more natural fusion.

Fig. 5: more exposure fusions: " cave " image sequence, Bartlomiej Okonek are provided free.

Fig. 8: the fusion that time-lapse photography-illuminates more: in four colored input figures that the different time in daytime and evening captures Picture, the fusion results of Eynard et al. and POP method.POP result has much more natural colour and details.

5. conclusion

In this paper, we have proposed a kind of new image interfusion methods, are based on image derivative.It is avoided in ladder The degree integrability problem under integration method again, this by the main feature of the apposition of the Jacobian matrix based on image derivative to Amount calculates the projection in the input picture channel of every pixel.We demonstrate have the continuous multichannel of derivative on each point In the case where image, this generates the equivalent ladder from Di ZenZo structure tensor that derivative and Socolinsky and Wolff are found Spend equal output projected image.In true picture, derivative information is sparse, therefore we throw by input picture channel Projection coefficient is diffused into similar image-region using joint bilateral filter by shadow before with generating output image.We will This is known as the main feature vector of apposition image interfusion method.

We are it has been explained that how POP method optimizes, to improve how performance and this method of this method can answer For RGB-NIR image co-registration, colour arrives gradation conversion and multifocal image co-registration.We also compare we method and For RGB-NIR image co-registration, for colour anomaly observer image optimization and remote sensing the prior art method, and Provide the fusion of multifocal image co-registration and be delayed image of the example results for being imaged based on full light.

POP method generates visually better than other methods of our tests as a result, it, which exports image, has high level Details, and have least pseudomorphism.

It thanks you

Thank very much to EPSRC, provides with funds and support the scholarship of Alex E.Hayes.This study portion also by EPSRC approval M001768 provides with funds.Thank very much Davide Eynard to provide code and is used for Laplce color mapped side Method and example fusion results.In addition, frequency spectrum edge Co., Ltd is thanked to provide their figures of the method used in this paper As fusion results.

Bibliography

[1] D.Connah, M.S.Drew and G.D.Finlayson, " frequency spectrum edge image fusion: theory and practice ", " meter Calculation machine vision ", European Conference, the 65-80 pages, 2014.

[2] Z.Wang and Y.Ma, " being merged using the medical image of m-pcnn ", " information fusion " was rolled up for the 9, the 2nd phase, the 176-185 pages, 2008.

[3] F.Nencini, A.Garzelli, S.Baronti and L.Alparone " use the remote sensing figure of curve wave conversion As fusion ", " information fusion " was rolled up for the 8, the 2nd phase, the 143-156 pages, 2007 years.

[4] S.Li and B.Yang, " the multifocal image co-registration of using area segmentation and spatial frequency ", " image and vision Calculate ", it rolled up for the 26, the 7th phase, the 971-979 pages, 2008 years.

[5] N.Dalal and B.Triggs, " histogram of the orientation gradient of mankind's detection ", " computer vision and pattern are known Not ", IEEE meeting, volume 1, the 886-893 pages, 2005.

[6] C.Wang, Q.Yang, X.Tang and Z.Ye, " prominent using dynamic range compression retains image co-registration ", " image procossing ", ieee international conference, the 989-992 pages, 2006.

[7] W.Zhang and W.-K.Cham, " more exposure diagrams of gradient guidance ", " image procossing ", IEEE proceedings, volume 21, 4th phase, the 2318-2323 pages, 2012 years.

[8] G.Pajares and J.M.De La Cruz, " being based on Wavelet Image Fusion study course ", " image recognition ", volume 37, 9th phase, the 1855-1872 pages, 2004 years.

[9] A.Toet, " passing through the image co-registration of the tower-shaped filtering ratio of low pass ", " pattern identification journal " rolled up for the 9, the 4th phase, The 245-253 pages, 1989.

[10] S.Li, J.T.Kwok and Y.Wang, " using the multifocal image co-registration of artificial neural network ", " pattern is known Other journal " was rolled up for the 23, the 8th phase, the 985-997 pages, 2002 years.

[11] S.Di Zenzo, " concerns of more image gradients ", " computer vision, figure and image procossing ", volume 33, the 1 phase, the 116-125 pages, 1996.

【12】W." the correspondence algorithm based on feature for images match ", " the international shelves of photography and remote sensing Case " was rolled up for the 26, the 3rd phase, the 150-166 pages.1986.

[13] J.Bigun, " vision and direction ", Springer Verlag database, 2006 years.

[14] S.Han, W.Tao, D.Wang, X.-C.Tai and X.Wu, " based on the multiple dimensioned nonlinear structure tensor of set The image segmentation of Image Segmentation frame ", " image procossing ", IEEE proceedings were rolled up for the 18, the 10th phase, the 2289-2302 pages, 2009 years.

[15] B.Lu, R.Wang and C.Miao " are melted with the medical image of adaptability local geometry and wavelet transformation Close ", " energy environment science ", volume 8, the 262-269 pages, 2011.

[16] D.A.Socolinsky and L.B.Wolff, " being visualized by the multispectral image that the first rank is merged ", " figure As processing ", IEEE proceedings was rolled up for the 11, the 8th phase, the 923-931 pages, 2002 years.

[17] A.Agrawal, R.Raskar and R.Chellappa, " range from the surface reconstruction of gradient field is assorted ? ", " computer vision ", European Conference, the 578-591 pages, 2006.

[18] R.Montagna and G.D.Finlayson " reduces accumulating for the colored tensor gradient for image co-registration Divide property error ", " image procossing ", IEEE proceedings was rolled up for the 22, the 10th phase, the 4072-4085 pages, 2013 years.

[19] M.S.Drew, D.Connah, G.D.Finlayson and M.Bloj " are corrected improved by integrability Colour arrives gray scale ", " IS&T/SPIE electronic imaging ", the 72401B-72401B pages, 2009.

[20] D.Reddy, A.Agrawal and R.Chellappa " can be accumulated by using the reinforcing of 1 minimum error correction Divide property ", " computer vision and pattern identification ", IEEE meeting, the 2350-2357 pages, 2009 years.

[21] G.Piella, " for enhancing visual image co-registration: changing method ", " international computer vision is miscellaneous Will " was rolled up for the 83, the 1st phase, the 1-11 pages, 2009 years.

[22] D.A.Socolinsky, " variational method of image co-registration ", doctoral thesis, Johns Hopkins University, 2000.

[23] G.D.Finlayson, D.Connah and M.S.Drew, " the gradient field reconstruct based on look-up table ", " image Processing ", IEEE proceedings were rolled up for the 20, the 10th phase, the 2827-2836 pages, 2011 years.

[24] L.T.Moloney, " colored constant calculation method ", doctoral thesis, Stanford University, 1984 years.

[25] J.Kopf, M.F.Cohen, D.Lischinski and M.Uyttendaele, " combining bilateral amplification sampling ", ACM image proceedings was rolled up for the 26, the 3rd phase, page 96,2007.

[26] M.Brown and S.Susstrunk, and " for scene type identification multispectral screening ", " computer vision and Pattern identification ", IEEE meeting, the 177-184 pages, 2011 years.

[27] M.Cad í k, " perception of colour to greyscale image transitions is assessed ", " computer picture forum ", volume the 27, the 7th Phase, the 1745-1754 pages, 2008.

[28] D.Eynard, A.Kovnatsky and M.M.Bronstein, " mapping of Laplce's color: structure keeps face Color conversion frame ", " computer picture forum ", roll up the 33, the 2nd phase, the 215-224 pages, 2014 years.

[29] G.R.Kuhn, M.M.Oliveira and L.A.Fernandes, " for colored to the improved of grey scale mapping Contrast improvement method ", " visual computer ", volume 24, the 7-9 phase, the 505-514 pages, 2008 years.

[30] C.Xydeas and V." target image fusion performance measurement ", " electronic letters, vol ", volume the 36, the 4th Phase, the 308-309 pages, 2000.

[31] Z.Wang, A.C.Bovik, H.R.Sheikh and E.P.Simoncelii, " image quality evaluation: from error Visuality arrives structural similarity ", " image procossing ", IEEE proceedings was rolled up for the 13, the 4th phase, the 600-612 pages, 2014 years.

[32] C.Yang, J.-Q.Zhang, X.-R.Wang and X.Liu, " novel for image co-registration is based on similarity Quality metric ", " information fusion ", roll up the 9, the 2nd phase, the 156-160 pages, 2008 years.

[33] M.Hossny, S.Nahavandi and D.Creighton, " to the information measurement for image co-registration performance Comment ", " electronic letters, vol " was rolled up for the 44, the 18th phase, the 1066-1067 pages, 2008 years.

[34] Z.Zhou, S.Li and B.Wang, " fusion based on multiple dimensioned weighted gradient for multifocal point image ", " information fusion ", 2014.

[35] R.Ng, M.Levoy, M.Br é dif, G.Duval, M.Horowitz and P.Hanrahan is " complete with hand-held The light field photography of photocamera shooting ", " computer science and technology report ", volume 2, o. 11th, 2005.

[36] E.Reinhard, W.Heidrich, P.Debevec, S.Pattanaik, G.Ward and K.Myszkowski, " high dynamic range imaging: acquisition, display and the illumination based on image ", rub root Kaufman, and 2010.

[37] K.Ma, K.Zeng and Z.Wang, " being assessed for the perceived quality of more exposure images fusion ", " at image Reason ", IEEE proceedings, volume 24, o. 11th, the 3345-3356 pages, 2015.

[38] T.Mertens, J.Kautz and F.Van Reeth, " exposure fusion: for the simple of high dynamic range imaging And the substitution that can be practiced ", " computer picture forum " rolled up for the 28, the 1st phase, the 161-171 pages, 2009 years.

[39] Z.G.Li, J.H.Zheng and S.Rahardja, " details enhancing exposure fusion ", " image procossing ", IEEE meeting Periodical, volume 21, o. 11th, the 4672-4676 pages, 2012.

[40] " No. 5 pictures of NASA:Landsat ", http://landsat.usgs.gov, access time: 2015-4-22.

Alex E.Hayes Alex E.Hayes obtained his English Master of Arts in 2009 from University of St Andrews Degree, and obtained from University of East Anglia in 2013 his development of games Master of Science's degree.He is at present in Dongan lattice Leah university is dedicated to his doctor's research.His research concentrates on image co-registration, the method for being based particularly on derivative, and answers For RGB-NIR image co-registration.

Graham D.Finlayson Graham D.Finlayson is College of Computer Science of University of East Anglia Professor.He was in addition UEA in 1999, and formal professorship was awarded at 30 years old for he at that time.He is and is still the institute The minimus people for obtaining professor and appointing.Graham receives an education in the College of Computer Science of University of Strathclyde first And his master's doctorate is then majored in Simonfraser University, best doctoral thesis president prize is awarded in he there Chapter.Before UEA is added, Graham is the lecturer of York University, followed by the color of Derby university and imaging (Color and Imaging founder and senior lecturer).Professor Finlayson, which pays close attention to, calculates how we see, and his research Cover computer science (algorithm), engineering (embedded system) and psychophysics (visual identity).

Claims (44)

1. one kind is used for from multiple, N number of, the method that corresponding input picture channel generates output image, which comprises

Determine the Jacobian matrix in the multiple corresponding input picture channel；

Calculate the main feature vector of the apposition of the Jacobian matrix；

Setting symbol associated with the main feature vector, thus by the input picture channel picture of the main eigenvector projection Element leads to positive scalar value；And

Generate every pixel projection of the output image as the input channel on principal eigenvector direction.

2. the method as described in claim 1, wherein it is further comprising the steps of to calculate step:

For each element of the Jacobian matrix for non-zero, sparse N vector projection image is generated from the Jacobian matrix；With And

By the element for the Jacobian matrix that the sparse N vector projection image filling is zero.

3. according to the method described in claim 2, wherein the filling includes by the way that the vector of each neutral element is defined as this The average value of ground neighborhood is inserted.

4. method as claimed in claim 3, wherein the average value is edge sensitive.

5. the method as described in claim 2,3 or 4, wherein the filling includes carrying out to the sparse N vector projection image Bilateral filtering.

6. method as claimed in claim 5, wherein the two-sided filter includes intersecting two-sided filter.

7. the method according to any one of claim 2 to 6, wherein filling step includes making the N vector projection image Smoothing.

8. the method according to any one of claim 2 to 7, wherein filling step includes N vector projection figure described in interpolation Picture.

9. the method according to any one of claim 2 to 6, wherein filling step includes to the N vector projection image Execute edge sensitive diffusion.

10. the method as described in any one of claim 5 or 6, wherein filter step includes independent to the N vector projection figure Each channel of picture is filtered.

11. the method according to any one of claim 2 to 10 further includes scaling each vector after filling, with tool There is unit length.

It, will be every 12. the method according to any one of claim 2 to 10 further includes the spread vector after filling A component of a vector moves away the angle of the average value one fixed multiple.

13. method according to any of the preceding claims, further comprising the steps of:

The determination is executed to the input picture channel for reducing sampling and calculates step, and to the calculated main feature vector Sampling is amplified, for using in the generation step.

14. method according to any preceding claims, wherein each unique input picture vector maps directly to individually Projection vector.

15. method according to any of the preceding claims, wherein by unique input picture vector and main feature vector Between Mapping implementation be look-up table.

16. method according to any of the preceding claims, wherein the input picture has N number of channel and described Exporting image has M channel, and the main feature vector includes being mapped to the Jacobian of the N of the input picture × 2 Target M × 2 exports every pixel M × N matrix transformation of Jacobian.

17. the method according to claim 11 further includes respective M × N transformation pair by the input picture channel The input picture channel carries out the step of every pixel transform.

18. the method as described in claim 16 or 17, wherein the M × N is converted the input picture Jacobi's row of the N × 2 Column is mapped to M × 2 and strengthens Jacobi's pairing determinant.

19. method as claimed in claim 18, wherein it is further comprising the steps of to calculate step:

The image of sparse N × 2 is inserted by the element for the Jacobian matrix for being zero to generate sparse M × N changing image

20. method described in any one of 6 to 19 according to claim 1, further comprising the steps of:

The determination is executed to the input picture channel for reducing sampling and calculates step, and calculated M × N is converted and is amplified Sampling, for being used in the generation step.

21. method described in any one of 6 to 20 according to claim 1, wherein each unique input picture vector directly maps Vector is converted to single M × N.

22. the method as described in any one of claim 16 to 21, wherein unique input picture vector and M × N are converted it Between mapping be embodied as look-up table.

23. one kind is for from multiple, N number of, the system that corresponding input picture channel generates output image, the system comprises:

It is arranged to access the input in N number of input picture channel；

It is configured to execute the processor of the computer program code for executing image processing module, comprising:

It is configured to determine the computer program code of the Jacobian matrix in the multiple corresponding input picture channel；

It is configured to calculate the computer program code of the main feature vector of the apposition of the Jacobian matrix；

It is configured to setting symbol associated with the main feature vector, thus by the input of the main eigenvector projection Image channel pixel leads to the computer program code of positive scalar value；And

It is configured to generate calculating of the output image as every pixel projection of the input channel on principal eigenvector direction Machine program code.

24. system according to claim 23, the computer program code to calculate is further comprising the steps of:

It is configured to each element for the Jacobian matrix of non-zero and generates sparse N vector from the Jacobian matrix and throw The computer program code of shadow image；And

It is configured to insert the computer of the sparse N vector projection image for the element for the Jacobian matrix for being zero Program code.

25. system according to claim 24, wherein filling includes by the way that the vector of each neutral element is defined as local The average value of neighborhood is inserted.

26. system according to claim 25, wherein the average value is edge sensitive.

27. according to system described in claim 23,24 or 25, wherein filling include to the sparse N vector projection image into Row bilateral filtering.

28. system according to claim 27, wherein two-sided filter includes intersecting two-sided filter.

29. the system according to any one of claim 24 to 28, wherein being configured to the computer journey of filling Sequence code includes the computer program code for being configured to make the N vector projection image smoothing.

30. the system according to any one of claim 24 to 29, wherein being configured to the computer of filling Program code includes the computer program code for being configured to N vector projection image described in interpolation.

31. the system according to any one of claim 24 to 28, wherein being configured to the computer of filling Program code includes the computer program code for being configured to execute the N vector projection image edge sensitive diffusion.

32. the system according to any one of claim 27 or 28, wherein filter is arranged to independently to the N Each channel of vector projection image is filtered.

33. the system according to any one of claim 24 to 32, wherein the processor is configured to execute meter Calculation machine program code, to scale each vector after filling, with unit length.

34. the system according to any one of claim 24 to 33, wherein the processor is configured to execute meter Each component of a vector is moved away average value one fixed multiple with the spread vector after filling by calculation machine program code Angle.

35. the system according to any one of claim 23 to 34, wherein the processor is configured to execute calculating Machine program code executes the determination and meter to the input picture channel for reducing sampling to obtain the input channel for reducing sampling Step is calculated, and sampling is amplified to calculated main feature vector, to be used for generation step.

36. the system according to any one of claim 23 to 35, wherein each unique input picture vector directly maps To single projection vector.

37. the system according to any one of claim 23 to 36, further include unique main feature of input picture vector sum to Look-up table mapping between amount, the system are arranged to access the look-up table, with the determination main feature vector, be used for Generate the output image.

38. the system according to any one of claim 23 to 37, wherein the input picture has N number of channel and institute Stating output image has M channel, and the main feature vector includes mapping the Jacobian of the N of the input picture × 2 Every pixel M × N matrix transformation of Jacobian is exported to target M × 2.

39. the system according to claim 38, wherein the processor is further configured to execute computer program Code, to carry out every pixel transform to the input picture channel by the respective M × N in input picture channel transformation.

40. the system according to claim 38 or 39 the, wherein M × N is converted the input picture Jacobi of the N × 2 Determinant Mapping is that M × 2 reinforces Jacobi's pairing determinant.

41. system according to claim 40, wherein the processor is configured to execute computer program code, with The image of sparse N × 2 is inserted by the element for the Jacobian matrix for being zero to generate sparse M × N changing image

42. the system according to any one of claim 38 to 41, wherein the processor is further configured to execute Computer program code to execute the determination and calculating to the input picture channel for reducing sampling, and is calculated to described M × N the transformation come amplifies sampling, for generating the output image.

43. the system according to any one of claim 38 to 42, wherein each unique input picture vector directly maps Vector is converted to single M × N.

44. the system according to any one of claim 38 to 43 further includes unique input picture vector sum M × N transformation Between look-up table mapping, the system is arranged to access the look-up table, to determine for generating the output image M × N transformation.