CN107369169A - The approximate most like image block matching method transmitted based on direction alignment with matching that a kind of GPU accelerates - Google Patents

The approximate most like image block matching method transmitted based on direction alignment with matching that a kind of GPU accelerates Download PDF

Info

Publication number
CN107369169A
CN107369169A CN201710426618.8A CN201710426618A CN107369169A CN 107369169 A CN107369169 A CN 107369169A CN 201710426618 A CN201710426618 A CN 201710426618A CN 107369169 A CN107369169 A CN 107369169A
Authority
CN
China
Prior art keywords
image
pixel coordinate
coordinate value
source images
target image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710426618.8A
Other languages
Chinese (zh)
Other versions
CN107369169B (en
Inventor
赵汉理
郭和炀
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wenzhou University
Original Assignee
Wenzhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wenzhou University filed Critical Wenzhou University
Priority to CN201710426618.8A priority Critical patent/CN107369169B/en
Publication of CN107369169A publication Critical patent/CN107369169A/en
Application granted granted Critical
Publication of CN107369169B publication Critical patent/CN107369169B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/30Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/32Determination of transform parameters for the alignment of images, i.e. image registration using correlation-based methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/22Matching criteria, e.g. proximity measures

Abstract

The embodiment of the invention discloses the approximate most like image block matching method transmitted based on direction alignment with matching that a kind of GPU accelerates, including target image and source images are determined, calculate the deflection image of target image and source images;And define similitude adaptation function;Each pixel coordinate value in target image is calculated using similitude adaptation function, and each corresponding similitude adaptation function value in the target image of transmitting is preceding K minimum most like image blocks;The preceding K most like image blocks in target image, the similitude adaptation function value between the corresponding K*K in source images most like image blocks of each pixel coordinate value is obtained using similitude adaptation function, filtering out each pixel coordinate value in target image, each the corresponding similitude adaptation function value in source images is preceding K minimum most like image blocks while exported.Implement the present invention, the process performance of image block matching method can be effectively improved, play GPU efficient parallel disposal abilities.

Description

The approximate most like image transmitted based on direction alignment with matching that a kind of GPU accelerates Block matching method
Technical field
The present invention relates to computer graphics and technical field of image processing, more particularly to a kind of speed based on direction alignment with Match the most like image block matching method transmitted.
Background technology
Most like image block matching method is a kind of each most like image block calculated included in target image in source The Computer Image Processing method of most like K most like image blocks in image, leads in computer graphics and image procossing Domain has a wide range of applications.In recent years, as the development of computer technology, increasing researcher focus on most like figure As the research work of block matching method.
At present, there is the method much on most like image Block- matching to be proposed out, can substantially be divided into three kinds Method:Matching process based on tree structure, the matching process based on Hash table structure and based on image space search strategy Matching process.
, it is necessary to build relative complex tree form data structure in the match party method of completing the square based on tree structure, and need Very big internal memory is consumed (referring specifically to Kumar N, Zhang L, Nayar S.What is a good nearest neighbors algorithm for finding similar patches in images[C]//European Conference on Computer Vision,Springer-Verlag,Berlin,Heidelberg,2008,pp.364– 378)。
In the matching process based on Hash table structure, matching efficiency is improved using Hash table search, but Hash table Foundation be difficult to the Parallel Implementation on GPU (Graphics Processing Unit, graphics processing unit), efficiency improves limited (referring specifically to Korman S, Avidan S.Coherency sensitive hashing [J] .IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,38(6),1099–1112)。
In matching process based on image space search strategy, searched at random with image space neighbouring relations and global image Rope strategy improves the efficiency of matching (referring specifically to Barnes C, Shechtman E, Finkelstein A, et al.PatchMatch:a randomized correspondence algorithm for structural image editing[J].ACM Transactions on Graphics,2009,28(3):24:1-24:11)。
Certainly, also there is the matching process based on image space search strategy by traveling through multiple given anglecs of rotation and contracting Ratio is put further to improve matching precision (referring specifically to Barnes C, Shechtman E, Goldman D B, et al.The generalized PatchMatch correspondence algorithm[C]//European Conference on Computer Vision, Springer, Berlin, 2010, pp.29-43), but the time complexity of above-mentioned traversal is with giving The fixed anglec of rotation and the quantity of scaling are directly proportional.
As can be seen here, the matching efficiency of most like image block matching method also has very big room for improvement in the prior art.
The content of the invention
The purpose of the embodiment of the present invention is to provide a kind of the approximate with matching transmission most based on direction alignment of GPU acceleration Similar image block matching method, the efficiency of most like image Block- matching can be effectively improved, having each step can be simultaneously Row, play efficient parallel disposal ability in GPU.
In order to solve the above-mentioned technical problem, the embodiments of the invention provide a kind of GPU accelerate based on direction alignment with Approximate most like image block matching method with transmission, methods described include:
S1, target image and source images are determined, and the target image and the source images are based respectively on structure tensor Respective deflection image is calculated, and further according to obtained by the target image and the source images and each calculating Deflection image, define calculate two image block distances similitude adaptation function;
S2, determine that each pixel coordinate value corresponds to the K transmitted in the target image respectively in the target image Individual most like image block, and utilize each pixel coordinate value in the similitude adaptation function calculating target image and its Similitude adaptation function value between the corresponding K being delivered in the target image most like image block, obtains the target figure Corresponding be delivered in the target image with similitude adaptation function value is minimum to each pixel coordinate value respectively as in Preceding K most like image blocks;K is more than 1 positive integer;
S3, by each pixel coordinate value in the obtained target image respectively correspond to be delivered in the target image Transmit, obtain each in the target image again for preceding K minimum most like image blocks with similitude adaptation function value Individual pixel coordinate value, which is correspondingly delivered in the source images, has K*K most like image blocks, and is matched using the similitude It is individual most like that function calculates the corresponding K*K being delivered in the source images of each pixel coordinate value in the target image Similitude adaptation function value between image block, filters out in the target image and is each calculated in each pixel coordinate value Similitude adaptation function value exports simultaneously for preceding K minimum most like image blocks.
Wherein, " target image and the source images are based respectively on into structure tensor to calculate respectively in the step S1 From corresponding deflection image " specifically include:
3X3 convolution weight template is given, and utilizes mesh described in the convolution weight formwork calculation of the given 3X3 The gradient field image of the horizontal direction of logo image and the gradient field image of vertical direction, and calculate the level side of the source images To gradient field image and vertical direction gradient field image;
According to the gradient field image of the horizontal direction of the target image calculated and vertical direction and described calculate Source images the gradient field image of horizontal direction and the gradient field image of vertical direction, calculate respectively the target image and Each self-corresponding structure tensor image of source images, and the target image and source images that are further calculated according to are each Corresponding structure tensor image, obtain the deflection image of the target image and the source images.
Wherein, the deflection image of the target image is to pass through formulaTo realize, the side of the source images It is to pass through formula to angle image To realize;Wherein,
The structure tensor graphical representation of the target image is SA=(SAx,SAy,SAz);SAx(p)=Ax(p)·Ax(p), SAy(p)=Ax(p)·Ay(p), SAz(p)=Ay(p)·Ay(p); P is any one pixel coordinate value of the target image;I, j is whole Number,
The structure tensor graphical representation of the source images is SB=(SBx,SBy,SBz);SBx(q)=Bx(q)·Bx (q), SBy(q)=Bx(q)·By(q), SBz(q)=By(q)·By(q); Q is any one pixel coordinate value of the source images.
Wherein, the step S2 is specifically included:
S21, obtain in the target image any one pixel coordinate value and its in the target image by parallel K the first random pixel coordinate values of random number generation function generation, obtain each pixel coordinate value pair in the target image The K most like image blocks formed in the target image by K the first random pixel coordinate values should be delivered in;
S22, the first iterations is set;
S23, the first spatial offset and the first random search region length of side are set;
Each pixel coordinate value corresponds to respectively in S24, current first spatial offset of acquisition and the target image Current K most like image blocks;
S25, judge whether current first spatial offset is more than or equal to 1;If it is, perform next step S26; If it is not, then jump to step S27;
S26, current first spatial offset got according to, by each pixel coordinate in the target image Corresponding current K most like image blocks are updated value respectively, and calculate the mesh using the similitude adaptation function Pixel coordinate value in the target image that each pixel coordinate value is each got correspondingly with it respectively in logo image And the similitude adaptation function value between current K most like image blocks, and further retain each in the target image Pixel coordinate value calculates preceding K most like image blocks of the similitude adaptation function value for minimum as each self-corresponding current K most like image blocks, and by behind current first spatial offset divided by 2, return to step S24;
S27, obtain each pixel coordinate value point in the current first random search region length of side and the target image Not corresponding current K most like image blocks;
S28, continue to judge whether the current first random search region length of side is more than or equal to 1;If it is, under performing One step S29;If it is not, then jump to step S210;
In S29, the target image the current K most like image blocks of each pixel coordinate value respectively to be respectively in It is secondary more that the heart each correspondingly randomly selects a pixel coordinate value progress in the range of the current first random search region length of side Newly, and using the similitude adaptation function each pixel coordinate value is calculated in the target image respectively each and one Similitude matching letter between pixel coordinate value and current K a most like image block in the target image got corresponding to one Numerical value, and further retain after secondary renewal that each pixel coordinate value calculates similitude matching letter in the target image Numerical value is preceding K minimum most like image blocks as each self-corresponding current K most like image blocks, and will it is described currently After the first random search region length of side divided by 2, return to step S27;
S210, judge whether first iterations is equal to 0;If it is, perform next step S211;If not, After first iterations is subtracted into 1, return to step S23;
S211, terminate to calculate, and each pixel coordinate value in the target image each finally obtained respectively similar Property adaptation function value passed for preceding K minimum most like image blocks as each pixel coordinate value in the target image is corresponding Pass the preceding K most like image block outputs in the target image.
Wherein, the step S3 is specifically included:
Any one pixel coordinate value passes through parallel random number in the source images in S31, the acquisition target image K the second random pixel coordinate values of generating function generation, obtain the corresponding transmission of each pixel coordinate value in the target image The K most like image blocks formed in the source images by K the second random pixel coordinate values;
S32, secondary iteration number is set;
S33, second space offset and the second random search region length of side are set;
Each pixel coordinate value corresponds to respectively in S34, the current second space offset of acquisition and the target image The current K*K most like image blocks formed in the source images;
S35, judge whether the current second space offset is more than or equal to 1;If it is, perform next step S36; If it is not, then jump to step S37;
S36, the current second space offset got according to, by each pixel coordinate in the target image The current K most like image blocks that value corresponds in the source images respectively are updated, and utilize similitude matching letter Number is calculated in the source images that each pixel coordinate value is each got correspondingly with it respectively in the target image Pixel coordinate value and current K most like image blocks between similitude adaptation function value, and further retain by the mesh Each pixel coordinate value calculates similitude adaptation function value as preceding K minimum most like image block conducts in logo image Each self-corresponding current K most like image block, and will be after the currently second space offset divided by 2, return to step S33;
S37, obtain each pixel coordinate value point in the current second random search region length of side and the target image The current K*K most like image blocks formed in the source images are not corresponded to;
S38, continue to judge whether the current second random search region length of side is more than or equal to 1;If it is, under performing One step S39;If it is not, then jump to step S310;
Each pixel coordinate value corresponds to the current K most phases in the source images respectively in S39, the target image Each corresponded in the range of the current second random search region length of side centered on each respectively like image block and randomly select one Individual pixel coordinate value carries out secondary renewal, and calculates each picture in the target image using the similitude adaptation function Pixel coordinate value and current K a most like image block in the source images that plain coordinate value is got correspondingly with it respectively Between similitude adaptation function value, and further retain by each pixel coordinate value institute in the target image after secondary renewal It is preceding K minimum most like image blocks as each self-corresponding current K most like figures to calculate similitude adaptation function value As block, and by behind the current second random search region length of side divided by 2, return to step S37;
Step S310, each pixel coordinate value is obtained in the target image in the target image according to step S2 K most like image blocks, each pixel coordinate value in the obtained target image is corresponded to respectively and is delivered in the mesh There is similitude adaptation function value to be transmitted again for preceding K minimum most like image blocks in logo image, obtain the target figure Each pixel coordinate value is correspondingly delivered in the source images as in, and there are K*K most like image blocks to be updated three times, And calculate each pixel coordinate value in the target image using the similitude adaptation function and corresponded respectively with it The source images got in pixel coordinate value and the similitude adaptation function value between current K most like image blocks, and Further retain and similitude adaptation function value is calculated by each pixel coordinate value in the target image after updating three times It is preceding K minimum most like image blocks as current K most like image blocks;
S311, judge whether the secondary iteration number is equal to 0;If it is, perform next step S312;If not, After the secondary iteration number is subtracted into 1, return to step S33;
S312, terminate to calculate, and gained phase is calculated by each last in each pixel coordinate value in the target image Exported simultaneously for preceding K minimum most like image blocks like property adaptation function value.
It is approximate most like with matching transmission that the embodiment of the present invention additionally provides being alignd based on direction for another GPU acceleration Image block matching method, methods described include:
S5, target image and source images are determined, and the target image and the source images are based respectively on structure tensor Respective deflection image is calculated, and further according to obtained by the target image and the source images and each calculating Deflection image, define calculate two image block distances similitude adaptation function;
S6, determine that each pixel coordinate value corresponds to K transmitted in the source images most respectively in the source images Similar image block, and calculate the corresponding biography of each pixel coordinate value in the source images using the similitude adaptation function The similitude adaptation function value between the K most like image blocks in the source images is passed, obtains each in the source images Pixel coordinate value, which corresponds to be delivered in respectively, has similitude adaptation function value for preceding K minimum most like figures in the source images As block;K is more than 1 positive integer;
S7, determine that each pixel coordinate value is correspondingly delivered in K most phases in the source images in the target image Like image block, and corresponded to respectively according to each pixel coordinate value in the source images there is similitude in the source images It is preceding K minimum most like image blocks with functional value, obtains the corresponding transmission of each pixel coordinate value in the target image There are K*K most like image blocks in the source images, the side according to corresponding to the target image and source images and its they To angle image, definition calculates the similitude adaptation function of two image block distances, and further utilizes similitude matching letter Number calculates the corresponding K*K being delivered in the source images of each pixel coordinate value most like figure in the target image As the similitude adaptation function value between block, each calculated phase is filtered out in the target image in each pixel coordinate value Exported simultaneously for preceding K minimum most like image blocks like property adaptation function value.
Wherein, " target image and the source images are based respectively on into structure tensor to calculate respectively in the step S5 From corresponding deflection image " specifically include:
3X3 convolution weight template is given, and utilizes mesh described in the convolution weight formwork calculation of the given 3X3 The gradient field image of the horizontal direction of logo image and the gradient field image of vertical direction, and calculate the level side of the source images To gradient field image and vertical direction gradient field image;
According to the gradient field image of the horizontal direction of the target image calculated and vertical direction and described calculate Source images the gradient field image of horizontal direction and the gradient field image of vertical direction, calculate respectively the target image and Each self-corresponding structure tensor image of source images, and the target image and source images that are further calculated according to are each Corresponding structure tensor image, obtain the target image and each self-corresponding deflection image of the source images.
Wherein, the deflection image of the target image is to pass through formulaTo realize, the source images Deflection image is to pass through formula To realize;Wherein,
The structure tensor graphical representation of the target image is SA=(SAx,SAy,SAz);SAx(p)=Ax(p)·Ax (p), SAy(p)=Ax(p)·Ay(p), SAz(p)=Ay(p)·Ay(p); P is any one pixel coordinate value of the target image;I, j is whole Number,
The structure tensor graphical representation of the source images is SB=(SBx,SBy,SBz);SBx(q)=Bx(q)·Bx (q), SBy(q)=Bx(q)·By(q), SBz(q)=By(q)·By(q); Q is any one pixel coordinate value of the source images.
Wherein, the step S6 is specifically included:
S61, obtain in the source images any one pixel coordinate value and its in the source images by parallel random Number generating function K the 3rd random pixel coordinate values of generation, obtain each pixel coordinate value in the source images and correspond to respectively It is delivered in the K most like image blocks formed in the source images by K the 3rd random pixel coordinate values;
S62, the 3rd iterations is set;
S63, the 3rd spatial offset and the 3rd random search region length of side are set;
S64, obtain in current 3rd spatial offset and the source images corresponding to each pixel coordinate value difference Current K most like image blocks;
S65, judge whether current 3rd spatial offset is more than or equal to 1;If it is, perform next step S66; If it is not, then jump to step S67;
S66, current 3rd spatial offset got according to, by each pixel coordinate value in the source images Corresponding current K most like image blocks are updated respectively, and calculate the source figure using the similitude adaptation function Pixel coordinate value as in the source images that are each got correspondingly with it respectively of each pixel coordinate value and work as Similitude adaptation function value between preceding K most like image blocks, and further retain each pixel coordinate in the source images Value calculates preceding K most like image blocks of the similitude adaptation function value for minimum as each self-corresponding current K most phases Like image block, and by behind current 3rd spatial offset divided by 2, return to step S64;
S67, obtain each pixel coordinate value difference in the current 3rd random search region length of side and the source images Corresponding current K most like image blocks;
S68, continue to judge whether the current 3rd random search region length of side is more than or equal to 1;If it is, under performing One step S69;If it is not, then jump to step S610;
The current K most like image blocks of each pixel coordinate value are respectively centered on each in S69, the source images It is each corresponding in the range of the length of side of current 3rd random search region to randomly select a secondary renewal of pixel coordinate value progress, And it is one-to-one with it respectively using the similitude adaptation function to calculate each pixel coordinate value in the source images The similitude adaptation function value between pixel coordinate value and current K a most like image block in the source images got, and enter Each pixel coordinate value calculates similitude adaptation function value as minimum in the source images after the secondary renewal of one step reservation Preceding K most like image blocks searched at random as each self-corresponding current K most like image blocks, and by the described current 3rd After the rope region length of side divided by 2, return to step S67;
S610, judge whether the 3rd iterations is equal to 0;If it is, perform next step S611;If not, After 3rd iterations is subtracted into 1, return to step S63;
S611, terminate to calculate, and each pixel coordinate value in the source images is each finally obtained into similitude respectively Adaptation function value is each corresponded to as each pixel coordinate value in the source images for preceding K minimum most like image blocks and passed Pass the preceding K most like image block outputs in the source images.
Wherein, the step S7 is specifically included:
Any one pixel coordinate value passes through parallel random number in the source images in S71, the acquisition target image K the 4th random pixel coordinate values of generating function generation, obtain the corresponding transmission of each pixel coordinate value in the target image The K most like image blocks formed in the source images by K the 4th random pixel coordinate values;
S72, the 4th iterations is set;
S73, the 4th spatial offset and the 4th random search region length of side are set;
Each pixel coordinate value corresponds to respectively in S74, current 4th spatial offset of acquisition and the target image The current K*K most like image blocks formed in the source images;
S75, judge whether current 4th spatial offset is more than or equal to 1;If it is, perform next step S76; If it is not, then jump to step S77;
S76, current 4th spatial offset got according to, by each pixel coordinate in the target image Value corresponds to the current K*K most like image blocks formed in the source images and is updated respectively, and utilizes the similitude Adaptation function calculates the source that each pixel coordinate value is each got correspondingly with it respectively in the target image Similitude adaptation function value between pixel coordinate value and current K in image a most like image block, and further retain by Each pixel coordinate value calculates similitude adaptation function value as preceding K minimum most like images in the target image Block is as each self-corresponding current K most like image blocks, and by behind current 4th spatial offset divided by 2, returns Step S74;
S77, obtain each pixel coordinate value point in the current 4th random search region length of side and the target image The current K*K most like image blocks formed in the source images are not corresponded to;
S78, continue to judge whether the current 4th random search region length of side is more than or equal to 1;If it is, under performing One step S79;If it is not, then jump to step S710;
Each pixel coordinate value corresponds to the current K most phases in the source images respectively in S79, the target image Each corresponded in the range of the length of side of current 4th random search region centered on each respectively like image block and randomly select one Individual pixel coordinate value carries out secondary renewal, and calculates each picture in the target image using the similitude adaptation function Pixel coordinate value and current K a most like image block in the source images that plain coordinate value is got correspondingly with it respectively Between similitude adaptation function value, and further retain by each pixel coordinate value institute in the target image after secondary renewal It is preceding K minimum most like image blocks as each self-corresponding current K most like figures to calculate similitude adaptation function value As block, and by behind the current 4th random search region length of side divided by 2, return to step S77;
Step S710, K of each pixel coordinate value in the source images in the source images is obtained according to step S6 Individual most like image block, each pixel coordinate value in the obtained target image is corresponded to respectively and is delivered in the source images In there is similitude adaptation function value to be transmitted again for current K minimum most like image blocks, obtain in the target image Each pixel coordinate value is correspondingly delivered in the source images, and there are K*K most like image blocks to be updated three times, and profit Each pixel coordinate value in the target image is calculated with the similitude adaptation function to obtain correspondingly with it respectively The similitude adaptation function value between pixel coordinate value and current K a most like image block in the source images got, and enter one Step retains calculates similitude adaptation function value as most by each pixel coordinate value in the target image after updating three times Preceding K small most like image blocks are as current K most like image blocks;
S711, judge whether the 4th iterations is equal to 0;If it is, perform next step S712;If not, After first iterations then is subtracted into 1, return to step S73;
S712, terminate to calculate, and gained phase is calculated by each last in each pixel coordinate value in the target image Exported simultaneously for preceding K minimum most like image blocks like property adaptation function value.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing There is the required accompanying drawing used in technology description to be briefly described, it should be apparent that, drawings in the following description are only this Some embodiments of invention, for those of ordinary skill in the art, without having to pay creative labor, according to These accompanying drawings obtain other accompanying drawings and still fall within scope of the invention.
Fig. 1 is provided by the invention based on a reality of the direction alignment with matching the most like image block matching method transmitted Apply the flow chart of example;
Fig. 2 is the flow chart of step S2 concrete methods of realizing in Fig. 1;
Fig. 3 is the flow chart of step S3 concrete methods of realizing in Fig. 1;
Fig. 4 is another of the most like image block matching method with matching transmission provided by the invention that alignd based on direction The flow chart of embodiment;
Fig. 5 is the flow chart of step S6 concrete methods of realizing in Fig. 4;
Fig. 6 is the flow chart of step S7 concrete methods of realizing in Fig. 4.
Embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, it is right below in conjunction with drawings and Examples The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and It is not used in the restriction present invention.
As shown in figure 1, for a kind of GPU for being proposed in the present invention accelerate based on direction align with match transmission it is approximate most One embodiment of similar image block matching method, in the embodiment method lay particular emphasis on after first pretreatment goal image again with source figure As the flow matched, specifically include:
Step S1, target image and source images are determined, and the target image and the source images are based respectively on structure Extensometer calculates respective deflection image, and further according to the target image and the source images and each calculating The deflection image of gained, define the similitude adaptation function for calculating two image block distances;
Detailed process, which is that a given image is to be matched, is used as target image A, and the width and height of image are denoted as w respectively1 And h1, the pixel coordinate value p in target image A is denoted as p=(xp,yp), then xp∈{0,1,2,K,w1- 1 }, yp∈{0,1,2,K, h1-1};Give another image and be denoted as w respectively as source images B, the width and height of image2And h2, the pixel in source images B Coordinate value q is denoted as q=(xq,yq), then xq∈{0,1,2,K,w2- 1 }, yq∈{0,1,2,K,h2-1}。
It should be noted that each pixel coordinate value p in target image A uniquely correspond to the pixel coordinate value Centered on square image block, be designated as PA(p).Each pixel coordinate value q in source images B uniquely correspond to the pixel Square image block centered on coordinate value, is designated as PB(q)。
3X3 convolution weight template is given, such asAnd profit With 3X3 given herein above convolution weight template, according to formulaWithCalculate the gradient field image A of target image A horizontal directionxWith vertical side To gradient field image Ay;Wherein, p is target image A any one pixel coordinate value;I, j is integer;And according to calculating The target image A horizontal direction and the gradient field image of vertical direction gone out, target image A structure tensor image is calculated, And target image A deflection image O is further obtained according to the target image A calculated structure tensor imageA
Similarly, using 3X3 convolution weight template, such asRoot According to formulaWithCalculate The gradient field image B of source images B horizontal directionxWith the gradient field image B of vertical directiony;Wherein, q is any of source images B One pixel coordinate value;I, j is integer;And according to the source images B horizontal direction and the gradient fields of vertical direction calculated Image, source images B structure tensor image is calculated, and further obtained according to the source images B calculated structure tensor image To source images B deflection image OB
It should be noted that target image A structure tensor image is designated as SA=(SAx,SAy,SAz), and for target figure As any pixel coordinate value p in A, calculation formula uses:SAx(p)=Ax(p)·Ax(p), SAy(p)=Ax(p)·Ay(p), SAz(p)=Ay(p)·Ay(p);Wherein, symbol represents dot product operator.
According to target image A structure tensor image, target image A deflection image OAPass through formulaTo realize;Wherein, Arctan represents the arctan function in mathematics, for calculating angle.
Similarly, source images B structure tensor image is designated as SB=(SBx,SBy,SBz), and for any in source images B Pixel coordinate value q, calculation formula use:SBx(p)=Bx(p)·Bx(p), SBy(p)=Bx(p)·By(p), SBz(p)=By (p)·By(p);Wherein, symbol represents dot product operator.
According to source images B structure tensor image, source images B deflection image OBPass through formulaTo realize;Wherein, Arctan represents the arctan function in mathematics, for calculating angle.
In embodiments of the present invention, the similitude adaptation function of two image block distances is located at place according to pixel coordinate value The position of image determines, specific as follows:
In same image, such as p is designated as any pixel coordinate value in target image A/Image block PA(p/) and target Any pixel coordinate value is designated as q in image A/Image block PA(q/The distance between), the similitude adaptation function of use calculates public affairs Formula is
In formula, R (q'+ (i, j), OA(q')-OA(p') image block P) is representedA(q/) in pixel coordinate value q'+ (i, j) around Image block center pixel coordinate value q/Rotated counterclockwise by angle OA(q')-OA(p') pixel coordinate value obtained after.
Similarly, in different images, such as it is designated as p for any pixel coordinate value in target image A/Image block PA(p/) Q is designated as with any pixel coordinate value in source images B/Image block PB(q/The distance between), the similitude adaptation function meter of use Calculating formula is
In formula, R (q'+ (i, j), OB(q')-OA(p') image block P in source images B) is representedB(q/) in pixel coordinate value Q'+ (i, j) is around image block center pixel coordinate value q/Rotated counterclockwise by angle OB(q')-OA(p') pixel coordinate obtained after Value.
It is understood that the similitude adaptation function of two image blocks is phase for different images or same image With, the difference that simply parameter value uses in function, as deflection image is same O in same imageAOr OB, and it is right For different images, deflection image uses two OAAnd OB
S2, determine that each pixel coordinate value corresponds to the K transmitted in the target image respectively in the target image Individual most like image block, and utilize each pixel coordinate value in the similitude adaptation function calculating target image and its Similitude adaptation function value between the corresponding K being delivered in the target image most like image block, obtains the target figure Corresponding be delivered in the target image with similitude adaptation function value is minimum to each pixel coordinate value respectively as in Preceding K most like image blocks;K is more than 1 positive integer;
Detailed process is to search K corresponding to each pixel coordinate value most like images in target image A first Block, preprocessing process is as shown in Fig. 2 specifically include:
Step S21, obtain any one pixel coordinate value in the target image and its pass through in the target image Parallel K the first random pixel coordinate values of random number generation function generation, obtain each pixel coordinate in the target image Value is corresponding to be delivered in the K most like image blocks formed in the target image by K the first random pixel coordinate values;
Specifically, it is designated as p for each pixel coordinate value/Image block PA(p/), utilize CUDA (Compute Unified Device Architecture, unified calculation equipment framework) in the parallel generating random number letter that provides of cuRAND storehouses K the first random pixel coordinate values of number generation.Now, formed by K the first random pixel coordinate values each in target image A Initial K most like image blocks of individual pixel coordinate value.
Step S22, first iterations is set;If the first iterations is n=20;
Step S23, first spatial offset and the first random search region length of side are set;
Specifically, utilize the spatial method and figure that are formed by the coherent property of texture information between spatial neighborhood pixels As block stochastic search methods are respectively come the pixel coordinate value corresponding to updating K most like image blocks, to obtain accuracy more K high most like image blocks.In one embodiment, the first spatial offset is J=8, the first random search region length of side For L, and L=2 × max (w1,h1)+1。
Step S24, each pixel coordinate value in current first spatial offset and the target image is obtained to distinguish Corresponding current K most like image blocks;
Step S25, judge whether current first spatial offset is more than or equal to 1;If it is, perform next step S26;If it is not, then jump to step S27;
Step S26, current first spatial offset got according to, by each pixel in the target image Corresponding current K most like image blocks are updated coordinate value respectively, and calculate institute using the similitude adaptation function The pixel stated in the target image that each pixel coordinate value is each got correspondingly with it respectively in target image is sat Similitude adaptation function value between scale value and current K most like image blocks, and further retain every in the target image One pixel coordinate value calculates preceding K most like image blocks of the similitude adaptation function value for minimum as each self-corresponding Current K most like image block, and by behind current first spatial offset divided by 2, return to step S24;
Specifically, the pixel coordinate value calculation formula for choosing image block is:{NNF2(A(p/)),NNF2(A(p/+(J, 0)))-(J,0),NNF2(A(p/-(J,0)))+(J,0),NNF2(A(p/+(0,J)))-(0,J),NNF2(A(p/-(0,J)))+ (0,J);Wherein, NNF2(A(p/)) it is that any pixel coordinate value is designated as p/Image block PA(p/) K in target image A most The pixel coordinate value of similar image block;
It is and right respectively by each pixel coordinate value in selected pixel coordinate value calculation formula renewal target image The current K most like image blocks answered, and utilize similitude adaptation function DAACalculate each pixel coordinate in target image A Pixel coordinate value and current K a most like image block in the target image that value is each got correspondingly with it respectively Between similitude adaptation function value, and further retain each pixel coordinate value in target image A and calculate similitude With the preceding K most like image blocks that functional value is minimum as each self-corresponding current K most like image blocks, and by currently After first spatial offset J divided by 2, return to step S24.
Step S27, each pixel coordinate in the current first random search region length of side and the target image is obtained Current K most like image blocks corresponding to value difference;
Step S28, continue to judge whether the current first random search region length of side is more than or equal to 1;If it is, hold Row next step S29;If it is not, then jump to step S210;
Step S29, current K most like image blocks of each pixel coordinate value are respectively with respective in described target image Centered on each corresponding in the range of the current first random search region length of side randomly select a pixel coordinate value and carry out two Secondary renewal, and using the similitude adaptation function calculate in the target image each pixel coordinate value respectively each with The similitude between pixel coordinate value and current K a most like image block in its target image got correspondingly With functional value, and further retain after secondary renewal that each pixel coordinate value calculates similitude in the target image With the preceding K most like image blocks that functional value is minimum as each self-corresponding current K most like image blocks, and by described in After the current first random search region length of side divided by 2, return to step S27;
Specifically, it is designated as p for each pixel coordinate value/Image block PA(p/), the pixel for randomly selecting image block is sat The formula of scale value is:{NNF2(A(p/)),NNF2(A(p/))+(rand () %L-L/2, rand () %L-L/2) }, in formula, Rand () represents random number generation function, and symbol % represents that the division of mathematics takes the remainder operator;
And each pixel is sat in the secondary renewal target image of formula of the pixel coordinate value by randomly selecting image block Current K most like image blocks corresponding to scale value difference, and utilize similitude adaptation function DAACalculate each in target image A Pixel coordinate value and current K in the target image that individual pixel coordinate value is each got correspondingly with it respectively is individual most Similitude adaptation function value between similar image block, and further retain after secondary renewal that each pixel is sat in target image A It is individual most as each self-corresponding current K for preceding K minimum most like image blocks that scale value calculates similitude adaptation function value Similar image block, and by after current first random search region length of side L divided by 2, return to step S27.
Step S210, judge whether first iterations is equal to 0;If it is, perform next step S211;If It is no, after first iterations is subtracted into 1, return to step S23;
Step S211, terminate to calculate, and each pixel coordinate value in the target image is each finally obtained respectively Similitude adaptation function value is preceding K minimum most like image blocks as each pixel coordinate value pair in the target image The preceding K most like image block outputs that should be delivered in the target image.
Step S3, each pixel coordinate value in the obtained target image is corresponded to respectively and is delivered in the target figure There is similitude adaptation function value to be transmitted again for preceding K minimum most like image blocks as in, obtain in the target image Each pixel coordinate value, which is correspondingly delivered in the source images, has K*K most like image blocks, and utilizes the similitude It is individual most that adaptation function calculates the corresponding K*K being delivered in the source images of each pixel coordinate value in the target image Similitude adaptation function value between similar image block, filters out in the target image and is each counted in each pixel coordinate value Similitude adaptation function value is calculated as preceding K minimum most like image blocks while is exported.
Detailed process is that the preceding K that target image A self transmit to each pixel coordinate value obtained is individual most like Image block is further transferred in source images B and matched, and matching process is as shown in figure 3, specifically include:
Step S31, any one pixel coordinate value is obtained in the target image and its in the source images by simultaneously K the second random pixel coordinate values of row random number generation function generation, obtain each pixel coordinate value in the target image It is corresponding to be delivered in the K most like image blocks formed in the source images by K the second random pixel coordinate values;
Step S32, secondary iteration number is set;If secondary iteration number is n=20;
Step S33, second space offset and the second random search region length of side are set;
Specifically, in embodiment, second space offset is J=8, the second a length of L of random search regional edge, and L=2 ×max(w2,h2)+1。
Step S34, each pixel coordinate value in current second space offset and the target image is obtained to distinguish The corresponding current K in the source images most like image blocks;
Step S35, judge whether the current second space offset is more than or equal to 1;If it is, perform next step S36;If it is not, then jump to step S37;
Step S36, the current second space offset got according to, by each pixel in the target image The current K most like image blocks that coordinate value corresponds in the source images respectively are updated, and utilize the similitude The source figure that each pixel coordinate value in the target image each gets correspondingly with it respectively is calculated with function The similitude adaptation function value between pixel coordinate value and current K a most like image block as in, and further retain by institute State each pixel coordinate value in target image and calculate similitude adaptation function value as preceding K minimum most like image blocks As each self-corresponding current K most like image blocks, and by after the current second space offset divided by 2, return to step Rapid S34;
Specifically, the pixel coordinate value calculation formula for choosing image block is:{NNF1(A(p/)),NNF1(A(p/+(J, 0)))-(J,0),NNF1(A(p/-(J,0)))+(J,0),NNF1(A(p/+(0,J)))-(0,J),NNF1(A(p/-(0,J)))+ (0,J);Wherein, NNF1(A(p/)) it is that any pixel coordinate value is designated as p/Image block PA(p/) K in source images image B The pixel coordinate value of most like image block.
It is and right respectively by each pixel coordinate value in selected pixel coordinate value calculation formula renewal target image The current K most like image blocks answered, and utilize similitude adaptation function DABCalculate each pixel coordinate in target image A Between pixel coordinate value and current K a most like image block in the source images that value is each got correspondingly with it respectively Similitude adaptation function value, and further retain each pixel coordinate value in target image A and calculate similitude and match Functional value is preceding K minimum most like image blocks as each self-corresponding current K most like image blocks, and by current the After one spatial offset J divided by 2, return to step S34.
Step S37, each pixel coordinate in the current second random search region length of side and the target image is obtained Value corresponds to the current K most like image blocks in the source images respectively;
Step S38, continue to judge whether the current second random search region length of side is more than or equal to 1;If it is, hold Row next step S39;If it is not, then jump to step S310;
Step S39, each pixel coordinate value corresponds to current K in the source images respectively in described target image The each corresponding random choosing in the range of the current second random search region length of side centered on each respectively of most like image block Take pixel coordinate value to carry out secondary renewal, and calculated using the similitude adaptation function each in the target image Pixel coordinate value and current K a most like figure in the source images that individual pixel coordinate value is got correspondingly with it respectively As the similitude adaptation function value between block, and further retain by each pixel coordinate in the target image after secondary renewal Value calculates preceding K most like image blocks of the similitude adaptation function value for minimum as each self-corresponding current K most phases Like image block, and by behind the current second random search region length of side divided by 2, return to step S37;
Specifically, it is designated as p for each pixel coordinate value/Image block PA(p/), the pixel for randomly selecting image block is sat The formula of scale value is:{NNF1(A(p/)),NNF1(A(p/))+(rand () %L-L/2, rand () %L-L/2) }, in formula, Rand () represents random number generation function, and symbol % represents that the division of mathematics takes the remainder operator;
And each pixel is sat in the secondary renewal target image of formula of the pixel coordinate value by randomly selecting image block Current K most like image blocks corresponding to scale value difference, and utilize similitude adaptation function DABCalculate each in target image A Pixel coordinate value and current K a most phase in the source images that individual pixel coordinate value is each got correspondingly with it respectively Like the similitude adaptation function value between image block, and further retain after secondary renewal each pixel coordinate in target image A Value calculates preceding K most like image blocks of the similitude adaptation function value for minimum as each self-corresponding current K most phases Like image block, and by after current first random search region length of side L divided by 2, return to step S37.
Step S310, each pixel coordinate value is obtained in the target image in the target image according to step S2 K most like image blocks, each pixel coordinate value in the obtained target image is corresponded to respectively and is delivered in the mesh There is similitude adaptation function value to be transmitted again for preceding K minimum most like image blocks in logo image, obtain the target figure Each pixel coordinate value is correspondingly delivered in the source images as in, and there are K*K most like image blocks to be updated three times, And calculate each pixel coordinate value in the target image using the similitude adaptation function and corresponded respectively with it The source images got in pixel coordinate value and the similitude adaptation function value between current K most like image blocks, and Further retain and similitude adaptation function value is calculated by each pixel coordinate value in the target image after updating three times It is preceding K minimum most like image blocks as current K most like image blocks;
Specifically, it is theoretical using the relation transmission in mathematics, that is, utilize each pixel coordinate value in target image A (correspondence image block x) has K most like image block x1,x2,…,xK, and the K most like image blocks also have in source images B The current K of oneself most like image blocks, it is assumed that be x11,x12,…,x1K,x21,x22,…,x2K,…,xK1,xK2,…,xKK, and Utilize similitude adaptation function DABEach pixel coordinate value in target image A is calculated each to obtain correspondingly with it respectively The similitude adaptation function value between pixel coordinate value and current K a most like image block in the source images got, and enter one Before each pixel coordinate value calculates similitude adaptation function value as minimum in target image A after step reservation updates three times K most like image blocks are as each self-corresponding current K most like image blocks;
Step S311, judge whether the secondary iteration number is equal to 0;If it is, perform next step S312;If It is no, after the secondary iteration number is subtracted into 1, return to step S33;
Step S312, terminate to calculate, and institute is calculated by each last in each pixel coordinate value in the target image Similitude adaptation function value is obtained to export simultaneously for preceding K minimum most like image blocks.
As shown in figure 4, for a kind of GPU for being provided in the present invention accelerate based on direction align with match transmission it is approximate most Another embodiment of similar image block matching method, method passes through mesh again after laying particular emphasis on first Pretreatment of Source image in the embodiment The flow that logo image is matched with pretreated source images, is specifically included:
Step S5, target image and source images are determined, and the target image and the source images are based respectively on structure Extensometer calculates respective deflection image, and further according to the target image and the source images and each calculating The deflection image of gained, define the similitude adaptation function for calculating two image block distances;
Detailed process is that step S5 is each corresponding using identical method calculating target image A and source images B with step S1 Deflection image, therefore can be found in step S1 for step S5 concrete processing procedure, this is no longer going to repeat them.
It should be noted that in same source images B, any pixel coordinate value is designated as p in source images B/Image block PB (p/) with any pixel coordinate value in source images B it is designated as q/Image block PB(q/The distance between), the similitude matching letter of use Number DBBCalculation formula is
In formula, R (q'+ (i, j), OB(q')-OB(p') image block P in source images B) is representedB(q/) in pixel coordinate value Q'+ (i, j) is around image block center pixel coordinate value q/Rotated counterclockwise by angle OB(q')-OB(p') pixel coordinate obtained after Value.
As can be seen here, the similitude adaptation function of two image blocks is identical for different images or same image , the difference that simply parameter value uses in function, as deflection image is same O in same imageAOr OB, and for For different images, deflection image uses two OAAnd OB
Step S6, determine that each pixel coordinate value corresponds to the K transmitted in the source images respectively in the source images Individual most like image block, and it is right with it using each pixel coordinate value in the similitude adaptation function calculating source images The similitude adaptation function value that should be delivered between K most like image blocks in the source images, obtain every in the source images One pixel coordinate value, which corresponds to be delivered in respectively, has similitude adaptation function value for preceding K minimum most phases in the source images Like image block;K is more than 1 positive integer;
Detailed process is to search K corresponding to each pixel coordinate value most like image blocks in source images B first, Preprocessing process is as shown in figure 5, specifically include:
Step S61, any one pixel coordinate value is obtained in the source images and its in the source images by parallel K the 3rd random pixel coordinate values of random number generation function generation, obtain each pixel coordinate value in the source images and distinguish It is corresponding to be delivered in the K most like image blocks formed in the source images by K the 3rd random pixel coordinate values;
Step S62,3rd iterations is set;
Step S63,3rd spatial offset and the 3rd random search region length of side are set;
Step S64, it is right respectively to obtain each pixel coordinate value in current 3rd spatial offset and the source images The current K most like image blocks answered;
Step S65, judge whether current 3rd spatial offset is more than or equal to 1;If it is, perform next step S66;If it is not, then jump to step S67;
Step S66, current 3rd spatial offset got according to, each pixel in the source images is sat Corresponding current K most like image blocks are updated scale value respectively, and are calculated using the similitude adaptation function described Pixel coordinate value in the source images that each pixel coordinate value is each got correspondingly with it respectively in source images with And the similitude adaptation function value between current K most like image blocks, and further retain each pixel in the source images It is individual as each self-corresponding current K for preceding K minimum most like image blocks that coordinate value calculates similitude adaptation function value Most like image block, and by behind current 3rd spatial offset divided by 2, return to step S64;
Step S67, each pixel coordinate value in the current 3rd random search region length of side and the source images is obtained Current K most like image blocks corresponding to respectively;
Step S68, continue to judge whether the current 3rd random search region length of side is more than or equal to 1;If it is, hold Row next step S69;If it is not, then jump to step S610;
Step S69, current K most like image blocks of each pixel coordinate value are respectively with respectively in described source images It is secondary that center each correspondingly randomly selects a pixel coordinate value progress in the range of the length of side of current 3rd random search region Renewal, and using the similitude adaptation function calculate in the source images each pixel coordinate value respectively with a pair of one The similitude adaptation function value between pixel coordinate value and current K a most like image block in the source images got answered, And further retain after secondary renewal each pixel coordinate value in the source images and calculate similitude adaptation function value and be Preceding K minimum most like image blocks as each self-corresponding current K most like image blocks, and by the described current 3rd with After the machine region of search length of side divided by 2, return to step S67;
Step S610, judge whether the 3rd iterations is equal to 0;If it is, perform next step S611;If It is no, after the 3rd iterations is subtracted into 1, return to step S63;
Step S611, terminate to calculate, and each pixel coordinate value in the source images is each finally obtained into phase respectively It is each right as each pixel coordinate value in the source images for preceding K minimum most like image blocks like property adaptation function value The preceding K most like image block outputs that should be delivered in the source images.
It should be noted that step S6 and step S2 uses identical spatial method and image block stochastic search methods Respectively to update the pixel coordinate value corresponding to most like image block, therefore step is can be found in for step S6 concrete processing procedure Rapid S2, this is no longer going to repeat them.
Step S7, determine that the K that each pixel coordinate value is correspondingly delivered in the source images in the target image is individual Most like image block, and corresponded to respectively in the source images with similar according to each pixel coordinate value in the source images Property adaptation function value be preceding K minimum most like image blocks, it is corresponding to obtain each pixel coordinate value in the target image Being delivered in the source images has K*K most like image blocks, and further calculates institute using the similitude adaptation function State between each pixel coordinate value in target image corresponding K*K being delivered in the source images most like image blocks Similitude adaptation function value, filter out each calculated similitude matching in each pixel coordinate value in the target image Functional value exports simultaneously for preceding K minimum most like image blocks.
Detailed process is that each pixel coordinate value, which is transferred in source images B, in target image A K most like images Block, and extend to K*K most like image blocks after the pre-treatment in the source images B that corresponding K most like image blocks can be so that The K*K most like image blocks that each pixel coordinate value can be in corresponding source images B in target image A are matched, With process as shown in fig. 6, specifically including:
Step S71, obtain in the target image any one pixel coordinate value in the source images by parallel with K the 4th random pixel coordinate values of machine number generating function generation, it is corresponding to obtain each pixel coordinate value in the target image It is delivered in the K most like image blocks formed in the source images by K the 4th random pixel coordinate values;
Step S72,4th iterations is set;
Step S73,4th spatial offset and the 4th random search region length of side are set;
Step S74, each pixel coordinate value in current 4th spatial offset and the target image is obtained to distinguish The corresponding current K in the source images most like image blocks;
Step S75, judge whether current 4th spatial offset is more than or equal to 1;If it is, perform next step S76;If it is not, then jump to step S77;
Step S76, current 4th spatial offset got according to, by each pixel in the target image The current K most like image blocks that coordinate value corresponds in the source images respectively are updated, and utilize the similitude The source figure that each pixel coordinate value in the target image each gets correspondingly with it respectively is calculated with function The similitude adaptation function value between pixel coordinate value and current K most like image blocks as in, and further retain by Each pixel coordinate value calculates similitude adaptation function value as preceding K minimum most like images in the target image Block is as each self-corresponding current K most like image blocks, and by behind current 4th spatial offset divided by 2, returns Step S74;
Step S77, each pixel coordinate in the current 4th random search region length of side and the target image is obtained Value corresponds to the current K most like image blocks in the source images respectively;
Step S78, continue to judge whether the current 4th random search region length of side is more than or equal to 1;If it is, hold Row next step S79;If it is not, then jump to step S710;
Step S79, each pixel coordinate value corresponds to current K in the source images respectively in described target image The each corresponding random choosing in the range of the length of side of current 4th random search region centered on each respectively of most like image block Take pixel coordinate value to carry out secondary renewal, and calculated using the similitude adaptation function each in the target image Pixel coordinate value and current K a most like figure in the source images that individual pixel coordinate value is got correspondingly with it respectively As the similitude adaptation function value between block, and further retain by each pixel coordinate in the target image after secondary renewal Value calculates preceding K most like image blocks of the similitude adaptation function value for minimum as each self-corresponding current K most phases Like image block, and by behind the current 4th random search region length of side divided by 2, return to step S77;
Step S710, K of each pixel coordinate value in the source images in the source images is obtained according to step S6 Individual most like image block, each pixel coordinate value in the obtained target image is corresponded to respectively and is delivered in the source images In there is similitude adaptation function value to be transmitted again for current K minimum most like image blocks, obtain in the target image Each pixel coordinate value is correspondingly delivered in the source images, and there are K*K most like image blocks to be updated three times, and profit Each pixel coordinate value in the target image is calculated with the similitude adaptation function to obtain correspondingly with it respectively The similitude adaptation function value between pixel coordinate value and current K a most like image block in the source images got, and enter one Step retains calculates similitude adaptation function value as most by each pixel coordinate value in the target image after updating three times Preceding K small most like image blocks are as current K most like image blocks;
Step S711, judge whether the 4th iterations is equal to 0;If it is, perform next step S712;If It is no, then after first iterations being subtracted into 1, return to step S73;
Step S712, terminate to calculate, and institute is calculated by each last in each pixel coordinate value in the target image Similitude adaptation function value is obtained to export simultaneously for preceding K minimum most like image blocks.
It should be noted that because step S7 and step S3 use identical spatial method and image block random search Method updates the pixel coordinate value corresponding to most like image block respectively, therefore can join for step S7 concrete processing procedure See step S3, this is no longer going to repeat them.
Implement the embodiment of the present invention, have the advantages that:
The present invention is using most like image block as images match base unit, with the parallel PatchMatch images of GPU Based on method of completing the square flow, integrated structure tensor computation obtains matching the field of direction figure of image, and alignment image block deflection makes Matching possesses directionality, is effectively improved matching accuracy, and is directed to target image and the asynchronous two kinds of situations of source images, The method that corresponding enlarged proximal neighbor search scope is proposed using relation transmission property, it is effectively improved matching efficiency and accurate Degree, realize each step can the parallel processing in GPU, make based on direction align with match transmission most like image block Matching process is more efficient.
Can be with one of ordinary skill in the art will appreciate that realizing that all or part of step in above-described embodiment method is The hardware of correlation is instructed to complete by program, described program can be stored in a computer read/write memory medium, Described storage medium, such as ROM/RAM, disk, CD.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all essences in the present invention All any modification, equivalent and improvement made within refreshing and principle etc., should be included in the scope of the protection.

Claims (10)

1. the approximate most like image block matching method transmitted based on direction alignment with matching that a kind of GPU accelerates, its feature are existed In methods described includes:
S1, target image and source images are determined, and the target image and the source images are based respectively on structure tensor and calculated Go out respective deflection image, and the further side according to obtained by the target image and the source images and each calculating To angle image, the similitude adaptation function for calculating two image block distances is defined;
S2, determine that each pixel coordinate value corresponds to K transmitted in the target image most respectively in the target image Similar image block, and it is corresponding using each pixel coordinate value in the similitude adaptation function calculating target image The similitude adaptation function value being delivered between K most like image blocks in the target image, is obtained in the target image Each pixel coordinate value, which corresponds to be delivered in respectively, has similitude adaptation function value for minimum preceding K in the target image Most like image block;K is more than 1 positive integer;
S3, by each pixel coordinate value in the obtained target image, corresponding be delivered in the target image has respectively Similitude adaptation function value is transmitted again for preceding K minimum most like image blocks, obtains each picture in the target image Plain coordinate value, which is correspondingly delivered in the source images, has K*K most like image blocks, and utilizes the similitude adaptation function Calculate the corresponding K*K being delivered in the source images most like images of each pixel coordinate value in the target image Similitude adaptation function value between block, filter out each calculated similar in each pixel coordinate value in the target image Property adaptation function value exported simultaneously for preceding K minimum most like image blocks.
2. the method as described in claim 1, it is characterised in that " by the target image and the source figure in the step S1 Respective deflection image is calculated as being based respectively on structure tensor " specifically include:
3X3 convolution weight template is given, and utilizes target figure described in the convolution weight formwork calculation of the given 3X3 The gradient field image of the horizontal direction of picture and the gradient field image of vertical direction, and calculate the horizontal direction of the source images The gradient field image of gradient field image and vertical direction;
According to the gradient field image and the source calculated of the horizontal direction of the target image calculated and vertical direction The gradient field image of the horizontal direction of image and the gradient field image of vertical direction, the target image and described is calculated respectively Each self-corresponding structure tensor image of source images, and the target image and source images that are further calculated according to each correspond to Structure tensor image, obtain the deflection image of the target image and the source images.
3. method as claimed in claim 2, it is characterised in that the deflection image of the target image is to pass through formulaTo realize, the side of the source images It is to pass through formula to angle image To realize;Wherein,
The structure tensor graphical representation of the target image is SA=(SAx,SAy,SAz);SAx(p)=Ax (p)·Ax(p), SAy(p)=Ax(p)·Ay(p), SAz(p)=Ay(p)·Ay(p);P is the mesh Any one pixel coordinate value of logo image;I, j is integer,
The structure tensor graphical representation of the source images is SB=(SBx,SBy,SBz);SBx(q)=Bx(q)·Bx (q), SBy(q)=Bx(q)·By(q), SBz(q)=By(q)·By(q);Q is the source figure Any one pixel coordinate value of picture.
4. the method as described in claim 1, it is characterised in that the step S2 is specifically included:
S21, obtain in the target image any one pixel coordinate value and its in the target image by parallel random Number generating function K the first random pixel coordinate values of generation, obtain the corresponding biography of each pixel coordinate value in the target image Pass the K most like image blocks formed in the target image by K the first random pixel coordinate values;
S22, the first iterations is set;
S23, the first spatial offset and the first random search region length of side are set;
S24, obtain in current first spatial offset and the target image and work as corresponding to each pixel coordinate value difference Preceding K most like image blocks;
S25, judge whether current first spatial offset is more than or equal to 1;If it is, perform next step S26;If It is no, then jump to step S27;
S26, current first spatial offset got according to, by each pixel coordinate value in the target image point Not corresponding current K most like image blocks are updated, and calculate the target figure using the similitude adaptation function Pixel coordinate value as in the target image that is each got correspondingly with it respectively of each pixel coordinate value and Similitude adaptation function value between current K most like image blocks, and further retain each pixel in the target image It is individual as each self-corresponding current K for preceding K minimum most like image blocks that coordinate value calculates similitude adaptation function value Most like image block, and by behind current first spatial offset divided by 2, return to step S24;
Each pixel coordinate value is right respectively in S27, the current first random search region length of side of acquisition and the target image The current K most like image blocks answered;
S28, continue to judge whether the current first random search region length of side is more than or equal to 1;If it is, perform next step Rapid S29;If it is not, then jump to step S210;
In S29, the target image the current K most like image blocks of each pixel coordinate value respectively centered on each Each corresponded in the range of the current first random search region length of side and randomly select a secondary renewal of pixel coordinate value progress, and Using the similitude adaptation function calculate in the target image each pixel coordinate value respectively each with a pair of one The similitude adaptation function between pixel coordinate value and current K a most like image block in the target image got answered Value, and further retain after secondary renewal that each pixel coordinate value calculates similitude adaptation function in the target image Be worth be preceding K minimum most like image blocks as each self-corresponding current K most like image blocks, and by described current the After the one random search region length of side divided by 2, return to step S27;
S210, judge whether first iterations is equal to 0;If it is, perform next step S211;If it is not, then will After first iterations subtracts 1, return to step S23;
S211, terminate to calculate, and each pixel coordinate value in the target image is each finally obtained into similitude respectively Correspondingly it is delivered in as each pixel coordinate value in the target image for preceding K minimum most like image blocks with functional value Preceding K most like image block outputs in the target image.
5. the method as described in claim 1, it is characterised in that the step S3 is specifically included:
Any one pixel coordinate value passes through parallel generating random number in the source images in S31, the acquisition target image K the second random pixel coordinate values of function generation, obtain each pixel coordinate value in the target image and are correspondingly delivered in institute State the K most like image blocks formed in source images by K the second random pixel coordinate values;
S32, secondary iteration number is set;
S33, second space offset and the second random search region length of side are set;
Each pixel coordinate value corresponds in institute respectively in S34, the current second space offset of acquisition and the target image State the current K*K most like image blocks formed in source images;
S35, judge whether the current second space offset is more than or equal to 1;If it is, perform next step S36;If It is no, then jump to step S37;
S36, the current second space offset got according to, by each pixel coordinate value in the target image point The current K in the source images most like image blocks are not corresponded to be updated, and utilize the similitude adaptation function meter Calculate the picture in the source images that each pixel coordinate value is each got correspondingly with it respectively in the target image Similitude adaptation function value between plain coordinate value and current K most like image blocks, and further retain by the target figure Each pixel coordinate value calculates preceding K most like image blocks of the similitude adaptation function value for minimum as respective as in Corresponding current K most like image blocks, and will be after the currently second space offset divided by 2, return to step S33;
Each pixel coordinate value is right respectively in S37, the current second random search region length of side of acquisition and the target image The current K*K most like image blocks that should be formed in the source images;
S38, continue to judge whether the current second random search region length of side is more than or equal to 1;If it is, perform next step Rapid S39;If it is not, then jump to step S310;
Each pixel coordinate value corresponds to the current K most like figures in the source images respectively in S39, the target image A picture is randomly selected as block is each corresponding in the range of the current second random search region length of side centered on each respectively Plain coordinate value carries out secondary renewal, and calculates each pixel in the target image using the similitude adaptation function and sit Between pixel coordinate value and current K a most like image block in the source images that scale value is got correspondingly with it respectively Similitude adaptation function value, and further retain and calculated by each pixel coordinate value in the target image after secondary renewal It is preceding K minimum most like image blocks as each self-corresponding current K most like image blocks to go out similitude adaptation function value, And by behind the current second random search region length of side divided by 2, return to step S37;
Step S310, K of each pixel coordinate value in the target image in the target image is obtained according to step S2 Individual most like image block, each pixel coordinate value in the obtained target image is corresponded to respectively and is delivered in the target figure There is similitude adaptation function value to be transmitted again for preceding K minimum most like image blocks as in, obtain in the target image Each pixel coordinate value is correspondingly delivered in the source images, and there are K*K most like image blocks to be updated three times, and profit Each pixel coordinate value in the target image is calculated with the similitude adaptation function to obtain correspondingly with it respectively The similitude adaptation function value between pixel coordinate value and current K a most like image block in the source images got, and enter one Step retains calculates similitude adaptation function value as most by each pixel coordinate value in the target image after updating three times Preceding K small most like image blocks are as current K most like image blocks;
S311, judge whether the secondary iteration number is equal to 0;If it is, perform next step S312;If it is not, then will After first iterations subtracts 1, return to step S33;
S312, terminate to calculate, and gained similitude is calculated by each last in each pixel coordinate value in the target image Adaptation function value exports simultaneously for preceding K minimum most like image blocks.
6. the approximate most like image block matching method transmitted based on direction alignment with matching that a kind of GPU accelerates, its feature are existed In methods described includes:
S5, target image and source images are determined, and the target image and the source images are based respectively on structure tensor and calculated Go out respective deflection image, and the further side according to obtained by the target image and the source images and each calculating To angle image, the similitude adaptation function for calculating two image block distances is defined;
S6, determine that it is most like to correspond to K transmitted in the source images respectively for each pixel coordinate value in the source images Image block, and be delivered in using each pixel coordinate value is corresponding in the similitude adaptation function calculating source images The similitude adaptation function value between K most like image blocks in the source images, obtains each pixel in the source images Coordinate value, which corresponds to be delivered in respectively, has similitude adaptation function value for preceding K minimum most like images in the source images Block;K is more than 1 positive integer;
S7, determine that each pixel coordinate value is correspondingly delivered in K most like figures in the source images in the target image As block, and being corresponded to respectively according to each pixel coordinate value in the source images in the source images there is similitude to match letter Numerical value is preceding K minimum most like image blocks, obtains each pixel coordinate value in the target image and is correspondingly delivered in institute Stating in source images has K*K most like image blocks, according to the target image and source images and its deflection corresponding to them Image, definition calculate the similitude adaptation function of two image block distances, and further utilize the similitude adaptation function meter Calculate the corresponding K*K being delivered in the source images most like image blocks of each pixel coordinate value in the target image Between similitude adaptation function value, filter out in the target image in each pixel coordinate value each calculated similitude Adaptation function value exports simultaneously for preceding K minimum most like image blocks.
7. method as claimed in claim 6, it is characterised in that " by the target image and the source figure in the step S5 Respective deflection image is calculated as being based respectively on structure tensor " specifically include:
3X3 convolution weight template is given, and utilizes target figure described in the convolution weight formwork calculation of the given 3X3 The gradient field image of the horizontal direction of picture and the gradient field image of vertical direction, and calculate the horizontal direction of the source images The gradient field image of gradient field image and vertical direction;
According to the gradient field image and the source calculated of the horizontal direction of the target image calculated and vertical direction The gradient field image of the horizontal direction of image and the gradient field image of vertical direction, the target image and described is calculated respectively Each self-corresponding structure tensor image of source images, and the target image and source images that are further calculated according to each correspond to Structure tensor image, obtain the target image and each self-corresponding deflection image of the source images.
8. method as claimed in claim 7, it is characterised in that the deflection image of the target image is to pass through formulaTo realize, the source images Deflection image be to pass through formula To realize;Wherein,
The structure tensor graphical representation of the target image is SA=(SAx,SAy,SAz);SAx(p)=Ax (p)·Ax(p), SAy(p)=Ax(p)·Ay(p), SAz(p)=Ay(p)·Ay(p);P is the target Any one pixel coordinate value of image;I, j is integer,
The structure tensor graphical representation of the source images is SB=(SBx,SBy,SBz);SBx(q)=Bx(q)· Bx(q), SBy(q)=Bx(q)·By(q), SBz(q)=By(q)·By(q);Q is the source figure Any one pixel coordinate value of picture.
9. method as claimed in claim 6, it is characterised in that the step S6 is specifically included:
S61, obtain any one pixel coordinate value in the source images and its given birth in the source images by parallel random number Into K the 3rd random pixel coordinate values of function generation, obtain each pixel coordinate value in the source images and correspond to transmission respectively The K most like image blocks formed in the source images by K the 3rd random pixel coordinate values;
S62, the 3rd iterations is set;
S63, the 3rd spatial offset and the 3rd random search region length of side are set;
S64, obtain current K corresponding to each pixel coordinate value difference in current 3rd spatial offset and the source images Individual most like image block;
S65, judge whether current 3rd spatial offset is more than or equal to 1;If it is, perform next step S66;If It is no, then jump to step S67;
S66, current 3rd spatial offset got according to, each pixel coordinate value in the source images is distinguished Corresponding current K most like image blocks are updated, and are calculated using the similitude adaptation function in the source images Pixel coordinate value and current K in the source images that each pixel coordinate value is each got correspondingly with it respectively Similitude adaptation function value between most like image block, and further retain each pixel coordinate value in the source images and counted It is preceding K minimum most like image blocks as each self-corresponding current K most like images to calculate similitude adaptation function value Block, and by behind current 3rd spatial offset divided by 2, return to step S64;
Each pixel coordinate value corresponds to respectively in S67, the current 3rd random search region length of side of acquisition and the source images Current K most like image blocks;
S68, continue to judge whether the current 3rd random search region length of side is more than or equal to 1;If it is, perform next step Rapid S69;If it is not, then jump to step S610;
Current K most like image blocks of each pixel coordinate value are being worked as centered on each respectively in S69, the source images It is each corresponding in the range of the length of side of preceding 3rd random search region to randomly select a secondary renewal of pixel coordinate value progress, and profit Each pixel coordinate value in the source images is calculated with the similitude adaptation function to obtain correspondingly with it respectively To source images in pixel coordinate value and the similitude adaptation function value between current K most like image blocks, and further Retain after secondary renewal that each pixel coordinate value calculates similitude adaptation function value as minimum preceding K in the source images Individual most like image block is as each self-corresponding current K most like image block, and by the current 3rd random search area After the domain length of side divided by 2, return to step S67;
S610, judge whether the 3rd iterations is equal to 0;If it is, perform next step S611;If not, by institute State after the 3rd iterations subtracts 1, return to step S63;
S611, terminate to calculate, and each pixel coordinate value in the source images is each finally obtained into similitude matching respectively Functional value is each corresponded to as each pixel coordinate value in the source images for preceding K minimum most like image blocks and is delivered in Preceding K most like image block outputs in the source images.
10. method as claimed in claim 6, it is characterised in that the step S7 is specifically included:
Any one pixel coordinate value passes through parallel generating random number in the source images in S71, the acquisition target image K the 4th random pixel coordinate values of function generation, obtain each pixel coordinate value in the target image and are correspondingly delivered in institute State the K most like image blocks formed in source images by K the 4th random pixel coordinate values;
S72, the 4th iterations is set;
S73, the 4th spatial offset and the 4th random search region length of side are set;
Each pixel coordinate value corresponds in institute respectively in S74, current 4th spatial offset of acquisition and the target image State the current K*K most like image blocks formed in source images;
S75, judge whether current 4th spatial offset is more than or equal to 1;If it is, perform next step S76;If It is no, then jump to step S77;
S76, current 4th spatial offset got according to, by each pixel coordinate value in the target image point The current K*K most like image blocks formed in the source images are not corresponded to be updated, and are matched using the similitude Function calculates the source images that each pixel coordinate value is each got correspondingly with it respectively in the target image In pixel coordinate value and the similitude adaptation function value between current K most like image blocks, and further retain by described Each pixel coordinate value calculates similitude adaptation function value and made for preceding K minimum most like image blocks in target image For each self-corresponding current K most like image block, and by behind current 4th spatial offset divided by 2, return to step S74;
Each pixel coordinate value is right respectively in S77, the current 4th random search region length of side of acquisition and the target image The current K*K most like image blocks that should be formed in the source images;
S78, continue to judge whether the current 4th random search region length of side is more than or equal to 1;If it is, perform next step Rapid S79;If it is not, then jump to step S710;
Each pixel coordinate value corresponds to the current K most like figures in the source images respectively in S79, the target image A picture is randomly selected as block is each corresponding in the range of the length of side of current 4th random search region centered on each respectively Plain coordinate value carries out secondary renewal, and calculates each pixel in the target image using the similitude adaptation function and sit Between pixel coordinate value and current K a most like image block in the source images that scale value is got correspondingly with it respectively Similitude adaptation function value, and further retain and calculated by each pixel coordinate value in the target image after secondary renewal It is preceding K minimum most like image blocks as each self-corresponding current K most like image blocks to go out similitude adaptation function value, And by behind the current 4th random search region length of side divided by 2, return to step S77;
Step S710, K of each pixel coordinate value in the source images are obtained in the source images most according to step S6 Similar image block, by each pixel coordinate value in the obtained target image, corresponding be delivered in the source images has respectively There is similitude adaptation function value to be transmitted again for current K minimum most like image blocks, obtain each in the target image Individual pixel coordinate value is correspondingly delivered in the source images, and there are K*K most like image blocks to be updated three times, and utilize institute State similitude adaptation function and calculate each pixel coordinate value in the target image and got correspondingly with it respectively Source images in pixel coordinate value and the similitude adaptation function value between current K most like image blocks, and further protect It is minimum to stay and calculate similitude adaptation function value by each pixel coordinate value in the target image after updating three times Preceding K most like image blocks are as current K most like image blocks;
S711, judge whether the 4th iterations is equal to 0;If it is, perform next step S712;If it is not, then will After first iterations subtracts 1, return to step S73;S712, terminate to calculate, and by each picture in the target image Each the last gained similitude adaptation function value that calculates exports simultaneously for preceding K minimum most like image blocks in plain coordinate value.
CN201710426618.8A 2017-06-08 2017-06-08 A kind of approximate most like image block matching method transmitted with matching based on direction alignment that GPU accelerates Active CN107369169B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710426618.8A CN107369169B (en) 2017-06-08 2017-06-08 A kind of approximate most like image block matching method transmitted with matching based on direction alignment that GPU accelerates

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710426618.8A CN107369169B (en) 2017-06-08 2017-06-08 A kind of approximate most like image block matching method transmitted with matching based on direction alignment that GPU accelerates

Publications (2)

Publication Number Publication Date
CN107369169A true CN107369169A (en) 2017-11-21
CN107369169B CN107369169B (en) 2018-09-04

Family

ID=60304977

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710426618.8A Active CN107369169B (en) 2017-06-08 2017-06-08 A kind of approximate most like image block matching method transmitted with matching based on direction alignment that GPU accelerates

Country Status (1)

Country Link
CN (1) CN107369169B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109493370A (en) * 2018-10-12 2019-03-19 西南交通大学 A kind of method for tracking target based on spatial offset study
CN109615591A (en) * 2018-11-27 2019-04-12 东莞信大融合创新研究院 A kind of three-dimensional Block- matching noise-reduction method accelerated parallel based on GPU
CN111819579A (en) * 2018-08-03 2020-10-23 谷歌有限责任公司 Distribution tensor calculation across computing devices
CN112861976A (en) * 2021-02-11 2021-05-28 温州大学 Sensitive image identification method based on twin graph convolution hash network

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103632362A (en) * 2012-08-28 2014-03-12 中国电信股份有限公司 Image matching processing method, device and system
CN106572354A (en) * 2015-10-09 2017-04-19 腾讯科技(北京)有限公司 Image block-based search matching method and system and video processing device

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103632362A (en) * 2012-08-28 2014-03-12 中国电信股份有限公司 Image matching processing method, device and system
CN106572354A (en) * 2015-10-09 2017-04-19 腾讯科技(北京)有限公司 Image block-based search matching method and system and video processing device

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
CONNELLY BARNES 等: "PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing", 《ACM TRANSACTIONS ON GRAPHICS》 *
CONNELLY BARNES 等: "The Generalized PatchMatch Correspondence Algorithm", 《ECCV"10 PROCEEDINGS OF THE 11TH EUROPEAN CONFERENCE ON COMPUTER VISION: PART III》 *
于沛: "图像处理中块匹配算法的GPU并行化研究", 《中国优秀硕士学位论文全文数据库》 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111819579A (en) * 2018-08-03 2020-10-23 谷歌有限责任公司 Distribution tensor calculation across computing devices
CN111819579B (en) * 2018-08-03 2022-02-08 谷歌有限责任公司 Method, system, and medium for distributed tensor computation across computing devices
CN109493370A (en) * 2018-10-12 2019-03-19 西南交通大学 A kind of method for tracking target based on spatial offset study
CN109493370B (en) * 2018-10-12 2021-07-02 西南交通大学 Target tracking method based on space offset learning
CN109615591A (en) * 2018-11-27 2019-04-12 东莞信大融合创新研究院 A kind of three-dimensional Block- matching noise-reduction method accelerated parallel based on GPU
CN112861976A (en) * 2021-02-11 2021-05-28 温州大学 Sensitive image identification method based on twin graph convolution hash network
CN112861976B (en) * 2021-02-11 2024-01-12 温州大学 Sensitive image identification method based on twin graph convolution hash network

Also Published As

Publication number Publication date
CN107369169B (en) 2018-09-04

Similar Documents

Publication Publication Date Title
CN107369169B (en) A kind of approximate most like image block matching method transmitted with matching based on direction alignment that GPU accelerates
Sharp et al. Multiview registration of 3D scenes by minimizing error between coordinate frames
Chow et al. Surface registration using a dynamic genetic algorithm
Jin et al. An efficient self-organizing map designed by genetic algorithms for the traveling salesman problem
CN110427968A (en) A kind of binocular solid matching process based on details enhancing
CN101599178A (en) Tracking point detecting device and method, program and recording medium
CN103971366B (en) A kind of solid matching method being polymerize based on double weights
CN107329962A (en) Image retrieval data library generating method, the method and device of augmented reality
CN106875443A (en) The whole pixel search method and device of the 3-dimensional digital speckle based on grayscale restraint
CN107230233A (en) The scaling method and device of telecentric lens 3-D imaging system based on bundle adjustment
CN101794459A (en) Seamless integration method of stereoscopic vision image and three-dimensional virtual object
CN103400393B (en) A kind of image matching method and system
CN109801325A (en) A kind of Binocular Stereo Vision System obtains the method and device of disparity map
Mahapatra et al. A multi-view video synopsis framework
CN106023317B (en) A kind of weighted Voronoi diagrams drawing generating method for big data test
CN106815871A (en) A kind of modeling method of SEM imaging system
CN109191579A (en) Method, apparatus, computer equipment and the storage medium of coordinate conversion
CN107977986A (en) The Forecasting Methodology and device of a kind of movement locus
CN106780602A (en) A kind of rifle ball positioning method and device
CN109993338B (en) Link prediction method and device
Tang et al. Fundamental matrix estimation by multiobjective genetic algorithm with Taguchi's method
CN116307328A (en) Greedy solving method for travel business problem
Laskowski Objects auto-selection from stereo-images realised by self-correcting neural network
CN108305282A (en) A kind of method for registering images and system based on hybrid rice algorithm
CN107341151A (en) Image retrieval data library generating method, the method and device of augmented reality

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant