CN103632362A - Image matching processing method, device and system - Google Patents

Abstract

The invention discloses an image block matching processing method, an image block matching processing device and an image block matching processing system. The method includes the following steps that: color feature vectors of a target image block are calculated and are adopted first color feature vectors; image blocks with the same size as a target image block are adopted as inquired image blocks, and more than one inquired image blocks exists, and color feature vectors of the more than one inquired image blocks are respectively calculated and are adopted as second color feature vectors; and one inquired image block is selected from the more than one inquired image blocks as an image block which is matched with the target image block according to the similarity metric value of the first color feature vectors and the second color feature vectors of the more than one inquired image block. With the image matching processing method adopted, the robustness of image matching can be provided, and at the same time, calculation quantity in an image matching process can be decreased; and the image matching processing method is easy to realize.

Description

Images match disposal route, Apparatus and system

Technical field

The present invention relates to multimedia signal processing field, particularly a kind of images match disposal route, Apparatus and system.

Background technology

Along with the development of multimedia signal processing and image matching technology, image matching technology is widely applied to the various fields such as image recognition, graphical analysis.The object of image matching technology is in image block set, to search the image block the most similar to the image block of appointment.

Conventional image matching method is divided into exact matching and robust mates two classes.Exact matching method is compared the pixel of two image blocks one by one, to compare the similarities and differences between two image blocks.Yet, in digital picture or frame of video, to encode, transmitting, in printing, scanning process, usually can introduce geometric distortion and interference noise, the accuracy of exact matching method can be affected, and causes mating accuracy and declines.For this problem, robust matching process, by extracting above-mentioned distortion from image block, disturbing the robust features with unchangeability, is found matching image piece according to robust features.Existing robust matching process is roughly divided into based on image Hash and based on point of interest detection (Interest Point Detection) two classes.

By the image information of higher-dimension, the eigenvector with lower dimension characterizes robust matching process based on image Hash, and common feature comprises: color histogram, PCA coefficient, DCT/ wavelet coefficient, add up not bending moment etc.The relative position that the method detecting based on point of interest occurs by the point of interest in comparison chart picture, thereby the similarity of movement images interblock.Yet, the dimension of the needed eigenvector of this two classes robust matching process is higher, calculated amount is large, computation complexity is high, not only need enough computational resources, and cannot be applied in the scene that requirement of real-time is higher and use, for example, cannot in Video processing, need by using in similar coupling estimated motion vector.

Summary of the invention

According to the embodiment of the present invention aspect, a technical matters to be solved is: a kind of images match disposal route, Apparatus and system are provided, when the robustness of image matching method is provided, reduce the computing time of images match process.

A kind of images match disposal route that the embodiment of the present invention provides, comprising:

Calculate the color characteristic vector of target image piece, as the first color characteristic vector;

Successively in being queried image, choose an image block with described target image block size as being queried image block, the described image block that is queried has more than one, calculates respectively a color characteristic vector that is queried above image block, as the second color characteristic vector;

Calculate described the first color characteristic vector and a similarity measurement value that is queried above the second color characteristic vector of image block;

From one, be queried above and image block, select one to be queried image block as the image block matching with described target image piece.

A kind of images match treating apparatus that the embodiment of the present invention provides, comprising:

The first color characteristic vector computing unit, for calculating the color characteristic vector of target image piece, as the first color characteristic vector;

The second color characteristic vector computing unit, successively in being queried image, choose an image block with described target image block size as being queried image block, the described image block that is queried has more than one, calculate respectively a color characteristic vector that is queried above image block, as the second color characteristic vector;

Similarity measurement value computing unit, for calculating described the first color characteristic vector and a similarity measurement value that is queried above the second color characteristic vector of image block;

Matching image selected cell, for according to described similarity measurement value, is queried above and image block, selects one to be queried image block as the image block matching with described target image piece from one.

A kind of image block matching treatment system that the embodiment of the present invention provides, comprising: central processor CPU, graphic process unit GPU;

Described CPU is for calculating the color characteristic vector of target image piece, as the first color characteristic vector; According to the similarity measurement value of described GPU output, from one, be queried above and image block, select one to be queried image block as the image block matching with described target image piece.

Described GPU is for being queried image successively, choose an image block with described target image block size as being queried image block, the described image block that is queried has more than one, calculates respectively a color characteristic vector that is queried above image block, as the second color characteristic vector; Calculate described the first color characteristic vector and a similarity measurement value that is queried above the second color characteristic vector of image block.

The image block matched processing method, the Apparatus and system that based on the above embodiment of the present invention, provide, calculate the color characteristic vector of target image piece as the first color characteristic vector, and in target image piece, choose there is target image block size image block as being queried image block, calculating is queried the color characteristic vector of image block, as the second color characteristic vector, according to the similarity measurement value of the first color characteristic vector and the second color characteristic vector, select the image block matching with target image piece.By adopting the color characteristic vector of low dimension, the robustness of image matching method is not only provided, and makes the computation process calculated amount of images match little, computation complexity is low, thereby has saved computational resource.Simultaneously, because color characteristic vector has lower dimension, be convenient to export with texture form, therefore, can utilize the parallel computation characteristic of GPU rendering pipeline section tinter to calculate, realize comprising that the view data of frame of video is carried out fast, processing in real time, thereby can be applied to Video coding, video denoising, image registration, image copy detections etc. need quick comparison chart as the application scenarios of block similarity.

By the detailed description to exemplary embodiment of the present invention referring to accompanying drawing, it is clear that further feature of the present invention and advantage thereof will become.

Accompanying drawing explanation

In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, to the accompanying drawing of required use in embodiment or description of the Prior Art be briefly described below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skills, do not paying under the prerequisite of creative work, can also obtain according to these accompanying drawings other accompanying drawing.

, it should be understood that for convenience of description, the size of the various piece shown in accompanying drawing is not to draw according to actual proportionate relationship meanwhile.In similar label and letter accompanying drawing below, represent similar terms, therefore, once be defined in an a certain Xiang Yi accompanying drawing, in accompanying drawing subsequently, do not need it to be further discussed.

The accompanying drawing that forms a part for instructions has been described embodiments of the invention, and together with the description for explaining principle of the present invention.

With reference to accompanying drawing, according to detailed description below, can more be expressly understood the present invention, wherein:

Fig. 1 illustrates the schematic flow sheet of a kind of embodiment of images match disposal route provided by the present invention;

Fig. 2 illustrates in a kind of embodiment of images match disposal route provided by the present invention, color component images corresponding to each Color Channel of image block is divided into the schematic diagram of two sub regions;

Fig. 3 illustrates in a kind of embodiment of images match disposal route provided by the present invention, the result schematic diagram of color characteristic vector to the robustness test of gaussian additive noise;

Fig. 4 illustrates in a kind of embodiment of images match disposal route provided by the present invention, the result schematic diagram of color characteristic vector to the robustness test of fuzzy operation;

Fig. 5 illustrates the schematic flow sheet of the another kind of embodiment of images match disposal route provided by the present invention;

Fig. 6 illustrates in a kind of embodiment of images match disposal route provided by the present invention, target image piece is divided into the schematic diagram of target image piece sub-block;

Fig. 7 illustrates the structural representation of a kind of embodiment of images match treating apparatus provided by the present invention;

Fig. 8 illustrates the structural representation of a kind of embodiment of images match disposal system provided by the present invention;

Fig. 9 illustrates the structural representation of the another kind of embodiment of images match disposal system provided by the present invention;

Figure 10 illustrates the position view of the target image piece that images match disposal route provided by the present invention chooses;

Figure 11 illustrates the corresponding image block position view that is queried of maximal value that images match disposal route provided by the present invention finds similarity measurement value.

Embodiment

Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is only the present invention's part embodiment, rather than whole embodiment.It should be noted that: unless illustrate in addition, the parts of setting forth in these embodiments and the positioned opposite of step do not limit the scope of the invention.

To the description only actually of at least one exemplary embodiment, be illustrative below, never as any restriction to the present invention and application or use.Embodiment based in the present invention, those of ordinary skills, not making the every other embodiment obtaining under creative work prerequisite, belong to the scope of protection of the invention.

For the known technology of person of ordinary skill in the relevant, method and apparatus, may not discuss in detail, but in suitable situation, described technology, method and apparatus should be regarded as authorizing a part for instructions.

In all examples with discussing shown here, it is exemplary that any occurrence should be construed as merely, rather than as restriction.Therefore, other example of exemplary embodiment can have different values.

image block matched processing method

Fig. 1 illustrates the schematic flow sheet of a kind of embodiment of images match disposal route provided by the present invention.Shown in Figure 1, the images match disposal route of this embodiment comprises following operation:

Step 101, the color characteristic vector of calculating target image piece, as the first color characteristic vector.

Step 102, in being queried image, choose an image block with target image block size as being queried image block successively, be queried image block and have more than one, calculate respectively a color characteristic vector that is queried above image block, as the second color characteristic vector.

Step 103, calculates the first color characteristic vector and a similarity measurement value that is queried above the second color characteristic vector of image block.

Step 104, is queried above and image block, selects one to be queried image block as the image block matching with target image piece from one.

It should be noted that, the operation of above-mentioned steps 101 can prior to or after in the operation of step 102, carry out, also can carry out simultaneously.In addition, in step 102, calculate difference and be queried the color characteristic vector of image block as the operation of the second color characteristic vector, can executed in parallel.In step 102, step 103, one of acquisition, be queried after the second color characteristic vector of image block, can start to perform step 103 fall into a trap calculate the first color characteristic vector and these and be queried the similarity measurement value of the second color characteristic vector of image block, and after not needing to wait for that all the second color characteristic vectors that is queried image block have calculated, then perform step 103 operation.

In the images match disposal route that above-described embodiment provides, by adopting the color characteristic vector of low dimension, the robustness of image matching method is not only provided, and makes the computation process calculated amount of images match little, computation complexity is low, thereby has saved computational resource.

Concrete example of images match disposal route embodiment according to the present invention and unrestricted, in operation 101,102 embodiment illustrated in fig. 1, target image piece be queried in image block, each image block comprises three Color Channels, calculating color characteristic vector can realize in the following way: color component images corresponding to each Color Channel of image block is divided into two sub regions, calculate respectively the color component value sum in every sub regions of each color component images, be expressed as R ₁, R ₂, G ₁, G ₂, B ₁, B ₂:

${\mathrm{<figure-callout\; id="1"\; label="R"\; filenames="HDA00002063078400031.png,HDA00002063078400061.png"\; state="\{\{state\}\}">R</figure-callout>}}_1=\underset{i=-n}{\overset{n}{\mathrm{\&Sigma;}}}\underset{j=i}{\overset{n}{\mathrm{\&Sigma;}}}{I}_r(i,j),$ ${\mathrm{<figure-callout\; id="1"\; label="G"\; filenames="HDA00002063078400031.png,HDA00002063078400061.png"\; state="\{\{state\}\}">G</figure-callout>}}_1=\underset{j=-n}{\overset{n}{\mathrm{\&Sigma;}}}\underset{i=j}{\overset{n}{\mathrm{\&Sigma;}}}{I}_g(i,j),$ ${\mathrm{<figure-callout\; id="1"\; label="B"\; filenames="HDA00002063078400031.png,HDA00002063078400061.png"\; state="\{\{state\}\}">B</figure-callout>}}_1=\underset{i=-n}{\overset{n}{\mathrm{\&Sigma;}}}\underset{j=i}{\overset{n}{\mathrm{\&Sigma;}}}{I}_b(i,j),$

$R_2=\underset{j=-n}{\overset{n}{\mathrm{\&Sigma;}}}\underset{i=j}{\overset{n}{\mathrm{\&Sigma;}}}{I}_r(i,j),$ $G_2=\underset{i=-n}{\overset{n}{\mathrm{\&Sigma;}}}\underset{j=i}{\overset{n}{\mathrm{\&Sigma;}}}{I}_g(i,j),$ $B_2=\underset{j=-n}{\overset{n}{\mathrm{\&Sigma;}}}\underset{i=j}{\overset{n}{\mathrm{\&Sigma;}}}{I}_b(i,j)$

With R ₁for example, wherein, i, j are respectively the position of pixel, and n is the i in first subregion of red color component image, the span of j, I _r(i, j) is the red color component value of the coordinate position pixel that is (i, j).R ₁for the red color component value sum of all pixels in first subregion of red color component image, R ₂red color component value sum for all pixels in the second sub regions of red color component image.

With G ₁for example, wherein, i, j are respectively the position of pixel, and n is the i in first subregion of green color component image, the span of j, I _g(i, j) is the green color component value of the coordinate position pixel that is (i, j).G ₁for the green color component value sum of all pixels in first subregion of green color component image, G ₂green color component value sum for all pixels in the second sub regions of green color component image.

With B ₁for example, wherein, i, j are respectively the position of pixel, and n is the i in first subregion of blue color component image, the span of j, I _b(i, j) is the blue color component value of the coordinate position pixel that is (i, j).B ₁for the blue color component value sum of all pixels in first subregion of blue color component image, B ₂blue color component value sum for all pixels in the second sub regions of blue color component image.

Color characteristic vector is 4 dimensional feature vector t=[t ₁, t ₂, t ₃, t ₄] ^t, wherein:

${\mathrm{<figure-callout\; id="1"\; label="t"\; filenames="HDA00002063078400031.png,HDA00002063078400061.png"\; state="\{\{state\}\}">t</figure-callout>}}_1=\frac{R_1}{R_1+{\mathrm{<figure-callout\; id="1"\; label="G"\; filenames="HDA00002063078400031.png,HDA00002063078400061.png"\; state="\{\{state\}\}">G</figure-callout>}}_1},$ $t_2=\frac{B_1}{B_1+{\mathrm{<figure-callout\; id="1"\; label="G"\; filenames="HDA00002063078400031.png,HDA00002063078400061.png"\; state="\{\{state\}\}">G</figure-callout>}}_1},$ ${\mathrm{<figure-callout\; id="3"\; label="t"\; filenames="HDA00002063078400041.png"\; state="\{\{state\}\}">t</figure-callout>}}_3=\frac{R_2}{R_2+G_2},$ $t_4=\frac{B_2}{B_2+G_2}$

It should be noted that, above-mentioned formula is for exemplarily, and those skilled in the art benefit from the above-mentioned thought of the present invention, can similarly be out of shape above-mentioned formula, for example, use G ₁, G ₂substitute respectively t ₁, t ₂r in middle molecule ₁, R ₂, do not affect the realization of the object of the invention.

Diverse ways is divided into two sub regions by color component images corresponding to each Color Channel of image block again, for example, by the diagonal line of color component images, or by the level of color component images or perpendicular bisector, color component images is divided into two sub regions.

Fig. 2 illustrates the schematic diagram that in a kind of embodiment of images match disposal route provided by the present invention, color component images corresponding to each Color Channel of image block is divided into two sub regions.In Fig. 2, be followed successively by from left to right the division schematic diagram of red component image that red R Color Channel is corresponding, green component image that green G Color Channel is corresponding, blue component image that blue B Color Channel is corresponding.Diagonal line by color component images is divided, and can in concrete computation process, avoid the processing rounding to operate when image block is of a size of odd number, thereby when specific implementation, improve the efficiency of function call.Cornerwise direction can be for from left to bottom right, also can be from lower-left to upper right.As shown in Figure 2, by red component image, according to the diagonal division from left to bottom right, be two sub regions, calculate respectively red color component value sum R1, R2 in two sub regions.Green value G ₁, G ₂, with blue component value B ₁, B ₂calculating and red color component value similar.

Above-mentioned color characteristic vector utilized simultaneously image block colouring information and and positional information, therefore, to adding, make an uproar, smoothly there is higher robustness, can improve in image block coupling the robustness under distortion situation.

Fig. 3, Fig. 4 illustrate respectively the result schematic diagram of color characteristic vector to the robustness test of gaussian additive noise, fuzzy operation in above-described embodiment provided by the present invention.Wherein, corresponding a kind of Y-PSNR (the Peak Signal to Noise Ratio of each ROC curve in Fig. 3, PSNR) the detection effect under situation, the detection effect in Fig. 4 under corresponding a kind of fog-level (igma controls by the gaussian kernel function parameter s) situation of each ROC curve.

Take below the robustness test of color characteristic vector Gaussian additive noise is described as example.

1) from a width RGB image, randomly draw the image block A of a size 24 * 24, and utilize the computing method of above-mentioned color characteristic vector from image block A, to extract eigenvector t _a.

2) to image block A, add gaussian additive noise and obtain noisy image block B, utilize the computing method of above-mentioned color characteristic vector from image block B, to extract eigenvector t _b.

3) from this RGB image, the image block of a size 24 * 24 is randomly drawed in another position that is different from image block A, and add gaussian additive noise and obtain noisy image block C, utilize the computing method of above-mentioned color characteristic vector from image block B, to extract eigenvector t _c.

4) calculate t _awith t _bsimilarity measurement value s, i.e. normalized correlation coefficient between the two.Calculate t _awith t _csimilarity measurement value

5) repeating step 1) to step 4) N time altogether, obtain respectively s _i, i=1,2 ..., N.

6) image block B overlaps with image block A position into the image of coupling, therefore with s _ifor positive example, image block C is different from image block A position, can not mate, therefore for counter-example.Draw ROC curve.This ROC curve shows the detection performance of above-mentioned color characteristic vector.

In Fig. 3, Fig. 4, ROC curve shows in above-described embodiment, color characteristic vector at similar image piece through adding after gaussian additive noise or fuzzy operation, still can keep certain discrimination, visible this color characteristic vector changes less after adding gaussian additive noise or fuzzy operation, there is good unchangeability, thus the robustness of images match preferably.

Fig. 5 illustrates the schematic flow sheet of the another kind of embodiment of images match disposal route provided by the present invention.Shown in Figure 5, the images match disposal route of this embodiment comprises following operation:

Step 501, is divided into target image sub-block by target image piece, calculates the color characteristic vector of each target image sub-block, respectively using the color characteristic vector of each target image sub-block respectively as vectorial one-component, obtain the first color feature vector.

Step 502, employing is divided into target image piece the mode of target image sub-block, by one, being queried above image block is divided into and is queried image subblock, calculate the color characteristic vector that each is queried image subblock, respectively each is queried to the color characteristic vector of image subblock respectively as vectorial one-component, obtains the second color feature vector.

Step 503, calculates the first color feature vector and a similarity measurement value that is queried above the second color feature vector of image block.

Step 504, according to similarity measurement value, is queried above and image block, selects one to be queried image block as the image block matching with target image piece from one.

In the images match disposal route that above-described embodiment provides, by target image piece be queried image block and be divided into less sub-block according to identical mode, calculate respectively target image sub-block and the color characteristic vector that is queried image subblock, to obtain two color feature vectors, in the calculating of carrying out similarity measurement value, thereby with trickleer image block unit, mate, obtain matching result more accurately.

Exemplarily, target image piece can be divided into one by one to N * N target image sub-block, the value of N is to be greater than 1 integer.Accordingly, will be queried image block and be divided into one by one N * N target image sub-block, the value of N is to be greater than 1 integer.

Fig. 6 illustrates in a kind of embodiment of images match disposal route provided by the present invention, target image piece is divided into the schematic diagram of target image piece sub-block.As shown in Figure 6, target image piece is divided into 3 * 3 target image piece sub-blocks.In step 301, calculate the color characteristic vector of each target image sub-block, respectively by the color characteristic vector of each target image sub-block

i=0,1 ..., 8 respectively as vectorial one-component, obtains the first color feature vector.The first color feature vector has 9 components, and each component is the color characteristic vector of one 4 dimension.Accordingly, each is queried image block and according to the mode identical with target image piece, is divided into 9 and is queried image block sub-block, calculate the color characteristic vector that each is queried image subblock, respectively each is queried to the color characteristic vector of image subblock respectively as vectorial one-component, obtains the second color feature vector.

Concrete example of images match disposal route embodiment according to the present invention and unrestricted, in being queried image, choose in the operation that is queried image block, a kind of specific implementation is to be queried centered by the part or all of picture element of image, in being queried image, choose respectively an image block with target image block size as being queried image block successively.When selecting to be queried image block mate calculating from be queried image, can be according to the actual needs, to be queried centered by the part or all of picture element of image, choose and be queried image block, thereby partly or entirely travel through being queried image.While choosing, can the color value that exceed the pixel that is queried image boundary be set according to boundary extension rule centered by the picture element that is queried image border.

Concrete example of images match disposal route embodiment according to the present invention and unrestricted, specifically according to normalized crosscorrelation Y-factor method Y (Normalized Crosscorrelation Coefficient, NCC) or sequential similarity detection algorithm (Sequential Similarity Detection Algorithm, SSDA) calculate similarity measurement value.

Take NCC as example, and the formula that similarity measurement value is calculated according to NCC is:

$s_{x,y}=\mathrm{NCC}(t^{*},t_{x,y})=\frac{t_{x,y}^T\&CenterDot;t^{*}}{\left|t^{*}\right|\&CenterDot;\left|t_{x,y}\right|}$

Wherein, t ^*represent the first color feature vector, what represent to be queried in image that centre coordinate is (x, y) is queried image block the second color feature vector, s _{x, y}represent the similarity measurement value that is queried image block the second color feature vector that the first color feature vector and centre coordinate are (x, y), | t ^*| represent vectorial t ^*delivery,

represent vectorial transposition, represent inner product of vectors.

Concrete example of images match disposal route embodiment according to the present invention and unrestricted, in step 304, specifically can first sort according to numerical values recited to calculate above similarity measurement value; Select the corresponding image block that is queried of the second color feature vector of similarity measurement value maximum as the image block matching with target image piece.

According to the present invention, a concrete example of images match disposal route embodiment is and unrestricted, and color component is the red R of RGB image model, green G, blue B color component.Although RGB image model is current comparatively conventional image model, those skilled in the art benefit from the above-mentioned thought of the present invention, also can select other image models with three color components, carry out images match process operation based on the various embodiments described above.

In addition, the image model of target image piece can be non-RGB image model, and before calculating the color characteristic vector of target image piece, the method also comprises: by target image piece, by non-RGB image mode transform, be RGB image model.Similarly, the image model that is queried image block is non-RGB image model, and be queried the color characteristic vector of image block in calculating before, the method also comprises: will be queried image block is RGB image model by non-RGB image mode transform.

Have between the image model of other color components and RGB image model and can change, concrete conversion method is well known to those skilled in the art, for example, change YCbCr image model to RGB image model.

image block matching treatment device

Fig. 7 illustrates the structural representation of a kind of embodiment of images match treating apparatus provided by the present invention.As shown in Figure 7, this image block matching treatment device embodiment comprises: the first color characteristic vector computing unit 701, the second color characteristic vector computing unit 702, similarity measurement value computing unit 703 and matching image selected cell 704.

The first color characteristic vector computing unit 701 is for calculating the color characteristic vector of target image piece, as the first color characteristic vector.

The second color characteristic vector computing unit 702 is for being queried image successively, choose an image block with target image block size as being queried image block, being queried image block has more than one, calculate respectively a color characteristic vector that is queried above image block, as the second color characteristic vector.

Similarity measurement value computing unit 703 is for calculating the first color characteristic vector and a similarity measurement value that is queried above the second color characteristic vector of image block.

Matching image selected cell 704 selects one to be queried image block as the image block matching with target image piece for being queried above image block according to similarity measurement value from one.

According to the present invention, a concrete example of images match treating apparatus embodiment is and unrestricted, and the first color characteristic vector computing unit 701 comprises color characteristic vector computing module, for calculating the first color characteristic vector, calculates; The second color characteristic vector computing unit 702 comprises color characteristic vector computing module, for calculating the second color characteristic vector, calculates.

The first color characteristic vector computing unit 701, the second color characteristic vector computing unit 702 comprise color characteristic vector computing module separately, calculate the first color characteristic vector, for being queried image block, calculate the second color characteristic vector respectively for target image piece.

Target image piece be queried in image block, each image block comprises three Color Channels, color characteristic vector computing module is divided into two sub regions by color component images corresponding to each Color Channel of image block, calculate respectively the color component value sum in every sub regions of each color component images, be expressed as R ₁, R ₂, G ₁, G ₂, B ₁, B ₂:

${\mathrm{<figure-callout\; id="1"\; label="R"\; filenames="HDA00002063078400031.png,HDA00002063078400061.png"\; state="\{\{state\}\}">R</figure-callout>}}_1=\underset{i=-n}{\overset{n}{\mathrm{\&Sigma;}}}\underset{j=i}{\overset{n}{\mathrm{\&Sigma;}}}{I}_r(i,j),$ ${\mathrm{<figure-callout\; id="1"\; label="G"\; filenames="HDA00002063078400031.png,HDA00002063078400061.png"\; state="\{\{state\}\}">G</figure-callout>}}_1=\underset{j=-n}{\overset{n}{\mathrm{\&Sigma;}}}\underset{i=j}{\overset{n}{\mathrm{\&Sigma;}}}{I}_g(i,j),$ ${\mathrm{<figure-callout\; id="1"\; label="B"\; filenames="HDA00002063078400031.png,HDA00002063078400061.png"\; state="\{\{state\}\}">B</figure-callout>}}_1=\underset{i=-n}{\overset{n}{\mathrm{\&Sigma;}}}\underset{j=i}{\overset{n}{\mathrm{\&Sigma;}}}{I}_b(i,j),$

$R_2=\underset{j=-n}{\overset{n}{\mathrm{\&Sigma;}}}\underset{i=j}{\overset{n}{\mathrm{\&Sigma;}}}{I}_r(i,j),$ $G_2=\underset{i=-n}{\overset{n}{\mathrm{\&Sigma;}}}\underset{j=i}{\overset{n}{\mathrm{\&Sigma;}}}{I}_g(i,j),$ $B_2=\underset{j=-n}{\overset{n}{\mathrm{\&Sigma;}}}\underset{i=j}{\overset{n}{\mathrm{\&Sigma;}}}{I}_b(i,j)$

Wherein, i, j are respectively the position of pixel.

Color characteristic vector is 4 dimensional feature vector t=[t ₁, t ₂, t ₃, t ₄] ^t, wherein:

${\mathrm{<figure-callout\; id="1"\; label="t"\; filenames="HDA00002063078400031.png,HDA00002063078400061.png"\; state="\{\{state\}\}">t</figure-callout>}}_1=\frac{R_1}{{\mathrm{<figure-callout\; id="1"\; label="R"\; filenames="HDA00002063078400031.png,HDA00002063078400061.png"\; state="\{\{state\}\}">R</figure-callout>}}_1+{\mathrm{<figure-callout\; id="1"\; label="G"\; filenames="HDA00002063078400031.png,HDA00002063078400061.png"\; state="\{\{state\}\}">G</figure-callout>}}_1},$ $t_2=\frac{B_1}{{\mathrm{<figure-callout\; id="1"\; label="B"\; filenames="HDA00002063078400031.png,HDA00002063078400061.png"\; state="\{\{state\}\}">B</figure-callout>}}_1+{\mathrm{<figure-callout\; id="1"\; label="G"\; filenames="HDA00002063078400031.png,HDA00002063078400061.png"\; state="\{\{state\}\}">G</figure-callout>}}_1},$ ${\mathrm{<figure-callout\; id="3"\; label="t"\; filenames="HDA00002063078400041.png"\; state="\{\{state\}\}">t</figure-callout>}}_3=\frac{R_2}{R_2+G_2},$ $t_4=\frac{B_2}{B_2+G_2};$

Color characteristic vector computing module can be by the diagonal line of color component images, or by the level of color component images or perpendicular bisector, color component images is divided into two sub regions.

Concrete example of images match treating apparatus embodiment according to the present invention and unrestricted, the first color characteristic vector computing unit 701 is specifically for being divided into target image sub-block by target image piece, color characteristic vector computing module calculates the color characteristic vector of each target image sub-block, respectively using the color characteristic vector of each target image sub-block respectively as vectorial one-component, obtain the first color feature vector; The second color characteristic vector computing unit 702 is specifically for adopting the mode that target image piece is divided into target image sub-block, by one, being queried above image block is divided into and is queried image subblock, color characteristic vector computing module calculates each color characteristic vector that is queried image subblock, respectively each is queried to the color characteristic vector of image subblock respectively as vectorial one-component, obtains the second color feature vector; Similarity measurement value computing unit 703, specifically for calculating the first color feature vector and a similarity measurement value that is queried above the second color feature vector of image block.

The first color characteristic vector computing unit 701, can be divided into target image piece N * N target image sub-block one by one, and the value of N is to be greater than 1 integer; Accordingly, the second color characteristic vector computing unit 702, can be queried above image block by one and be divided into one by one N * N target image sub-block, and the value of N is to be greater than 1 integer.

The second color characteristic vector computing unit 702 can be queried centered by the part or all of picture element of image, in being queried image, chooses respectively an image block with target image block size as being queried image block successively.

Similarity measurement value computing unit 703 can calculate similarity measurement value according to NCC or SSDA.The formula that a kind of NCC calculates similarity measurement value is:

$s_{x,y}=\mathrm{NCC}(t^{*},t_{x,y})=\frac{t_{x,y}^T\&CenterDot;t^{*}}{\left|t^{*}\right|\&CenterDot;\left|t_{x,y}\right|}$

Wherein, t ^*represent the first color feature vector, what represent to be queried in image that centre coordinate is (x, y) is queried image block the second color feature vector, s _{x, y}represent the similarity measurement value that is queried image block the second color feature vector that the first color feature vector and centre coordinate are (x, y), | t ^*| represent vectorial t ^*delivery, represent vectorial transposition, represent inner product of vectors.

Can also calculate similarity measurement value computing unit 703 one above similarity measurement value of matching image selected cell 704 sorts according to numerical values recited, selects the corresponding image block that is queried of the second color feature vector of similarity measurement value maximum as the image block matching with target image piece.

According to the present invention, a concrete example of images match treating apparatus embodiment is and unrestricted, and color component is the red R of RGB image model, green G, blue B color component.

Concrete example of images match treating apparatus embodiment according to the present invention and unrestricted, target image piece is non-RGB image model with the image model that is queried image block, this device also comprises: image mode transform unit, for being RGB image model by target image piece by non-RGB image mode transform before the color characteristic vector calculating target image piece; Be queried the color characteristic vector of image block in calculating before, will be queried image block is RGB image model by non-RGB image mode transform.

image block matching treatment system

Fig. 8 illustrates the composition schematic diagram of a kind of embodiment of images match disposal system provided by the present invention.As shown in Figure 8, a kind of image block matching treatment system comprises: central processing unit (CPU) 801, graphic process unit (Graphic Processing Unit, GPU) 802.

CPU801 is for calculating the color characteristic vector of target image piece, as the first color characteristic vector; According to the similarity measurement value of GPU802 output, from one, be queried above and image block, select one to be queried image block as the image block matching with target image piece.

GPU802 is for being queried image successively, choose an image block with target image block size as being queried image block, be queried image block and have more than one, calculate respectively a color characteristic vector that is queried above image block, as the second color characteristic vector; Calculate the first color characteristic vector and a similarity measurement value that is queried above the second color characteristic vector of image block.

Fig. 9 illustrates the composition schematic diagram of the another kind of embodiment of images match disposal system provided by the present invention.As shown in Figure 9, in this images match disposal system: CPU801 calculates the color characteristic vector of target image piece, as the first color characteristic vector.

GPU802 receives the first color characteristic vector of CPU801 output, and the first color characteristic vector is set to the unified uniform variable of fragment shader; To be queried image setting is data texturing; Fragment shader is played up the picture element of data texturing, particularly, calculating is queried the color characteristic vector of image block, as the second color characteristic vector, calculate the first color characteristic vector and a similarity measurement value that is queried above the second color characteristic vector of image block, similarity value is represented to be queried the texture formal output of image.

CPU801 is according to the similarity measurement value of GPU802 output, from one, is queried above and image block, selects one to be queried image block as the image block matching with target image piece.More specifically, CPU801 can also sort to similarity value; Select the corresponding image block that is queried of the second color characteristic vector of similarity measurement value maximum as the image block matching with target image piece.

Embodiment is corresponding with preceding method, CPU801, GPU802 can also be respectively to target image piece be queried image block and divide, accordingly, calculate respectively the first color feature vector and the second color feature vector that is queried image block, and calculate two vectorial similarity measurement values by CPU801.

With a concrete example, image matching method provided by the present invention is described, the step 1) that some step wherein is for example introduced in example is below also nonessential.Figure 10 illustrates the position view of the target image piece that images match disposal route provided by the present invention chooses.Shown in Figure 10, be queried image I _inbe the coloured image of 512 * 768, to be queried in image block one, to be of a size of 24 * 24 pixel coordinates and to be positioned at the image block I (300,400) locating _blkfor target image piece.In Figure 10, with square frame, marked selected target image piece.The target that images match is processed is to be queried image I _inmiddle inquiry and target image piece I _blkthe most approximate image block, according to a kind of embodiment of this aspect image matching method, concrete steps are:

1) by I _blk, I _inbe converted to RGB image model, for example, need to be from YCbCr color space conversion to RGB color space for jpeg image;

2) utilize CPU to calculate target image piece I _blk4 dimension color characteristic vector t ^*, as the first color characteristic vector;

3) by t ^*bind the uniform variable of GPU;

4) will be queried image I _inas data texturing input GPU;

5) use GPU fragment shader to being queried image I _inprocess, and by result with texture formal output, obtain output image I _out, I _outfor the gray level image (span 0-255) of 512x768, wherein, the concrete grammar that fragment shader is processed is: from being queried image I _ineach location of pixels (x, y) obtain and be of a size of 24 * 24 image block as being queried image block, calculate the 4 dimension color characteristic vector t that each is queried image block _{x, y}, as the second color characteristic vector.Calculate the first color characteristic vector t ^*respectively with the second color characteristic vector t _{x, y}between normalized correlation coefficient as the similarity measurement value s of image block _{x, y}, and assignment is given corresponding I _outthe data texturing of (x, y), and section tinter is to being queried image I _inin the processing of every bit can walk abreast and carry out.Below for using GPU fragment shader to being queried image I _inthe image library shading language of processing (Graphics Library Shader language, GSLS) false code is described:

6) the similarity measurement value s to output by CPU _{x, y}sort, select the corresponding image block that is queried of similarity measurement value maximal value, for the highest image block of target image piece similarity as output.

Figure 11 illustrates the corresponding image block position view that is queried of maximal value that images match disposal route provided by the present invention finds similarity measurement value.As used in Figure 11 as shown in circle frame, the corresponding image block position coordinates that is queried of the maximal value of similarity measurement value, for (300,400), shows to have realized images match accurately.CPU specifically can realize by the first color characteristic vector computing unit, matching image selected cell in the above embodiment of the present invention.GPU specifically can realize by the second color characteristic vector computing unit, similarity measurement value computing unit in the above embodiment of the present invention.

The image block matching treatment system providing based on the above embodiment of the present invention, has calculated amount compared with the color characteristic vector of low dimension little, is convenient to utilize the feature of GPU texture formal output.In prior art, realize the method for robust coupling and need to carry out integer calculations and branch's judgement, therefore, be unfavorable for carrying out parallel computation based on GPU software and hardware architecture.And images match disposal route provided by the present invention, device and system are easy to adopt the software and hardware architecture of GPU processor to realize, especially, adopt the GPU of the unified rendering pipeline realization of programming, thereby take full advantage of the characteristic of the parallel computation of GPU fragment shader, realization is processed in real time to comprising the image of frame of video, can be applied to Video coding, video denoising, image registration, image copy detection etc. need quick comparison chart as the application scenarios of block similarity.

So far, described in detail according to a kind of images match disposal route of the present invention, Apparatus and system.For fear of covering design of the present invention, details more known in the field are not described.Those skilled in the art, according to description above, can understand how to implement technical scheme disclosed herein completely.

In this instructions, each embodiment all adopts the mode of going forward one by one to describe, and each embodiment stresses is the difference with other embodiment, same or analogous part cross-references between each embodiment.For device, system embodiment, because it is substantially corresponding with embodiment of the method, so description is fairly simple, relevant part is referring to the part explanation of embodiment of the method.

May realize in many ways images match disposal route of the present invention, Apparatus and system.For example, can realize method and system of the present invention by any combination of software, hardware, firmware or software, hardware, firmware.The said sequence that is used for the step of described method is only in order to describe, and the step of method of the present invention is not limited to above specifically described order, unless otherwise specified.In addition, in certain embodiments, can be also the program being recorded in recording medium by the invention process, these programs comprise for realizing the machine readable instructions of the method according to this invention.Thereby the present invention also covers storage for carrying out the recording medium of the program of the method according to this invention.

Although specific embodiments more of the present invention are had been described in detail by example, it should be appreciated by those skilled in the art, above example is only in order to describe, rather than in order to limit the scope of the invention.It should be appreciated by those skilled in the art, can without departing from the scope and spirit of the present invention, above embodiment be modified.Scope of the present invention is limited by claims.

Claims (24)

1. an image block matched processing method, is characterized in that, the method comprises:

Calculate the color characteristic vector of target image piece, as the first color characteristic vector;

Successively in being queried image, choose an image block with described target image block size as being queried image block, the described image block that is queried has more than one, calculates respectively a color characteristic vector that is queried above image block, as the second color characteristic vector;

Calculate described the first color characteristic vector and a similarity measurement value that is queried above the second color characteristic vector of image block;

According to described similarity measurement value, from one, be queried above and image block, select one to be queried image block as the image block matching with described target image piece.

2. method according to claim 1, is characterized in that, calculates described color characteristic vector and comprises:

Target image piece be queried in image block, each image block comprises three Color Channels, color component images corresponding to each Color Channel of described image block is divided into two sub regions, calculate respectively the color component value sum in every sub regions of each color component images, be expressed as R ₁, R ₂, G ₁, G ₂, B ₁, B ₂;

Described color characteristic vector is 4 dimensional feature vector t=[t ₁, t ₂, t ₃, t ₄] ^t, wherein:

$t_1=\frac{R_1}{R_1+G_1},$ $t_2=\frac{B_1}{B_1+G_1},$ $t_3=\frac{R_2}{R_2+G_2},$ $t_4=\frac{B_2}{B_2+G_2}.$

3. method according to claim 2, is characterized in that, described color component images corresponding to each Color Channel of described image block is divided into two sub regions, comprising:

By the diagonal line of described color component images, or by the level of described color component images or perpendicular bisector, described color component images is divided into two sub regions.

4. method according to claim 3, is characterized in that, the color characteristic vector of described calculating target image piece, comprising:

Described target image piece is divided into target image sub-block, calculates the color characteristic vector of each target image sub-block, respectively using the color characteristic vector of each target image sub-block respectively as vectorial one-component, obtain the first color feature vector;

Describedly calculate respectively a color characteristic vector that is queried above image block, as the second color characteristic vector, comprising:

Employing is divided into described target image piece the mode of target image sub-block, by described one, being queried above image block is divided into and is queried image subblock, calculate the color characteristic vector that each is queried image subblock, respectively each is queried to the color characteristic vector of image subblock respectively as vectorial one-component, obtains the second color feature vector;

Described the first color feature vector of described calculating and a similarity measurement value that is queried above the second color feature vector of image block, comprising:

Calculate described the first color feature vector and a similarity measurement value that is queried above the second color feature vector of image block.

5. method according to claim 4, is characterized in that, described described target image piece is divided into target image sub-block, comprising:

Described target image piece is divided into N * N target image sub-block, and the value of N is to be greater than 1 integer;

Describedly by described one, be queried above image block and be divided into and be queried image subblock, comprising:

By described one, be queried above image block and be divided into one by one N * N target image sub-block, the value of N is to be greater than 1 integer.

6. method according to claim 5, is characterized in that, described successively in being queried image, chooses an image block with described target image block size as being queried image block, comprising:

Centered by the described part or all of picture element that is queried image, described, be queried in image successively, choose respectively an image block with described target image block size as being queried image block.

7. method according to claim 1, is characterized in that, specifically according to normalized crosscorrelation Y-factor method Y NCC or sequential similarity detection algorithm SSDA, calculates similarity measurement value.

8. method according to claim 7, is characterized in that, described similarity measurement value is calculated according to NCC, and computing formula is:

$s_{x,y}=\mathrm{NCC}(t^{*},t_{x,y})=\frac{t_{x,y}^T\&CenterDot;t^{*}}{\left|t^{*}\right|\&CenterDot;\left|t_{x,y}\right|}$

Wherein, t ^*represent the first color feature vector,

what represent to be queried in image that centre coordinate is (x, y) is queried image block the second color feature vector, s _{x, y}represent the similarity measurement value that is queried image block the second color feature vector that the first color feature vector and centre coordinate are (x, y), | t ^*| represent vectorial t ^*delivery,

represent vectorial transposition, represent inner product of vectors.

9. method according to claim 4, is characterized in that, describedly from one, is queried above and image block, selects to be queried described in one image block as the image block matching with described target image piece, comprising:

Calculate above similarity measurement value is sorted according to numerical values recited;

Select the corresponding image block that is queried of the second color feature vector of similarity measurement value maximum as the image block matching with described target image piece.

10. method according to claim 1, is characterized in that, described color component is the red R of RGB image model, green G, blue B color component.

11. methods according to claim 1, it is characterized in that, the image model of described target image piece is non-RGB image model, before calculating the color characteristic vector of target image piece, also comprises: by described target image piece, by non-RGB image mode transform, be RGB image model;

The described image model that is queried image block is non-RGB image model, before being queried the color characteristic vector of image block, also comprises in calculating: described in inciting somebody to action, being queried image block is RGB image model by non-RGB image mode transform.

12. 1 kinds of image block matching treatment devices, is characterized in that, described device comprises:

The first color characteristic vector computing unit, for calculating the color characteristic vector of target image piece, as the first color characteristic vector;

The second color characteristic vector computing unit, successively in being queried image, choose an image block with described target image block size as being queried image block, the described image block that is queried has more than one, calculate respectively a color characteristic vector that is queried above image block, as the second color characteristic vector;

Similarity measurement value computing unit, for calculating described the first color characteristic vector and a similarity measurement value that is queried above the second color characteristic vector of image block;

Matching image selected cell, for according to described similarity measurement value, is queried above and image block, selects one to be queried image block as the image block matching with described target image piece from one.

13. devices according to claim 12, is characterized in that, described the first color characteristic vector computing unit comprises color characteristic vector computing module, for calculating the first color characteristic vector, calculate;

Described the second color characteristic vector computing unit comprises color characteristic vector computing module, for calculating the second color characteristic vector, calculates;

Target image piece be queried in image block, each image block comprises three Color Channels, described color characteristic vector computing module is divided into two sub regions by color component images corresponding to each Color Channel of described image block, calculate respectively the color component value sum in every sub regions of each color component images, be expressed as R ₁, R ₂, G ₁, G ₂, B ₁, B ₂;

Described color characteristic vector is 4 dimensional feature vector t=[t ₁, t ₂, t ₃, t ₄] ^t, wherein:

$t_1=\frac{R_1}{R_1+G_1},$ $t_2=\frac{B_1}{B_1+G_1},$ $t_3=\frac{R_2}{R_2+G_2},$ $t_4=\frac{B_2}{B_2+G_2}.$

14. devices according to claim 13, it is characterized in that, described color characteristic vector computing module is pressed the diagonal line of described color component images, or by the level of described color component images or perpendicular bisector, described color component images is divided into two sub regions.

15. devices according to claim 14, it is characterized in that, described the first color characteristic vector computing unit, specifically for described target image piece is divided into target image sub-block, described color characteristic vector computing module calculates the color characteristic vector of each target image sub-block, respectively using the color characteristic vector of each target image sub-block respectively as vectorial one-component, obtain the first color feature vector;

The second color characteristic vector computing unit, specifically for adopting the mode that described target image piece is divided into target image sub-block, by described one, being queried above image block is divided into and is queried image subblock, described color characteristic vector computing module calculates the color characteristic vector that each is queried image subblock, respectively each is queried to the color characteristic vector of image subblock respectively as vectorial one-component, obtains the second color feature vector;

Described similarity measurement value computing unit, specifically for calculating described the first color feature vector and a similarity measurement value that is queried above the second color feature vector of image block.

16. devices according to claim 15, is characterized in that, described the first color characteristic vector computing unit, and specifically for described target image piece is divided into N * N target image sub-block, the value of N is to be greater than 1 integer;

Described the second color characteristic vector computing unit, is divided into N * N target image sub-block one by one specifically for being queried above image block by described one, and the value of N is to be greater than 1 integer.

17. devices according to claim 16, it is characterized in that, described the second color characteristic vector computing unit, specifically for centered by the described part or all of picture element that is queried image, described, be queried in image successively, choose respectively an image block with described target image block size as being queried image block.

18. devices according to claim 12, is characterized in that, described similarity measurement value computing unit specifically calculates similarity measurement value according to NCC or SSDA.

19. devices according to claim 18, is characterized in that, described similarity measurement value computing unit calculates similarity measurement value according to NCC, and computing formula is:

$s_{x,y}=\mathrm{NCC}(t^{*},t_{x,y})=\frac{t_{x,y}^T\&CenterDot;t^{*}}{\left|t^{*}\right|\&CenterDot;\left|t_{x,y}\right|}$

Wherein, t ^*represent the first color feature vector,

expression take be queried in image that centre coordinate is (x, y) be queried image block the second color feature vector, s _{x, y}represent the first color feature vector and take the similarity measurement value that is queried image block the second color feature vector that centre coordinate is (x, y), | t ^*| represent vectorial t ^*delivery,

represent vectorial transposition, represent inner product of vectors.

20. devices according to claim 15, is characterized in that, described matching image selected cell sorts according to numerical values recited to calculate above similarity measurement value; Select the corresponding image block that is queried of the second color feature vector of similarity measurement value maximum as the image block matching with described target image piece.

21. devices according to claim 12, is characterized in that, described color component is the red R of RGB image model, green G, blue B color component.

22. devices according to claim 12, it is characterized in that, described target image piece is non-RGB image model with the image model that is queried image block, described device also comprises: image mode transform unit, for being RGB image model by described target image piece by non-RGB image mode transform before the color characteristic vector calculating target image piece; Be queried the color characteristic vector of image block in calculating before, described in inciting somebody to action, being queried image block is RGB image model by non-RGB image mode transform.

23. 1 kinds of image block matching treatment systems, is characterized in that, comprising: central processor CPU, graphic process unit GPU;

Described CPU is for calculating the color characteristic vector of target image piece, as the first color characteristic vector; According to the similarity measurement value of described GPU output, from one, be queried above and image block, select one to be queried image block as the image block matching with described target image piece;

Described GPU is for being queried image successively, choose an image block with described target image block size as being queried image block, the described image block that is queried has more than one, calculates respectively a color characteristic vector that is queried above image block, as the second color characteristic vector; Calculate described the first color characteristic vector and a similarity measurement value that is queried above the second color characteristic vector of image block.

24. systems according to claim 23, is characterized in that, described GPU is specifically for receiving the described first color characteristic vector of described CPU output, and described the first color characteristic vector is set to the unified uniform variable of fragment shader; Described in inciting somebody to action, being queried image setting is data texturing; Fragment shader is played up calculating to the picture element of described data texturing, calculating is queried the color characteristic vector of image block, as the second color characteristic vector, calculate described the first color characteristic vector and a similarity measurement value that is queried above the second color characteristic vector of image block, described similarity is represented with the described texture formal output that is queried image.

Images