CN102375984B - Characteristic quantity calculating device, image connecting device, image retrieving device and characteristic quantity calculating method - Google Patents

Abstract

The invention provides a characteristic quantity calculating device, an image connecting device, an image retrieving device and a characteristic quantity calculating method, which can work out characteristic quantity being strong in rotation robustness of an image in a contrast-unrelated way. The characteristic quantity calculating device is provided with a phase angle quantity calculation part for calculating phase angle quantity, a characteristic point detection part for detecting a characteristic point, a submission ascendance orientation calculation part for calculating submission ascendance orientation in the vicinity of the characteristic point, a concentric circle detection part for detecting pixel data on circumferences of p circles taking the characteristic point as the center, a weighted differential value calculation part for respectively working out differential values between gradient angles of pixel values in all the pixel data on the circumferences and submission ascendance gradient and multiplying with the square roots of radii of the circles for getting weighted differential values, a degree distribution generation part for generating q-stage degree distribution of the weighted differential values and a description symbol vector calculation part for working out pXq-dimensional description symbol vectors according to q-dimensional vectors taking respective degrees of the p circles as components.

Description

Characteristic quantity calculation element, image coupling arrangement, image retrieving apparatus and characteristic quantity calculating method

Technical field

The present invention relates to characteristic quantity calculation element, image coupling arrangement, image retrieving apparatus and characteristic quantity calculating method.

background technology

For a plurality of images are connected and synthesize panoramic picture or retrieve other images from an image, the distinctive point (unique point) in each image of general using.For example, by detecting characteristic of correspondence point in each image, utilize the unique point of lap, can synthesize panoramic picture.

Each unique point by for each unique point for the characteristic quantity of intrinsic amount is described.Characteristic quantity is different according to the computing method of characteristic quantity.In United States Patent (USP) the 6711293rd, record a kind of SIFT of being called as (Scale Invariant Feature Transformation: characteristic quantity calculating method yardstick invariant features transfer algorithm).According to SIFT, can calculate the characteristic quantity to the ratiometric conversion resistance of image.

But, there is the characteristic quantity calculate by the SIFT problem a little less than aspect the rotation of image.And also there is the problem of the characteristic quantity that is not suitable for calculating the weak image of contrast in SIFT.

summary of the invention

The present invention proposes in order to solve above-mentioned problem, its object is, providing a kind of can independently calculate the characteristic quantity calculation element of the characteristic quantity of the rotation strong robustness of image, image coupling arrangement, image retrieving apparatus and characteristic quantity calculating method with contrast.

The characteristic quantity calculation element the present invention relates to, is characterized in that possessing:

Phase angle amount calculating part, a plurality of pixel datas of the view data that it is transfused to for formation, calculate respectively Harris's matrix, utilize Harris's matrix calculating to calculate respectively phase angle amount;

Feature point detecting unit, it detects in the pixel data that forms described view data, has the pixel data of the phase angle amount larger than the phase angle amount of adjacent pixel data around, as unique point;

Mastery orientation calculation portion, it calculates according to the pixel value of the pixel data of the pixel value of described unique point and this feature neighborhood of a point the mastery orientation that the angle of the gradient of the pixel value of this unique point vicinity is represented;

Concentric circles detection portion, it detects in described view data, the pixel data on the circumference of the individual circle of p (p >=2) that radius centered by described unique point is different;

Weight residual quantity value calculating part, it is for p circle, calculate respectively to the angle of the gradient of the pixel value in each pixel data on circumference, with the residual quantity value of described mastery gradient, be multiplied by the square root of radius of a circle and the weight residual quantity value that obtains;

Number of degrees distribution generating unit, it generates respectively for p circle individual other number of degrees of level of q (q >=2) that have about described weight residual quantity value and distributes; With

Descriptor vector calculating part, it calculates respectively and usings each number of degrees as the q dimensional vector of composition for p circle, and according to each q dimensional vector calculating, calculates p * q and tie up Descriptor vector.

According to the present invention, by Descriptor vector calculating part, calculate p * q and tie up Descriptor vector, as characteristic quantity.This Descriptor vector is not vulnerable to the impact of ratiometric conversion and rotation.In addition, even the weak view data of contrast, for each Descriptor vector of each unique point, independence is separately also high.Therefore, can provide a kind of and independently calculate the characteristic quantity calculation element to the characteristic quantity of the rotation resistance of image with contrast image.

And, the invention is characterized in, possesses image data set generating unit, its generation comprises a view data and by the image data set of other view data of dwindling after this view data smoothing and obtaining, replace described view data, the image data set of generation is inputed to described phase angle amount calculating part.

According to the present invention, owing to using the image data set being generated by image data set generating unit to carry out the detection of unique point, so can calculate the Descriptor vector that independence is higher.

And, the invention is characterized in, described mastery orientation calculation portion calculates: the angle mean value Ao of the gradient of the pixel value in 8 pixel datas of described unique point and this feature neighborhood of a point; The angle mean value Ac of the gradient of the pixel value of the pixel data of the radius of take centered by this unique point in the circle of 3 pixels; If Ac is greater than Ao, calculate the mean value of angle of gradient that the radius take centered by this unique point Grad in the circle of 3 pixels is less than the pixel data of Ac, this mean value is orientated as described mastery; If Ac is less than Ao, calculate the mean value of angle of gradient that the radius take centered by this unique point Grad in the circle of 3 pixels is greater than the pixel data of Ac, this mean value is orientated as described mastery; If Ac equals Ao, the mean value Ao of usining is orientated as described mastery.

According to the present invention, because the mastery of angle that can obtain the gradient of the pixel value further reflected unique point vicinity by mastery orientation calculation portion is orientated, so can calculate the Descriptor vector that independence is higher.

In addition, the invention is characterized in, described Concentric circles detection portion is set as 8 by the value of p,

Described number of degrees distribution generating unit is set as 8 by the value of q.

According to the present invention, by the value of p and q is set as to 8, can calculate the Descriptor vector that independence is higher.

And the image coupling arrangement the present invention relates to possesses: described characteristic quantity calculation element; With utilize described Descriptor vector that 2 above view data are connected, synthesize the image connecting portion of 1 view data.

According to the present invention, by possessing above-mentioned characteristic quantity calculation element, can provide high precision to connect the image coupling arrangement of view data.

And the image retrieving apparatus the present invention relates to possesses: described characteristic quantity calculation element; With utilize described Descriptor vector and from a view data, retrieve the image retrieval portion of other view data.

According to the present invention, by possessing above-mentioned characteristic quantity calculation element, can provide the image coupling arrangement of retrieve image data accurately.

In addition, the characteristic quantity calculating method the present invention relates to possesses: to forming a plurality of pixel datas of the view data being transfused to, calculate respectively Harris's matrix, utilize Harris's matrix calculating to calculate respectively the step of phase angle amount;

Detect in the pixel data that forms described view data, there is the pixel data of the phase angle amount larger than the phase angle amount of adjacent pixel data around, as the step of unique point;

According near the pixel value of the pixel data of the pixel value of described unique point and this unique point, calculate the step of the mastery orientation that the angle of the gradient of the pixel value of this unique point vicinity is represented;

Detection in described view data, the step of the pixel data on the circumference of the individual circle of p (p >=2) that radius centered by described unique point is different;

For p circle, calculate respectively to the angle of the gradient of the pixel value in each pixel data on circumference, with the residual quantity value of described mastery gradient, be multiplied by the square root of radius of a circle and the step of the weight residual quantity value that obtains;

For p circle, generate respectively the step having about other number of degrees distribution of the individual level of q (q >=2) of described weight residual quantity value; With

For p circle, calculate respectively and using the number of degrees as the q dimensional vector of key element, and according to each q dimensional vector calculating, calculate the step of p * q dimension Descriptor vector.

According to the present invention, can calculate p * q dimension Descriptor vector as characteristic quantity.This Descriptor vector is not vulnerable to the impact of ratiometric conversion and rotation.In addition, even the weak view data of contrast, for each Descriptor vector of each unique point, independence is separately also high.Therefore, can provide with the contrast of image and independently calculate the characteristic quantity calculating method to the characteristic quantity of the rotation resistance of image.

By following detailed explanation and drawing, can more know object of the present invention, characteristic and advantage.

Accompanying drawing explanation

Fig. 1 means the schematic diagram of the formation of characteristic quantity calculation element 1.

Fig. 2 means the process flow diagram of Descriptor vector computing.

Fig. 3 is the concept map of image pyramid G.

Fig. 4 means the process flow diagram of detailed step of the processing of steps A 21.

Fig. 5 means the process flow diagram of detailed step of the processing of steps A 23.

Fig. 6 means the process flow diagram of detailed step of the processing of steps A 24.

Fig. 7 means the process flow diagram of detailed step of the processing of steps A 25.

Fig. 8 means the process flow diagram of detailed step of the processing of steps A 251.

Fig. 9 means that image connects the process flow diagram of processing.

Figure 10 means the figure of subject 100.

Figure 11 means the figure of 4 view data 200A, 200B, 200C, 200D.

Figure 12 means the figure of the view data 300A after connection.

Figure 13 means the figure of the view data 300B after finishing (trimming) is processed.

Figure 14 means the process flow diagram that image retrieval is processed.

Embodiment

Below, with reference to accompanying drawing, the preferred embodiment of the present invention is described in detail.

Below, the characteristic quantity calculation element 1 for embodiments of the present invention describes.Characteristic quantity calculation element 1 is according to forming from a plurality of pixel datas of the view data of the inputs such as digital camera, scanner, calculates the device of the amount (characteristic quantity) of the feature that represents this view data.

Fig. 1 means the schematic diagram of the formation of characteristic quantity calculation element 1.Characteristic quantity calculation element 1 possesses: control algorithm portion 11, storage part 12, operating portion 13, display part 14, Department of Communication Force 15 and interface portion 16.

Control algorithm portion 11 is the devices that carry out the calculation process based on program and carry out the pipage control processing etc. of data at different electric rooms.As control algorithm portion 11, can use CPU (Central Processing Unit), DSP (Digital Signal Processor: digital signal processor) etc.

Double data rate SDRAM (Synchronous dynamic random access memory)) hard disk drive) storage part 12 comprises: DDR SDRAM (Double Data Rate SynchronousDynamic Random Access Memory: volatile memory and HDD (the Hard Disk Drive: the nonvolatile memory such as such as.In nonvolatile memory, store characteristic quantity calculation procedure described later.

Operating portion 13 is such as being mouse, keyboard, touch-screen etc.User can operate by 13 pairs of characteristic quantity calculation elements 1 of operating portion.Display part 14 is such as being liquid crystal indicator etc.On display part 14, the result of the calculation process that control algorithm portion 11 carries out etc. is shown as image.Department of Communication Force 15 is the devices that communicate by telephone line, internet etc. and characteristic quantity calculation element 1 device in addition.

Interface portion 16 is the connecting portions for USB (Universal Serial Bus) connection, the connection of ETHER net etc.For example, characteristic quantity calculation element 1 can be connected by interface portion 16 and digital camera etc., the view data of storing in this digital camera can be stored in storage part 12.

And characteristic quantity calculation element 1 is configured to, the view data being received by Department of Communication Force 15 is stored in storage part 12.And characteristic quantity calculation element 1 possesses not shown scanner, be configured to the view data being obtained by this scanner is stored in storage part 12.

In storage part 12, the characteristic quantity calculation procedure of storage, is for to being stored in as described above the view data of storage part 12, calculates the program that Descriptor vector is used as characteristic quantity.By processing according to characteristic quantity calculation procedure, control algorithm portion 11 and storage part 12 are as phase angle amount calculating part, feature point detecting unit, mastery orientation calculation portion, Concentric circles detection portion, weight residual quantity value calculating part, number of degrees distribution generating unit, Descriptor vector calculating part and the image data set generating unit performance function that the present invention relates to.

In the present invention, phase angle amount calculating part has a plurality of pixel datas of the view data being transfused to for formation, calculates respectively Harris's matrix, utilizes Harris's matrix calculating to calculate respectively the function of phase angle amount.Feature point detecting unit have by the pixel data of composing images data, there is the pixel data of the phase angle amount larger than the phase angle amount of pixel data around, as feature point detection function out.Mastery orientation calculation portion has according near the pixel value of the pixel data pixel value of unique point and this unique point, calculates the function of the mastery orientation that near the angle of the gradient of the pixel value this unique point is represented.

Concentric circles detection portion has in view data, detects the function of the pixel data on the circumference of the different individual circle of p (p >=2) of radius centered by unique point.Weight residual quantity value calculating part has for p circle, calculates respectively the residual quantity value to the angle of the gradient of the pixel value in each pixel data on circumference and mastery orientation, is multiplied by the square root of radius of a circle and the function of the weight residual quantity value that obtains.Number of degrees distribution generating unit has for p circle, generates respectively the function having about other number of degrees distribution of the individual level of q (q >=2) of weight residual quantity value.Descriptor vector calculating part has for p circle, calculates respectively and usings each number of degrees as the q dimensional vector of composition, and according to each q dimensional vector calculating, calculate the function of p * q dimension Descriptor vector.

And, in the present embodiment, image data set generating unit has to generate and comprises a view data and this view data is carried out being dwindled after smoothing and the image data set of other view data of obtaining, and is entered into the function of phase angle amount calculating part.And mastery orientation calculation portion is configured to, calculate the angle mean value Ao of the gradient of the pixel value in unique point and this unique point 8 pixel datas around; The angle mean value Ac of the gradient of the pixel value of the pixel data of the radius of take centered by this unique point in the circle of 3 pixels; If Ac is greater than Ao, calculate the mean value of angle of gradient that the radius take centered by this unique point Grad in the circle of 3 pixels is less than the pixel data of Ac, this mean value is orientated as described mastery; If Ac is less than Ao, calculate the mean value of angle of gradient that the radius take centered by this unique point Grad in the circle of 3 pixels is greater than the pixel data of Ac, this mean value is orientated as described mastery; If Ac equals Ao, the mean value Ao of usining is orientated as described mastery.

By the function of these each portions, can calculate Descriptor vector and be used as characteristic quantity.The Descriptor vector calculating is used in the connection of image described later and the retrieval of image etc.In addition, as other embodiments of the present invention, also can be substituted in and in storage part 12, store characteristic quantity calculation procedure, and by respectively independently electronic circuit form above-mentioned each portion with above-mentioned functions.

Below, the processing (Descriptor vector computing) that characteristic quantity calculation element 1 is calculated to Descriptor vector by the function of above-mentioned each portion is elaborated.Fig. 2 means the process flow diagram of Descriptor vector computing.Descriptor vector computing comprises: the pyramidal processing of synthetic image (steps A 21), calculate the phase angle amount of each pixel data in image pyramid processing (steps A 22), according to each phase angle amount detect local feature point processing (steps A 23), from local feature point, detect the processing (steps A 24) of remarkable characteristic and for remarkable characteristic, calculate the processing (steps A 25) of Descriptor vector.

First, the processing of steps A 21 is described.The processing of steps A 21 be according to image based on from inputs such as digital cameras, there is brightness value as the view data (hereinafter referred to as " view data of the 0th standard ") of pixel value, the pyramidal processing of synthetic image.

In the present invention, image pyramid refer to the view data of the 0th standard and to this view data carry out fixed processing and the view data of the n that obtains (using divide into n represent natural number) standard as the image data set of key element.Here, form each pixel data of the view data of the 0th standard, by the X as rectangular coordinate ₀y ₀each coordinate figure in coordinate

the brightness value of function represent, establish this X ₀y ₀each coordinate figure in coordinate

for X ₀coordinate and Y ₀any one of coordinate be continuous round values all.And, form each pixel data of the view data of n standard, by the X as rectangular coordinate _ny _neach coordinate figure (u in coordinate _xn, v _yn) the brightness value I of function _n(u _xn, v _yn) represent, establish this X _ny _neach coordinate figure (u in coordinate _xn, v _yn) be for X _ncoordinate and Y _nany one of coordinate be continuous round values all.

As an example of image pyramid, in Fig. 3, represented the concept map of image pyramid G.Image pyramid G shown in Fig. 3 is by the image pyramid G of the 0th standard ₀, the 1st standard view data G ₁, and the view data G of the 2nd standard ₂form.

Fig. 4 means the process flow diagram of detailed step of the processing of steps A 21.First, the value of variable m is set to 0 (steps A 211).Then, for the view data of m standard, carry out the smoothing (steps A 212) of the convolution based on Gaussian filter.

Specifically, in the processing of steps A 212, by following formula (1), by each pixel data I that forms the view data of m standard _m(u _xm, v _ym), obtain X _my _meach coordinate figure (s in coordinate _xm, t _ym) the J of function _m(s _xm, t _ym).

[numerical expression 1]

…(1)

$J_m(s_{\mathrm{Xm}},t_{\mathrm{Ym}})=\∫\∫\frac{\exp (-(u_{\mathrm{Xm}}^2+v_{\mathrm{Ym}}^2)/2{\mathrm{\σ}}^2)}{\sqrt{2\mathrm{\π\σ}}}$

$\×I_m(s_{\mathrm{Xm}}-u_{\mathrm{Xm}},t_{\mathrm{Ym}}-v_{\mathrm{Ym}}){\mathrm{du}}_{\mathrm{Xm}}{\mathrm{dv}}_{\mathrm{Ym}}$

In above-mentioned formula (1), the value of constant σ is for example set to

formation is passed through each pixel data of the view data after smoothedization of processing of steps A 212 by J _m(s _xm, t _ym) represent.

After the processing of steps A 212, for the view data after smoothing, by forming the resampling of the pixel data of this view data, process to carry out downsizing (steps A 213).More specifically, according to X _mthe sum of the pixel data in coordinate becomes 1/2 and Y _mthe sum of the pixel data in coordinate becomes 1/2 mode, and processing equably resamples.By this, resample and process, can obtain the view data as the m+1 standard of reduced view data.Therefore, can be by the coordinate (X of the m standard as the coordinate before processing that resamples _my _mcoordinate), represent the coordinate figure of each pixel data of the view data of m+1 standard, also can be by the coordinate (X of the 0th standard ₀y ₀coordinate) represent the coordinate figure of each pixel data of the view data of m+1 standard.

After the processing of steps A 213, for the view data of m+1 standard, judge whether picture size is less than 32 * 32 (steps A 214).That is,, for the view data of m+1 standard, determine whether X _m+1the number of the pixel data in coordinate is less than 32 and Y _m+1the number of the pixel data in coordinate is less than 32.In the situation that the result of determination of steps A 214 is picture size, be not less than 32 * 32, the value of variable m is increased to 1 (steps A 215), then turn back to the processing of steps A 212.

In the situation that being picture size, the result of determination of steps A 214 is less than 32 * 32, the processing of end step A21.From the processing that starts steps A 21 to the view data of the 1st～the n standard and the view data of the 0th standard that generate during finishing, be used as image pyramid and store in storage part 12.

As shown in Figure 2, after the processing of steps A 21, carry out the processing for the steps A 22 of the phase angle amount of each pixel data of computed image pyramid.Harris's matrix of the following explanation of calculating utilization of phase angle amount carries out.

Each pixel data of the view data of the formation z in image pyramid (representing more than 0 integer to divide into z) standard is by I _z(u _xz, v _yz) represent.When by I _z(u _xz, v _yz) at X _zpartial differential coefficient in coordinate is expressed as [numerical expression 2]

By it at Y _zpartial differential coefficient in coordinate is expressed as [numerical expression 3] time, Harris's matrix M (u _xz, v _yz) by following formula (2), represented.

[numerical expression 4]

$M(u_{\mathrm{Xz}},v_{\mathrm{Yz}})=\left[\begin{array}{cc}{\left(\frac{\∂I_z}{\∂X_z}\right)}^2& \left(\frac{\∂I_z}{\∂X_z}\right)\left(\frac{\∂I_z}{\∂Y_z}\right)\\ \left(\frac{\∂I_z}{\∂X_z}\right)\left(\frac{\∂I_z}{\∂Y_z}\right)& {\left(\frac{\∂I_z}{\∂Y_z}\right)}^2\end{array}\right]...\left(2\right)$

The phase angle amount R (u of each pixel data _xz, v _yz), according to Harris's matrix M (u in this pixel data _xz, v _yz) by following formula (3), calculated.

R(u _Xz，v _Yz)＝det(M(u _Xz，v _Yz))

-k×(trace(M(u _Xz，v _Yz))) ²…(3)

In above-mentioned formula (3), the value of constant k is set in 0.04～0.15 scope.In the present embodiment, k=0.04.

Like this, in the processing of steps A 22, for the key element of image pyramid, form the pixel data of the view data of each standard, by above-mentioned formula (2), (3), calculate phase angle amount R (u _xz, v _yz).The phase angle amount R (u calculating _xz, v _yz) be stored in storage part 12.

As shown in Figure 2, after the processing of steps A 22, carry out, according to each phase angle amount, from the pyramidal view data of composing images, detecting the processing of the steps A 23 of local feature point.

Fig. 5 means the process flow diagram of detailed step of the processing of steps A 23.First, the value of variable j is set to 0 (steps A 231).Then, the value of variable i is set to 1 (steps A 232).Variable i means the natural number of the pixel data of the view data that forms j standard.

Then,, to forming i pixel data of the view data of j standard, judge that whether phase angle amount is than the phase angle amount of pixel data around large (steps A 233).More specifically, when by i the pixel data of view data that forms j standard at X _jy _jcoordinate figure in coordinate is made as (u _xj, v _yj) time, judge the phase angle amount R (u of this pixel data _xj, v _yj) whether than R (u _xj, v _yj+ 1), R (u _xj+ 1, v _yj+ 1), R (u _xj+ 1, v _yj), R (u _xj+ 1, v _yj-1), R (u _xj, v _yj-1), R (u _xj-1, v _yj-1), R (u _xj-1, v _yj) and R (u _xj-1, v _yj+ 1) any one of this 8 phase angle amounts is all large.Wherein, in the present embodiment, although the number of the pixel data of the surrounding of the comparison other as i pixel data is made as to 8, for example, also can be made as 15.

If it is large unlike pixel around that the result of determination of steps A 233 is phase angle amounts, enter into the processing of steps A 235.If it is larger than pixel around that the result of determination of steps A 233 is phase angle amounts, form i the pixel data of view data of j standard at X _jy _jcoordinate figure (u in coordinate _xj, v _yj) be stored in (steps A 234) in storage part 12, then, enter into the processing of steps A 235.

In steps A 235, judge whether i pixel data is the last pixel data as the judgement object of steps A 233 in the view data of j standard.If the result of determination of steps A 235 is last pixel datas, enter into the processing of steps A 236, if not last pixel data, enter into the processing of steps A 237.

In the processing of steps A 237, the value of variable i increases by 1, then, turns back to the processing of steps A 233.In the processing of steps A 236, judge whether the view data of j standard is the last view data as the judgement object of steps A 233 in image pyramid.If the result of determination of steps A 236 is last view data, the processing of end step A23.If the result of determination of steps A 236 is not last view data, enter into the processing of steps A 238.In the processing of steps A 238, the value of variable j increases by 1, then turns back to the processing of steps A 232.

Like this, in the processing of steps A 23, stored the coordinate figure of pixel data.The pixel data that has been stored coordinate figure is local feature point.

As shown in Figure 2, after the processing of steps A 23, carry out for detect the processing of the steps A 24 of remarkable characteristic from local feature point.In the present embodiment, by the processing of steps A 24, from the view data of the pyramidal all standards of composing images, detect and be less than or equal F remarkable characteristic.

Fig. 6 means the process flow diagram of detailed step of the processing of steps A 24.Whether the number of first, judging local feature point is as more than the value of predefined constant F (steps A 241).That is whether, judge and in the processing of above-mentioned steps A 234, stored the number of the pixel data of coordinate figure, be more than the value of constant F.The value of constant F is for example 1000～10000.

If the result of determination of steps A 241 is more than the value of constant F, from more than F local feature point, according to phase angle amount order from big to small, extract F local feature point of larger phase angle amount, the local feature point extracting is as remarkable characteristic, and their coordinate figure is stored in (steps A 242) in storage part 12.Now the coordinate figure of storage is coordinate figure in the coordinate of the affiliated standard of each local feature point and at the coordinate (X of the 0th standard ₀y ₀coordinate) coordinate figure in.Wherein, 2 above local feature points that equate for phase angle amount, for example, preferentially extract the also less unique point of value value less, wherein i of the j in the processing of steps A 23.In addition, for form respectively various criterion view data, X ₀y ₀the equal local feature point of coordinate figure in coordinate, extracts 1 local feature point in the middle of their, for example extracts the local feature point of the value minimum of the j in the processing of steps A 23.

If the result of determination of steps A 241 is below the value of constant F, in steps A 23, detected all local unique points are remarkable characteristics, the coordinate figure in the coordinate of the standard under these local feature points and the coordinate (X of the 0th standard ₀y ₀coordinate) coordinate figure in is stored in (steps A 243) in storage part 12.But, for form respectively various criterion view data, X ₀y ₀the equal local feature point of coordinate figure in coordinate, only stores 1 local feature point in the middle of their, the coordinate figure of the local feature point of the value minimum of the j in a storing step A23 for example.

Processing based on steps A 242 or steps A 243 finishes, the processing of end step A24.Like this, in the processing of steps A 24, can detect remarkable characteristic.

As shown in Figure 2, after the processing of steps A 24, carry out remarkable characteristic to calculate the processing of the steps A 25 of Descriptor vector.The processing of steps A 25 is carried out respectively for each remarkable characteristic.

Fig. 7 means the process flow diagram of detailed step of the processing of steps A 25.The processing of steps A 25 comprises: calculate the processing (steps A 251) of mastery orientation, the processing (steps A 252) that detects the pixel data on the concentrically ringed circumference centered by remarkable characteristic, the processing (steps A 253) of calculating weight residual quantity value, the processing (steps A 254) that the generation number of degrees distribute and the processing (steps A 255) of calculating Descriptor vector.

First, the processing of steps A 251 is described.The processing of steps A 251 is according near the brightness value of the pixel data of the brightness value of remarkable characteristic and this remarkable characteristic, calculates the processing of the mastery orientation that the angle of the gradient of the brightness value of this remarkable characteristic vicinity is represented.

Fig. 8 means the process flow diagram of detailed step of the processing of steps A 251.First, for each pixel data on circle in the view data of the z standard under remarkable characteristic, centered by this remarkable characteristic, calculate respectively the angle (orientation) (steps A 2511) of the gradient of brightness value.This radius of a circle is for example 3 pixels.

Now, if by remarkable characteristic at X _zy _zcoordinate figure in coordinate is made as (u _z, v _z),, in the view data of z standard, there is the X that meets following formula (4) _zy _zcoordinate figure (α in coordinate _z, β _z) pixel data, be the pixel data on this circle, there is the X that meets following formula (5) _zy _zcoordinate figure (α in coordinate _z, β _z) pixel data, be the pixel data on the circumference of this circle.

[numerical expression 5]

$\sqrt{{(u_z-{\mathrm{\&alpha;}}_z)}^2+{(v_z-{\mathrm{\&beta;}}_z)}^2}<3.5...\left(4\right)$

[numerical expression 6]

$2.5\&le;\sqrt{{(u_z-{\mathrm{\&alpha;}}_z)}^2+{(v_z-{\mathrm{\&beta;}}_z)}^2}<3.5...\left(5\right)$

For the pixel data on such circle, gradient [numerical expression 7] by Sobel operational symbol, according to following formula (6), calculate, as the orientation of the angle of gradient, according to following formula (7), calculate.It should be noted that, in the present invention, all orientations that calculate according to following formula (7) are all 0 ° and are less than above 360 °.

[numerical expression 8]

$S_{u,v}^{\mathrm{Xz}}=I_z(u_z+1,v_z-1)$

$+2\&times;I_z(u_z+1,v_z)+I_z(u_z+1,v_z+1)$

$-I_z(u_z-1,v_z-1)-2\&times;I_z(u_z-1,v_z)$

$-I_z(u_z-1,v_z+1)$

$S_{u,v}^{\mathrm{Yz}}=I_z(u_z-1,v_z-1)$

$+2\&times;I_z(u_z,v_z-1)+I_z(u_z+1,v_z-1)$

$-I_z(u_z-1,v_z+1)-2\&times;I_z(u_z,v_z+1)$

$-I_z(u_z+1,v_z+1)...\left(6\right)$

[numerical expression 9]

$A_{u,v}^z={\tan }^{-1}(S_{u,v}^{\mathrm{Xz}}/S_{u,v}^{\mathrm{Yz}})...\left(7\right)$

Then, calculate mean value, i.e. the 1st average orientation Ao (steps A 2512) of orientation of 8 pixel datas of the surrounding of remarkable characteristic and this remarkable characteristic.The 1st average orientation Ao calculates according to following formula (8).

[numerical expression 10]

$A_o=\frac{1}{9}(A_{u,v}^z+A_{u-1,v-1}^z+A_{u,v-1}^z+A_{\mathrm{<figure-callout\; id="1"\; label="u\; +"\; filenames="HSA00000226035600011.png"\; state="\{\{state\}\}">u</figure-callout>}+1,v+1}^z$

$+A_{u-1,v}^z+A_{\mathrm{<figure-callout\; id="1"\; label="u\; +"\; filenames="HSA00000226035600011.png"\; state="\{\{state\}\}">u</figure-callout>}+1,v}^z+A_{u-1,v+1}^z+A_{u,v+1}^z{+A}_{\mathrm{<figure-callout\; id="1"\; label="u\; +"\; filenames="HSA00000226035600011.png"\; state="\{\{state\}\}">u</figure-callout>}+1,v+1}^z)...\left(8\right)$

Then, calculate the orientation [numerical expression 11] of all pixel datas in the round region that calculates gradient in steps A 2511 mean value, i.e. the 2nd average orientation Ac (steps A 2513).Then, judge whether the 1st average orientation Ao equates (steps A 2514) with the 2nd average orientation Ac.If the result of determination of steps A 2514 is that the value of the two is equal, the 1st average orientation Ao is stored in (steps A 2515) in storage part 12 as mastery orientation Ad.

If the result of determination of steps A 2514 is that the value of the two is unequal, judge that whether the 1st average orientation Ao is than the 2nd average orientation Ac large (steps A 2516).If the result of determination of steps A 2516 is that the 1st average orientation Ao is large, the value larger than the 2nd average orientation Ac is stored in (steps A 2517) in storage part 12 as mastery orientation Ad.More specifically, calculate in the pixel data in the round region of gradient, have the mean value of orientation of the pixel data of the orientation larger than the 2nd average orientation Ac in steps A 2511, as mastery orientation, Ad is stored in storage part 12.

If the result of determination of steps A 2516 is that the 1st average orientation Ao is little, the value less than the 2nd average orientation Ac is stored in (steps A 2518) in storage part 12 as mastery orientation Ad.More specifically, calculate in the pixel data in the round region of gradient, have the mean value of orientation of the pixel data of the orientation less than the 2nd average orientation Ac in steps A 2511, as mastery orientation, Ad is stored in storage part 12.

Processing based on steps A 2515, steps A 2517 or steps A 2518 finishes, the processing of the steps A that is through with 251.Like this, in the processing of steps A 251, can calculate mastery orientation Ad for each remarkable characteristic.

As shown in Figure 7, after the processing of steps A 251, carry out the processing of steps A 252 that the pixel data on concentrically ringed circumference in the view data of the z standard under remarkable characteristic, centered by this remarkable characteristic is detected.In the present embodiment, for radius, be 8 circles of 3 pixel～10 pixels, store the coordinate figure of each pixel data on circumference separately into storage part 12 respectively.In addition, radius of a circle and number are not limited to this.

Then, calculate each pixel data on the circumference of above-mentioned 8 circles orientation, with the residual quantity value θ of the mastery gradient calculating in steps A 251, by multiplying each other between the square root of the radius of a circle under this residual quantity value and each pixel data, for each pixel data, calculate respectively weight residual quantity value D (steps A 253).The orientation of each pixel data on circumference is calculated according to above-mentioned formula (6), (7).

For example, when the number that is the pixel data on the circumference of circle of 3 pixels at radius is 10, calculate respectively 10 weight residual quantity values

[numerical expression 12]

then, for radius, be other circles of 4 pixel～10 pixels, calculate similarly weight residual quantity value D.Wherein the numerical range of weight residual quantity value D is more than 0 to be less than 360, the in the situation that of beyond this scope, adds or deduct 360.The weight residual quantity value D calculating is stored in storage part 12.

Then, about the weight residual quantity value calculating, for 8 circles, generate respectively to have more than 0 to be less than more than 45,45 to be less than more than 90,90 to be less than more than 135,135 to be less than more than 180,180 to be less than more than 225,225 to be less than more than 270,270 to be less than and more than 315 and 315 be less than other number of degrees of 360 these 8 levels distribute (steps A 254).Therefore, generate 8 and there is other number of degrees distribution of 8 levels.

The square root of radius of a circle can increase the distance between different residual quantity values as weight, thereby make to obtain number of degrees distribution, becomes and disperses, and reduces number of degrees distribution and concentrates on an interval, improves the uniqueness of Descriptor vector.

Then, according to 8 number of degrees that generate, distribute, calculate Descriptor vector (steps A 255).More specifically, for 8 circles number of degrees separately, distribute, the number of degrees at different levels that generation is usingd in the number of degrees distribution generating are as 8 dimensional vectors of composition.Then, 88 dimensional vectors are unified into 1, generate the vector of 8 * 8=64 dimension.Finally, by this 64 dimensional vector divided by 64 each compositions square root sum square, calculate size and be 1 Descriptor vector.

Descriptor vector calculates by each remarkable characteristic, and with each remarkable characteristic at X ₀y ₀coordinate figure in coordinate is stored in storage part 12 together.Thus, the processing of steps A 25 finishes, and the Descriptor vector computing shown in Fig. 2 all finishes.

Like this, by the Descriptor vector computing shown in Fig. 2, can calculate distinctive point (unique point) in view data at X ₀y ₀coordinate figure in coordinate and for the Descriptor vector of this unique point.In the present invention, in the detection due to unique point, used Harris's matrix, so even the weak view data of contrast, the Descriptor vector of each unique point also becomes the vector that independence is high.Therefore, for example, when detecting consistent unique point in 2 view data, can reduce the possibility of error detection.

And, in the present invention, due to according to the orientation on the concentrically ringed circumference centered by unique point, with the residual quantity value of the mastery orientation of unique point vicinity, calculate Descriptor vector, so this Descriptor vector becomes, be difficult to be rotated and the characteristic quantity of the impact of ratiometric conversion.

Descriptor vector becomes the characteristic quantity of the impact that is not vulnerable to rotation and ratiometric conversion, mainly because circle self has good unchangeability to rotation, and orientation and the mastery orientation of pixel has synchronism to rotation on circumference, its residual quantity value remains unchanged to rotation.For example, a circumference has rotated 10 degree, and on mastery orientation and this circumference, the orientation of pixel also can have been rotated 10 degree so, although their orientation angles has converted, the difference between them still remains unchanged.

And, in the present embodiment, owing to utilizing, as the image pyramid of image data set, carry out the detection of unique point, so can calculate the Descriptor vector that independence is higher.In addition, owing to passing through to use the 1st average orientation Ao and the 2nd average orientation Ac, can obtain the mastery orientation Ad of unique point vicinity, on the circumference of utilization centered by unique point, the orientation of pixel is orientated this feature of Ad rotational invariance with respect to its mastery, thereby calculates according to the fixedly difference of the two Descriptor vector that independence is higher.And, can distribute based on thering are other 8 number of degrees of 8 levels, calculate the 64 dimension Descriptor vectors that independence is higher.

Characteristic quantity calculation element 1 is also as utilizing the Descriptor vector calculating as described above, the image coupling arrangement performance function of carrying out the connection of a plurality of view data.; by processing according to the characteristic quantity calculation procedure of storage in storage part 12; control algorithm portion 11 and storage part 12, as utilizing Descriptor vector, synthesize 2 above view data connections the image connecting portion performance function of the processing (image connects processing) of 1 view data.

Below, the image of characteristic quantity calculation element 1 is connected to processing and describe.Fig. 9 means that image connects the process flow diagram of processing.First, from inputs such as digital camera, scanners, become more than 2 view data (step B1) of the object of image connection processing.

Then, each view data being transfused to is described to symbol vector calculation and processes (step B2).The processing of step B2 is the processing same with the Descriptor vector computing of the steps A 21～steps A 25 shown in Fig. 2.

Then, between a plurality of view data, carry out the retrieval (step B3) of match point.Match point refers to that the distance between Descriptor vector is less than the unique point pair of predefined threshold value.That is, when each composition of the subsidiary Descriptor vector of unique point, with each difference that becomes to divide of the Descriptor vector of other unique points square root sum square while being less than predefined threshold value, these unique points are match points.Coordinate figure (the X of the match point in each view data ₀y ₀coordinate figure) be stored in storage part 12.

Whether the number of then, judging match point is 2 groups above (step B4).In the situation that the result of determination of step B4 is not more than 2 groups, on display part 14, show the image (step B6) that the connection of presentation video data cannot realize.

In the situation that the number that the result of determination of step B4 is match point is more than 2 groups, carries out the connection of view data and process (step B5).The connection of view data is processed according to X ₀y ₀coordinate figure carries out.

Below, to 2 groups of match point (w detected in 2 view data L, L ' ₁, w ₁'), (w ₂, w ₂') time the connection of view data process and to represent.Wherein, establish unique point w ₁, w ₂the pixel data in view data L, unique point w ₁', w ₂' be the pixel data in view data L '.And, at X ₀y ₀in coordinate, establish each unique point w ₁, w ₂, w ₁', w ₂' coordinate figure be respectively (g ₁, r ₁), (g ₂, r ₂), (g ₁', r ₁'), (g ₂', r ₂').

According to these each coordinate figures, calculate distance between the unique point in each view data L, L ', be coordinate figure difference square root sum square.2 unique point (w in view data L ₁, w ₂) between distance E be

[numerical expression 13]

2 unique point (w in view data L ' ₁', w ₂') between distance E ' be

[numerical expression 14]

then, according to apart from E with apart from E ', calculate ratio of distances constant E '/E.

And, according to each coordinate figure, calculate the angle of inclination between the unique point in each view data L, L '.2 unique point (w in view data L ₁, w ₂) between an angle of inclination δ be

[numerical expression 15]

Tan ^-1((r ₁-r ₂)/(g ₁-g ₂)), 2 unique point (w in view data L ' ₁', w ₂') between angle of inclination δ ' be

[numerical expression 16]

tan ^-1((r’ ₁-r’ ₂)/(g’ ₁-g’ ₂))。Then, according to δHe angle of inclination, angle of inclination δ ', calculate difference the δ '-δ at angle of inclination.

Subsequently, for view data L, with ratio of distances constant the E '/E calculating, wait doubly and to process, and be rotated processing with the value of difference the δ '-δ at the angle of inclination that calculates.View data L after these processing is according to unique point w ₁be overlapped in unique point w ₁', unique point w ₂be overlapped in unique point w ₂' mode, L ' is connected with view data.Then as required, to repairing processing by connecting the view data obtaining, synthetic 1 view data.

In addition, the in the situation that of detecting more than 3 groups match point in 2 view data, utilize the mean value of the mean value of a plurality of ratio of distances constants in each view data and the difference at a plurality of angle of inclination, carry out the connection of view data.For example, in the situation that the number of match point is 3 groups, ratio of distances constant is calculated 3, and the difference at angle of inclination is also calculated 3.Therefore, in this situation, with the mean value of 3 ratio of distances constants, wait doubly and process, with the mean value of the difference at 3 angles of inclination, be rotated processing.

In the processing of step B5, the image of the connection based on by view data as described above being processed to the view data obtaining is shown in display part 14.Processing based on step B5 or step B6 finishes, and the image that finishes characteristic quantity calculation element 1 connects to be processed.

Below represent to connect by the image of characteristic quantity calculation element 1 concrete example of processing the view data obtaining.Figure 10 has represented subject 100, and Figure 11 has represented 4 view data 200A, 200B, 200C, 200D.4 view data 200A, 200B, 200C, 200D are respectively the view data by utilizing digital camera that subject 100 is photographed and obtained.Figure 12 has represented the view data 300A after connection.View data 300A is being rotated processing by characteristic quantity calculation element 1 couple of view data 200A, 200D, on the basis of view data 200C being carried out etc. doubly processing, and these view data 200A, 200C, 200D is connected with view data 200B and the view data that obtains.Figure 13 has represented the view data 300B after finishing is processed.View data 300B repairs by 1 couple of view data 300A of characteristic quantity calculation element the view data of processing and obtaining.

As shown in figure 11, the image based on view data tilts mutually, ratio is mutually different.But, because characteristic quantity calculation element 1 utilizes Descriptor vector, carry out the connection of view data, thus as shown in figure 12 can high precision connect to tilt and image that ratio is different each other.In addition, characteristic quantity calculation element 1 also can be configured to, and substitutes the view data 300B shown in Figure 13, and by the image of the view data 300A based on shown in Figure 12 before finishing processing, is shown to display part 14.

And, the Descriptor vector that characteristic quantity calculation element 1 also calculates as utilization, the image retrieving apparatus performance function of carrying out the retrieval of other view data from a view data.; by processing according to the characteristic quantity calculation procedure of storage in storage part 12; control algorithm portion 11 and storage part 12, as utilizing Descriptor vector, are retrieved the image retrieval portion performance function of the processing (image retrieval processing) of other view data from a view data.

Below, the image retrieval of characteristic quantity calculation element 1 is processed and described.Figure 14 means the process flow diagram that image retrieval is processed.First, from inputs such as digital camera, scanners, become more than 2 view data (step C1) of the object of image retrieval processing.

Then, for each view data being transfused to, be described symbol vector calculation and process (step C2).The processing of step C2 is the processing same with the Descriptor vector computing of the steps A 21～steps A 25 shown in Fig. 2.

Then, between a plurality of view data, carry out the retrieval (step C3) of match point.Match point refers to that the distance between Descriptor vector is less than the unique point pair of predefined threshold value.That is, when each difference that becomes to divide of each composition of the subsidiary Descriptor vector of unique point, the Descriptor vector subsidiary with other unique points square root sum square, for the value that predetermines is when following, these unique points are match points.Coordinate figure (the X of the match point in each view data ₀y ₀coordinate figure) be stored in storage part 12.

Then, the number of judgement match point is to be mostly few (step C4).The number of match point is that the judgement that is mostly few can be carried out according to the absolute quantity of match point, also can carry out according to the relative populations of the number of relative characteristic point.For example, in the situation that wanting to retrieve searching object view data Q ' from the object image data Q that is retrieved, when the number of the match point that the number of the unique point in searching object view data Q ' is 50, detect in step C3 is 45, due to the unique point that can detect from the object image data Q that is retrieved with 9 one-tenth Feature Points Matching in searching object view data Q ', so it is many to be judged to be the number of match point.

In the situation that the result of determination of step C4 is the number of match point is few, by representing, do not find that the image of searching object view data Q ' is shown in display part 14 (step C6).

In the situation that the result of determination of step C4 is the number of match point is many, the image of the view data of the part based on suitable with searching object view data Q ' in the object image data Q that is retrieved, is shown in display part 14 (step C5).The part suitable with searching object view data Q ' is in the object image data Q that is retrieved, and comprises the part by the detected all match points of step C3.Processing based on step C5 or step C6 finishes, and finishes the image retrieval of characteristic quantity calculation element 1 and processes.

Like this, because characteristic quantity calculation element 1 utilizes Descriptor vector, from the object image data that is retrieved, carry out the retrieval of searching object view data, so can retrieve accurately.

The present invention can, in the situation that not departing from its purport or principal character, implement by other variety of ways.Therefore, the institute in above-mentioned embodiment is simple illustration a little only, and scope of the present invention, by the Range Representation of claim, is not subject to any restriction of instructions text.And, belong to the scope of claim distortion, change also all within the scope of the present invention.

Claims (7)

1. a characteristic quantity calculation element, is characterized in that, possesses:

Phase angle amount calculating part, a plurality of pixel datas of the view data that it is transfused to for formation, calculate respectively Harris's matrix, utilize Harris's matrix calculating to calculate respectively phase angle amount;

Feature point detecting unit, it detects in the pixel data that forms described view data, has the pixel data of the phase angle amount larger than the phase angle amount of adjacent pixel data around, is used as unique point;

Mastery orientation calculation portion, it calculates according to the pixel value of the pixel data of the pixel value of described unique point and this feature neighborhood of a point the mastery orientation that the angle of the gradient of the pixel value of this unique point vicinity is represented;

Concentric circles detection portion, it detects in described view data, the pixel data on the circumference of p the circle that radius centered by described unique point is different, wherein p >=2;

Weight residual quantity value calculating part, it is for p circle, calculate respectively to the angle of the gradient of the pixel value of each pixel data on circumference, with the residual quantity value of described mastery gradient, be multiplied by the square root of radius of a circle and the weight residual quantity value that obtains;

Number of degrees distribution generating unit, it generates respectively for p circle q other number of degrees of level that have about described weight residual quantity value and distributes, wherein q >=2; With

Descriptor vector calculating part, it calculates respectively and usings each number of degrees as the q dimensional vector of composition for p circle, and according to each q dimensional vector calculating, calculates p * q and tie up Descriptor vector.

2. characteristic quantity calculation element according to claim 1, is characterized in that,

Possesses image data set generating unit, its generation comprises a view data and by the image data set of other view data of dwindling after this view data smoothing and obtaining, replace described view data, the image data set of generation is inputed to described phase angle amount calculating part.

3. characteristic quantity calculation element according to claim 1 and 2, is characterized in that,

Described mastery orientation calculation portion calculates: the angle mean value Ao of the gradient of the pixel value of 8 pixel datas of described unique point and this feature neighborhood of a point; The angle mean value Ac of the gradient of the pixel value of the pixel data of the radius of take centered by this unique point in the circle of 3 pixels; If Ac is greater than Ao, calculate the mean value of angle of gradient that the radius take centered by this unique point Grad in the circle of 3 pixels is less than the pixel data of Ac, this mean value is orientated as described mastery; If Ac is less than Ao, calculate the mean value of angle of gradient that the radius take centered by this unique point Grad in the circle of 3 pixels is greater than the pixel data of Ac, this mean value is orientated as described mastery; If Ac equals Ao, the mean value Ao of usining is orientated as described mastery.

4. characteristic quantity calculation element according to claim 1, is characterized in that,

Described Concentric circles detection portion is set as 8 by the value of p,

Described number of degrees distribution generating unit is set as 8 by the value of q.

5. an image coupling arrangement, is characterized in that, possesses:

Characteristic quantity calculation element claimed in claim 1; With

Utilize described Descriptor vector that 2 above view data are connected, synthesize the image connecting portion of 1 view data.

6. an image retrieving apparatus, is characterized in that, possesses:

Characteristic quantity calculation element claimed in claim 1; With

Utilize the image retrieval portion of retrieving other view data in the view data of described Descriptor vector from the view data being transfused to described in a plurality of.

7. a characteristic quantity calculating method, is characterized in that, comprising:

To forming a plurality of pixel datas of the view data being transfused to, calculate respectively Harris's matrix, utilize Harris's matrix calculating to calculate respectively the step of phase angle amount;

Detect in the pixel data that forms described view data, there is the pixel data of the phase angle amount larger than the phase angle amount of adjacent pixel data around, as the step of unique point;

According to the pixel value of the pixel data of the pixel value of described unique point and this feature neighborhood of a point, calculate the step of the mastery orientation that the angle of the gradient of the pixel value of this unique point vicinity is represented;

Detect in described view data the step of the pixel data on the circumference of p the circle that radius centered by described unique point is different, wherein p >=2;

For p circle, calculate respectively to the angle of the gradient of the pixel value of each pixel data on circumference, with the residual quantity value of described mastery gradient, be multiplied by the square root of radius of a circle and the step of the weight residual quantity value that obtains;

For p circle, generate respectively q the step that other number of degrees of level distribute, wherein q >=2 having about described weight residual quantity value; With

For p circle, calculate respectively and using the number of degrees as the q dimensional vector of key element, and according to each q dimensional vector calculating, calculate the step of p * q dimension Descriptor vector.

Images