CN101866496A - Augmented reality method based on concentric ring pattern group - Google Patents

Abstract

The invention discloses an augmented reality method based on a concentric ring pattern group, mainly solving problems of weak anti-interference ability and inaccurate matching in the existing method. The steps of the method are that: firstly, make a background plane containing the concentric ring pattern group; secondly, solve projection of the center of the concentric ring and radius ratio of an inner circle and an outer circle, and recognize the corresponding relation of all circular centers on the spatial plane and the image plane by utilizing the uniqueness of the radius ratio combination; then, solve homography transformation matrix from the spatial plane and the image plane by utilizing the intersection points which are formed by the connecting line of circular centers, and a straight line obtained by the connecting line of the circular centers and the concentric ring pattern on the spatial plane and the image plane, so as to obtain parameters of a vidicon; finally, overlay a virtual scene to a real scene by utilizing the parameters of the vidicon; the augmented reality method has the advantages of strong antijamming capability and accurate matching and is applied to the field for realizing reality augment such as a virtual studio, teaching, scientific research, business and entertainment and the like.

Description

Augmented reality method based on the concentric ring pattern group

Technical field

The invention belongs to technical field of information processing, relate to a kind of method, can be used for the field that needs such as virtual studio and teaching, scientific research, commerce and amusement are realized augmented realities based on concentric ring pattern group realization augmented reality.

Technical background

Augmented reality is meant that the scene of in video image virtual object and reality being taken merges, and reaches visual perception's effect that reality and dummy object can't be distinguished.Its step mainly comprises: 1) take out from the image that reality is taken by scratching as the object with needs for every width of cloth image; 2) inside and outside parameter of calculating video camera makes virtual three-dimensional scenic form required virtual picture with correct projection relation; 3) object that plucks out is combined the effect that forms the actual situation combination with virtual picture.Wherein the detection of the inside and outside parameter of video camera is the difficult point of augmented reality technology.

At present, the inside and outside parameter detection method of video camera has: sensor-based detection method and based on the detection method of vision technique.

One. sensor-based detection method mainly contains:

1) use disk light beam encoder to measure the inside and outside parameter of camera, the method needs on the support of video camera and camera lens disk light beam encoder to be installed, and it costs an arm and a leg, the scaling method complexity;

2) use stepper motor driven mechanical hook-up auxiliary detection camera interior and exterior parameter, this device need use the motion of step motor drive video camera, photographer need operate through after the technical skill training, the operating process inconvenience, and also the hysterisis error of mechanical motion also needs actively to overcome;

3) infrared telemetry technology for detection camera interior and exterior parameter, this method wears infrared transmitting device for mobile object (host etc.), its infrared ray is absorbed by infrared camera, the degree of depth by the computer Recognition mobile object, its price is lower, but poor anti jamming capability all has requirement to dressing and mode of motion.

Two. based on the detection method of vision technique

This method is that special figure or sign are placed on background, detects these figures or sign in taken image, according to the world coordinates of figure or sign and the corresponding relation of image coordinate, calculates the inside and outside parameter of video camera.This method need not to change photographer's working method, and is easy to operate, and cost is low.

The early stage three-dimensional calibration thing that utilizes is because the difficulty on making is replaced by plane reference object now.Present widely used plane reference object is a chequer, but there is following shortcoming in this method: can not block before the grid plane, block and to cause some grid angle points invisible, thereby can't judge the corresponding relation of the angle point of real world to plane of delineation angle point; Grid plane in the image must be complete, if having only the part plane to be taken, equally also can cause part grid angle point invisible.

Another kind of effective method occurred for overcoming this shortcoming, utilized the plane reference object of inhomogeneous grid form, its unevenness is obtained by the Markov chain principle design, thereby to each angle point coding.This method needs two-dimentional edge detection algorithm test pattern edge to come grid angle point in the computed image, correctly mates the point in the world coordinate system and the corresponding relation of the point in the image with the unevenness of grid again.The deficiency of these class methods is: because chequer is a quadrilateral, usually obtain by four summits, and on the image arbitrarily four points can think the projection of chequer, therefore in prospect, for example, the also flase drop projection of measuring grid is most probably obscured with background in people's the clothes, this just can carry out the detection of chequer after just foreground area need being removed through chroma key technique, and the efficient of detection is low.

In order to overcome the shortcoming with the plane reference object detection camera inside and outside parameter of various grid forms, the method with donut detection camera inside and outside parameter has appearred.Circle is generally ellipse in the projection of the plane of delineation, and an ellipse needs 5 points to determine at least, so its antijamming capability is better than straight line.Document " Jun-Sik Kim and In SoKweon, Geometric and Algebraic Constraints of Projected Concentric Circlesand Their Applications to Camera Calibration, IEEE Trans.on Pattern Analysisand Machine Intelligence, 27 (4), pp.637-642,2005 " obtain the camera intrinsic parameter with donut, but owing to do not consider to distinguish concentric ring pattern with concentric ring pattern inside and outside circle radius ratio, this method can only be found the solution the camera intrinsic parameter, and calculation of complex, the line at infinity that needs first estimated image plane could calculate the position in the center of circle then.At document " Guang Jiang and Long Quan, Detection of concentriccircles for camera calibration, Proc.of IEEE Int.Conf.on Computer Vision, pp.333-340,2005 " search for the position in the center of circle in convergence method, determine optimum solution with nonlinear optimization, calculate still complicated; Determined the outer parameter of camera though used 12 donuts with different inside and outside circle radius ratios, but because the number of the different radii ratio that uses is more, difference between the radius ratio numerical value is just very little, thus the measuring error of being easy to generate, thus cause the mistake coupling.What above-mentioned in addition two documents adopted all is black, can't finish the stingy picture of video and handle.

Summary of the invention

The objective of the invention is to overcome the deficiency of above-mentioned prior art, propose a kind of augmented reality method based on the concentric ring pattern group, to strengthen antijamming capability, shortcut calculation improves matching accuracy.

For achieving the above object, the method based on concentric ring pattern group realization augmented reality provided by the invention comprises the steps:

(1) draw the background plane that contains the concentric ring pattern group:

1a) determine the number n of the donut that will draw on the background plane according to background plane size and target sizes, n 〉=2;

1b) background plane evenly is divided into n rectangular region, if the length of rectangular region is x, the wide y of being, with the center of circle, the center of each rectangular region, and on background plane, set up rectangular coordinate system, write down the coordinate of this n center of circle under this coordinate system as each donut;

1c) according to step 1b) in the length of rectangular region and the exradius R of wide definite n donut _m(m=1,2 ... n), 0＜R _m＜x/2,0＜R _m＜y/2;

1d) select k different greater than 0 and less than 1 numerical value as n donut inside and outside circle radius ratio ρ separately _m(m=1,2 ... k), k≤n is with this k different ρ _mDistribute to n donut, if any two adjacent donut D on the background plane _iAnd D _jRadius ratio combination be [ρ _i, ρ _j], then the combination of two of other position adjacent donut radius ratios can not with its repetition, to guarantee that promptly the combination of any two adjacent donut inside and outside circle radius ratios has nothing in common with each other;

1e) on space plane, set up the combination [ρ of each neighboring concentric annular radii ratio _i, ρ _j] with the combination [D of its corresponding concentric annulus _i, D _j] corresponding relation;

1f) according to the inside and outside circle radius ratio ρ of n donut _mWith exradius R _m(m=1,2 ... n) determine the interior radius of circle r of n donut _m(m=1,2 ... n): r _m=ρ _mR _m(m=1,2 ... n);

1g) according to step 1b) in the center of circle, the step 1c that determine) in the exradius and the step 1f that determine) in the interior radius of circle determined, on background plane, draw out a corresponding n donut, and other zone of the circle ring area of donut and background plane is identical with tone, the color differentiating that color saturation is different;

(2) background plane that will contain the concentric ring pattern group places the people that will take or the back of thing, takes the image that contains concentric ring pattern group background plane;

(3) on captured image, select two adjacent elliptical ring d that are not blocked _iAnd d _j, d _iAnd d _jBe the donut elliptical ring that projection forms on the plane of delineation of space plane, on the plane of delineation according to d _iAnd d _jThe edge, find the solution the o of center of circle projection coordinate of its corresponding donut _iAnd o _j

(4) according to o _iAnd o _jAnd the transformation relation of donut inside and outside circle, seek d _iAnd d _jRadius corresponding compares ρ _iAnd ρ _j

(5) according to ρ _iAnd ρ _jCombination [the ρ that forms _i, ρ _j], determine d _iAnd d _jCorresponding space plane donut D _iAnd D _j

(6) connect o _iAnd o _jForm straight line l _Ij, according to l _IjSolve two straight lines

With Connect D _iAnd D _jThe center of circle form straight line L _Ij, cross D _iAnd D _jThe center of circle do and L _IjTwo vertical straight lines With

L then _Ij,

With d _i, d _jOn the plane of delineation, form 16 intersection points, L _Ij,

With D _i, D _jOn space plane, also form 16 intersection points; According to the coordinate of corresponding intersection point, find the solution the homography transformation matrix of space plane to the plane of delineation at the plane of delineation and space plane;

(7) find the solution camera parameters according to the homography transformation matrix;

(8) according to camera parameters virtual three-dimensional body is added in the real scene.

The present invention compared with prior art has following advantage:

1) the present invention is in drawing concentric ring pattern group process, because the combination of the radius ratio of neighboring concentric annulus inside and outside circle has nothing in common with each other arbitrarily, and then determine the donut position according to the combination of neighboring concentric annular radii ratio, so do not need to determine its position according to the radius ratio of single donut;

2) the present invention because a part of donut uses identical radius ratio, so the number of the different radii ratio that uses reduces, makes radius ratio numerical value differ bigger in drawing concentric ring pattern group process, is easier to differentiate, and measuring error is littler;

3) the present invention is succinct and reliable to the calculating of center of circle projection;

4) finding the solution simply of the internal exradius ratio of the present invention, it is more accurate to calculate;

5) relatively based on the visible detection method of straight line, the donut antijamming capability that the present invention uses is stronger.

Description of drawings

Fig. 1 is a process flow diagram of the present invention;

Fig. 2 is that the present invention draws the background plane sub-process figure that contains the concentric ring pattern group;

Fig. 3 is the concentric ring pattern group synoptic diagram of drawing with the inventive method;

Fig. 4 is that each donut radius ratio of the present invention is distributed synoptic diagram;

Fig. 5 is the synoptic diagram of video camera photographic images of the present invention.

Embodiment

With reference to Fig. 1, the present invention includes following steps:

Step 1 is drawn the background plane that contains the concentric ring pattern group.

With reference to Fig. 2, being implemented as follows of this step:

1a) select the donut number n according to background plane size and target sizes, require n 〉=2, and satisfy and have at least two donuts not to be blocked when target is in the background front, present embodiment is got the long 200cm of being of background plane, wide is 150cm, chooses donut number n=12;

1b) background plane is divided into triplex row four row totally 12 rectangular region, the long x=50cm of rectangular region, wide y=50cm, with the center of circle, the center of each rectangular region as each donut, and on background plane, set up rectangular coordinate system, write down this 12 coordinates of the center of circle under this coordinate system;

1c) according to step 1b) in the long x of rectangular region and the exradius R that wide y determines each donut _m, require 0＜R _m＜x/2,0＜R _m＜y/2, the exradius of 12 donuts of this example all is chosen for R _m=23cm, m=1,2 ... 12;

1d) select k greater than 0 and less than 1 number as donut inside and outside circle radius ratio ρ separately _m, requiring k≤n, this example is chosen k=7, and this 7 number is respectively 0.1,0.2,0.3,0.4,0.5,0.6 and 0.7, with these 7 different ρ _mDistribute to 12 donuts, the combination that guarantees any two adjacent donut inside and outside circle radius ratios has nothing in common with each other, Fig. 4 shows the radius ratio numerical value that each donut distributes, promptly the radius ratio of four donuts of first row is 0.1,0.2,0.3 and 0.4 successively, the radius ratio of four donuts of second row is 0.4,0.2,0.5 and 0.4 successively, and the radius ratio of four donuts of the third line is 0.6,0.6,0.1 and 0.7 successively;

1e) on space plane, set up the combination [ρ of each neighboring concentric annular radii ratio _i, ρ _j] with the combination [D of its corresponding concentric annulus _i, D _j] corresponding relation, for example, first row first donut D ₁Radius ratio be second donut D of 0.1, the first row ₂Radius ratio be 0.2;

1f) according to the radius ratio ρ of 12 donuts _mWith exradius R _mDetermine interior radius of circle r separately _m, r _m=ρ _mR _mFor example the interior radius of circle of four donuts of first row is 2.3cm, 4.6cm, 6.9cm and 9.2cm successively, the interior radius of circle of four donuts of second row is 9.2cm, 4.6cm, 11.5cm and 9.2cm successively, and the interior radius of circle of four donuts of the third line is 13.8cm, 13.8cm, 2.3cm and 16.1cm successively;

1g) according to step 1b) in the center of circle and the step 1c that determine) in the exradius and the step 1f that determine) in the interior radius of circle determined, on background plane, draw out corresponding 12 donuts, and other zone of the circle ring area of donut and background plane is identical with tone, the color differentiating that color saturation is different.

According to above-mentioned steps 1a) to 1g) background patterns drawn out is as shown in Figure 3.

Step 2, the background plane that will contain the concentric ring pattern group place the people that will take or the back of thing, take the image that contains concentric ring pattern group background plane, and the image result of its shooting as shown in Figure 5.

Step 3 is selected two adjacent elliptical ring d that are not blocked on captured image _iAnd d _j, on the plane of delineation according to d _iAnd d _jThe edge, find the solution the o of center of circle projection coordinate of its corresponding donut _iAnd o _j

3a) d _iAnd d _jThe donut elliptical ring that projection forms on the plane of delineation for space plane utilizes d _iAnd d _jIts edge is searched in the variation of tone saturation degree, obtains d _iAnd d _jAt the equation of the plane of delineation, with the d that obtains _iInside and outside oval matrix form be expressed as With

d _jInside and outside oval matrix form be expressed as

With

3b) basis

With

Obtain following formula:

$({\left(d_O^i\right)}^{-1}{d}_I^i-\mathrm{\λI})o_i=0,$ $({\left(d_O^j\right)}^{-1}{d}_I^j-\mathrm{\λI})o_j=0$

λ is a constant in the formula, and I is 3 * 3 unit matrix;

3c) obtain o according to following formula _iAnd o _j

Step 4 is according to o _iAnd o _jAnd the transformation relation of donut inside and outside circle, seek d _iAnd d _jRadius corresponding compares ρ _iAnd ρ _j

4a) donut has 7 different radius ratios in this example, uses ρ _mRepresent this 7 radius ratios, for elliptical ring d _i, 7 ρ _mAll may be its radius corresponding ratio, by 7 ρ _mObtain 7 transformation matrix G _m, m=1,2 ... 7:

$G_m=I+({\mathrm{\ρ}}_m-1)\frac{o_i{o_i}^T{d}_O^i}{{d_i}^T{d}_O^i{o}_i},(m=\mathrm{1,2},\·\·\·7)$

4b) cross o _iDo any ray, the intersection point of this ray and interior ellipse is

With the intersection point of outer ellipse be

With k G _mDifference substitution equation That G that this equation is set up _mPairing radius ratio is exactly d _iRadius corresponding compares ρ _i

4c) cross o _jDo any ray, the intersection point of this ray and interior ellipse is

With the intersection point of outer ellipse be With k G _mDifference substitution equation That G that this equation is set up _mPairing radius ratio is exactly d _jRadius corresponding compares ρ _j

Step 5 is according to step 1d) combination of the radius ratio of any as can be known two neighboring concentric annulus has nothing in common with each other, so according to d _iAnd d _jRadius ratio combination [ρ _i, ρ _j] determine d _iAnd d _jThe donut D of corresponding space plane _iAnd D _j, because step 1b) in write down the coordinate of the center of circle of each donut at space plane, so can determine D _iAnd D _jThe center of circle at the coordinate O of space plane _iAnd O _j

Step 6 is found the solution the homography transformation matrix of space plane to the plane of delineation.

6a) on the plane of delineation, according to o _iAnd o _jObtain straight line l _Ij:

l _ij＝o _i×o _j

Symbol " * " is represented vectorial multiplication cross in the formula;

6b) according to described And l _IjObtain limit b:

$b={\left(d_I^i\right)}^{-1}{l}_{\mathrm{ij}}={\left(d_O^i\right)}^{-1}{l}_{\mathrm{ij}}={\left(d_I^j\right)}^{-1}{l}_{\mathrm{ij}}={\left(d_O^j\right)}^{-1}{l}_{\mathrm{ij}},$

Respectively with b point and o _iAnd o _jLink to each other, obtain straight line

And straight line

6c) straight line l _Ij, With elliptical ring d _i, d _jOn the plane of delineation, form 16 intersection points, according to d _i, d _jInside and outside oval matrix

And straight line l _Ij,

Obtain the coordinate of 16 intersection points on the described plane of delineation;

6d) establish plane of delineation straight line l _Ij,

With

Pairing space plane straight line is respectively L _Ij,

With According to the knowledge of visual geometric as can be known, L _IjBe D _iAnd D _jThe line in the center of circle,

With

Be respectively D _iAnd D _jThe center of circle and and L _IjVertical straight line, straight line L _Ij,

With concentric circles D _i, D _jOn space plane, form 16 intersection points, according to D _iAnd D _jCentral coordinate of circle O _iAnd O _jAnd D _iAnd D _jThe inside and outside circle radius, obtain the coordinate of 16 intersection points on the described space plane;

6e) utilize the coordinate of 16 intersection points of the above-mentioned plane of delineation and the coordinate of 16 intersection points of space plane to find the solution the homography transformation matrix of space plane to the plane of delineation.

Step 7 is found the solution camera parameters according to the homography transformation matrix, and utilizes camera parameters that virtual three-dimensional body is added in the real scene.

More than describing only is an example of the present invention, does not constitute any limitation of the invention, and obviously anyone can choose different parameters according to thought of the present invention, makes corresponding transition, but these are all at the row of protection scope of the present invention.

Claims (5)

1. the augmented reality method based on the concentric ring pattern group comprises the steps:

(1) draw the background plane that contains the concentric ring pattern group:

1a) determine the number n of the donut that will draw on the background plane according to background plane size and target sizes, n 〉=2;

1b) background plane evenly is divided into n rectangular region, if the length of rectangular region is x, the wide y of being, with the center of circle, the center of each rectangular region, and on background plane, set up rectangular coordinate system, write down the coordinate of this n center of circle under this coordinate system as each donut;

1c) according to step 1b) in the length of rectangular region and the exradius R of wide definite n donut _m(m=1,2 ... n), 0＜R _m＜x/2,0＜R _m＜y/2;

1d) select k different greater than 0 and less than 1 numerical value as n donut inside and outside circle radius ratio ρ separately _m(m=1,2 ... k), k≤n is with this k different ρ _mDistribute to n donut, if any two adjacent donut D on the background plane _iAnd D _jRadius ratio combination be [ρ _i, ρ _j], then the combination of two of other position adjacent donut radius ratios can not with its repetition, to guarantee that promptly the combination of any two adjacent donut inside and outside circle radius ratios has nothing in common with each other;

1e) on space plane, set up the combination [ρ of each neighboring concentric annular radii ratio _i, ρ _j] with the combination [D of its corresponding concentric annulus _i, D _j] corresponding relation;

1f) according to the inside and outside circle radius ratio ρ of n donut _mWith exradius R _m(m=1,2 ... n) determine the interior radius of circle r of n donut _m(m=1,2 ... n): r _m=ρ _mR _m(m=1,2 ... n);

1g) according to step 1b) in the center of circle, the step 1c that determine) in the exradius and the step 1f that determine) in the interior radius of circle determined, on background plane, draw out a corresponding n donut, and other zone of the circle ring area of donut and background plane is identical with tone, the color differentiating that color saturation is different;

(2) background plane that will contain the concentric ring pattern group places the people that will take or the back of thing, takes the image that contains concentric ring pattern group background plane;

(3) on captured image, select two adjacent elliptical ring d that are not blocked _iAnd d _j, d _iAnd d _jBe the donut elliptical ring that projection forms on the plane of delineation of space plane, on the plane of delineation according to d _iAnd d _jThe edge, find the solution the o of center of circle projection coordinate of its corresponding donut _iAnd o _j

(4) according to o _iAnd o _jAnd the transformation relation of donut inside and outside circle, seek d _iAnd d _jRadius corresponding compares ρ _iAnd ρ _j

(5) according to ρ _iAnd ρ _jCombination [the ρ that forms _i, ρ _j], determine d _iAnd d _jCorresponding space plane donut D _iAnd D _j

(6) connect o _iAnd o _jForm straight line l _Ij, according to l _IjSolve two straight lines

With

Connect D _iAnd D _jThe center of circle form straight line L _Ij, cross D _iAnd D _jThe center of circle do and L _IjTwo vertical straight lines With

L then _Ij, With d _i, d _jOn the plane of delineation, form 16 intersection points, L _Ij,

With D _i, D _jOn space plane, also form 16 intersection points; According to the coordinate of corresponding intersection point, find the solution the homography transformation matrix of space plane to the plane of delineation at the plane of delineation and space plane;

(7) find the solution camera parameters according to the homography transformation matrix;

(8) according to camera parameters virtual three-dimensional body is added in the real scene.

2. augmented reality method according to claim 1, step 1a wherein) the described number n of determining the donut that will draw on the background plane according to background plane size and target sizes is that big or small and target sizes will satisfy the principle that has at least two donuts not to be blocked and determines its concrete number when target is in the background front according to background plane.

3. augmented reality method according to claim 1, wherein step (3) described on the plane of delineation according to d _iAnd d _jThe edge, find the solution the o of center of circle projection coordinate of its corresponding donut _iAnd o _j, find the solution as follows:

3a) utilize d _iAnd d _jIts edge is searched in the variation of tone saturation degree, with the d that obtains _iInside and outside oval matrix representation be

With

d _jInside and outside oval matrix representation be

With

3b) basis

With Obtain following formula:

$({\left(d_O^i\right)}^{-1}{d}_I^i-\mathrm{\λI})o_i=0,({\left(d_O^j\right)}^{-1}{d}_I^j-\mathrm{\λI})o_j=0$

λ is a constant in the formula, and I is 3 * 3 unit matrix;

3c) obtain the o of projection coordinate according to following formula _iAnd o _j

4. augmented reality method according to claim 1, wherein step (4) is described according to o _iAnd o _jAnd the transformation relation of donut inside and outside circle, seek d _iAnd d _jRadius corresponding compares ρ _iAnd ρ _j, carry out as follows:

4a) by k ρ _mObtain k transformation matrix G _m, m=1,2 ... k:

$G_m=I+({\mathrm{\ρ}}_m-1)\frac{o_i{o_i}^T{d}_O^i}{{d_i}^T{d}_O^i{o}_i},(m=\mathrm{1,2},\·\·\·k);$

4b) cross o _iDo any ray, the intersection point of this ray and interior ellipse is With the intersection point of outer ellipse be

With k G _mDifference substitution equation

That G that this equation is set up _mPairing radius ratio is exactly d _iRadius corresponding compares ρ _i

4c) cross o _jDo any ray, the intersection point of this ray and interior ellipse is

With the intersection point of outer ellipse be

With k G _mDifference substitution equation

That G that this equation is set up _mPairing radius ratio is exactly d _jRadius corresponding compares ρ _j

5. augmented reality method according to claim 1, wherein step (6) is described according to l _IjSolve two straight lines

With

Carry out as follows:

5a) on the plane of delineation, according to o _iAnd o _jObtain straight line l _Ij:

l _ij＝o _i×o _j

Symbol " * " is represented vectorial multiplication cross in the formula;

5b) according to described And l _IjObtain limit b:

$b={\left(d_I^i\right)}^{-1}{l}_{\mathrm{ij}}={\left(d_O^i\right)}^{-1}{l}_{\mathrm{ij}}={\left(d_I^j\right)}^{-1}{l}_{\mathrm{ij}}={\left(d_O^j\right)}^{-1}{l}_{\mathrm{ij}},$

Respectively with b point and o _iAnd o _jLink to each other, get straight line

And straight line

Images