CN102509104A - Confidence map-based method for distinguishing and detecting virtual object of augmented reality scene - Google Patents

Confidence map-based method for distinguishing and detecting virtual object of augmented reality scene Download PDF

Info

Publication number
CN102509104A
CN102509104A CN2011102998574A CN201110299857A CN102509104A CN 102509104 A CN102509104 A CN 102509104A CN 2011102998574 A CN2011102998574 A CN 2011102998574A CN 201110299857 A CN201110299857 A CN 201110299857A CN 102509104 A CN102509104 A CN 102509104A
Authority
CN
China
Prior art keywords
virtual
augmented reality
virtual objects
actual situation
reality scene
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN2011102998574A
Other languages
Chinese (zh)
Other versions
CN102509104B (en
Inventor
陈小武
赵沁平
穆珺
王哲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beihang University
Original Assignee
Beihang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beihang University filed Critical Beihang University
Priority to CN 201110299857 priority Critical patent/CN102509104B/en
Publication of CN102509104A publication Critical patent/CN102509104A/en
Application granted granted Critical
Publication of CN102509104B publication Critical patent/CN102509104B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Abstract

The invention relates to a confidence map-based method for distinguishing and detecting a virtual object of an augmented reality scene. The method comprises the following steps of: selecting vitality and reality classification features; constructing a pixel level vitality and reality classifier by means of the vitality and reality classification features; extracting regional comparison features of the augmented reality scene and a real scene respectively by means of the vitality and reality classification features, and constructing a region level vitality and reality classifier; giving a test augmented reality scene, detecting by means of the pixel level vitality and reality classifier and a small-size detection window to acquire a virtual score plot which reflects each pixel vitality and reality classification result; defining a virtual confidence map, and acquiring the virtual confidence map of the test augmented reality scene by thresholding; acquiring the rough shape and the position of a virtual object bounding box according to the distribution situation of high virtual response points in the virtual confidence map; and detecting by means of the region level vitality and reality classifier and a large-size detection window in the test augmented reality scene to acquire a final detection result of the virtual object. The method can be applied to the fields of film and television manufacturing, digital entertainment, education training and the like.

Description

Augmented reality scene virtual objects based on degree of confidence figure is differentiated and detection method
Technical field
The present invention relates to Flame Image Process, computer vision and augmented reality field, specifically a kind of augmented reality scene virtual objects based on degree of confidence figure is differentiated and detection method.
Background technology
Augmented reality is the further expansion of virtual reality; It coexists as in the same augmented reality system the virtual objects that computing machine generates and the true environment of outwardness by the equipment of necessity, demonstrates the augmented reality environment that virtual objects and true environment combine together to the user on sense organ and the experience effect.Along with the development of augmented reality technology, the appearance with augmented reality scene of the higher image sense of reality, the standard and judgment of urgent need tolerance and evaluation augmented reality scene confidence level.How to judge whether augmented reality scene of a scene, and further the virtual objects in the augmented reality scene detected that an approach as augmented reality scene image confidence level is estimated has important Research Significance and application demand.
2011, the researchist of much of Italian special human relations proposed a kind of image forge discrimination method, and the computing machine generation composition detection that this method can will incorporate in the real scene is come out.This work be in the known work on hand unique one be process object with the augmented reality scene.But the detection that this work is carried out is not to be unit with the object, but only detects the virtual composition in the augmented reality scene, and promptly testing result possibly be a zone, possibly be the point of scattered distribution yet.
The researchist of U.S. Dartmouth University in 2005 has proposed based on the natural image statistical model of wavelet decomposition and has adopted SVMs and the classify method of virtual image and true picture of linear discriminant analysis.This at first extracts after the coloured image wavelet decomposition quadravalence statistical nature (average, variance, the degree of bias, kurtosis) of coefficient of dissociation on each subband and direction; Consider the quadravalence linear prediction error characteristic between the adjacent coefficient of dissociation after the wavelet decomposition simultaneously, utilize SVMs and linear discriminant analysis method to train sorter then, the sorter that again the test set input is trained obtains classification results.The actual situation classification of this method is carried out to whole image, and classification accuracy has than great fluctuation process along with the extraction area size difference of actual situation characteristic of division.
2007, the researchist of USA New York University of Science and Technology proposed to utilize the color filter array interpolation to detect the method that aberration consistance in characteristics and the image is distinguished virtual image and true picture.This method is at first extracted based on the color filter array interpolation from the positive negative sample of training set and is detected the conforming characteristic of aberration in characteristics and the image; With training sorter in the characteristic input SVMs that extracts, the sorter that again the test set input is trained obtains classification results then.
The researchist of Alberta, Canada university in 2009 has proposed to utilize the classify method of virtual image and true picture of the consistance of image block resampling parameter.The principle of this method is based on virtual image may use operations such as rotation to texture image, convergent-divergent to the process of model surface texture in generating, and causes the parameter that each image block resamples in the virtual image inconsistent.Whether the parameter that so just can resample through the detected image piece consistent virtual image and the true picture distinguished.The parameter estimation that the image block of this method resamples is to carry out to whole image.
2004, the researchist of Compag Computer's Cambridge Research Laboratories proposed to utilize based on the Ha Er wave filter and has adopted the AdaBoost sorting algorithm to carry out the method that people's face detects.This method is at first extracted characteristic of division from training set; Retraining goes out the sorter based on people's face and non-face statistical nature; The characteristic of division input category device of the image to be detected that will extract then and the number that reduces the detection window that need to calculate through cascade classifier finally obtain testing result to raise the efficiency.The feature extraction of this method is based on the Ha Er wave filter, description be the region contrast that people's face inherent structure brings.
2005, French state-run computing machine and the graduate researchist of robotization proposed to utilize direction gradient histogram and linear SVMs to carry out the method for person detecting.This method branch at first carries out color normalization to the input picture; Calculate the gradient in the picture then; Statistics drops on the pixel between different directions and azimuthal bin, and overlapping space piece is compared normalization, the direction gradient histogram of each detection window of regeneration; Sort out personage/inhuman object area with linear support vector machine classifier at last, obtain testing result.This method has higher detection effect than other detection methods, but requires personage in the picture will roughly keep the state of vertically standing.What this method feature extraction was adopted is the image gradient histogram, description be the inherent characteristics of human body contour outline.
The method of above-mentioned differentiation virtual image and true picture, common ground are that the actual situation characteristic of division that they extract all is not suitable for to the actual situation of given area classification arbitrarily in the image.In addition, in the work of existing object detection, the general object of handling all has the stronger appearance characteristics that is easy to describe as prior imformation.Comparatively speaking, the virtual objects in the augmented reality scene detects, and it detects target (being virtual objects) and does not have the explicit in appearance prior imformation that is easy to describe, and like color, shape, size etc., therefore differentiation and detection difficulty are bigger.
Summary of the invention
Technical solution of the present invention: the deficiency that overcomes prior art; Provide a kind of augmented reality scene virtual objects to differentiate and detection method based on degree of confidence figure; This method does not need to know in advance any appearance information of virtual objects, like color, shape, size, need not know virtual objects residing position in the augmented reality scene yet; But utilize the physics imaging difference of distinguishing virtual objects and true picture; Carry out the actual situation characteristic of division and extract, the regional unique characteristics and the regional correlation characteristic of the positive negative sample of difference calculation training collection, and construct Pixel-level actual situation sorter and region class actual situation sorter; On this basis, through differentiate based on the virtual objects of virtual degree of confidence figure with detection carry out virtual objects tentatively formalize the location with accurately detect.
The technical scheme that the present invention adopts: the augmented reality scene virtual objects based on degree of confidence figure is differentiated and detection method; Step is following: make up augmented reality scene training dataset; And utilize the physics imaging difference of virtual objects and true picture, choose the actual situation characteristic of division; On training dataset, utilize the actual situation characteristic of division, extract the regional unique characteristics of augmented reality scene and real scene respectively, make up Pixel-level actual situation sorter; On training dataset, utilize the actual situation characteristic of division, extract the regional correlation characteristic of augmented reality scene and real scene respectively, make up region class actual situation sorter; Given test augmented reality scene utilizes Pixel-level actual situation sorter and small size detection window to detect, and obtains reflecting the virtual shot chart of each pixel actual situation classification results; Defining virtual degree of confidence figure, and on the basis of virtual shot chart, utilize thresholding to obtain testing the virtual degree of confidence figure of augmented reality scene; According to the distribution situation of high virtual responsive point among the virtual degree of confidence figure, obtain the rough shape and the position of virtual objects bounding box; On the basis of virtual objects coarse localization, in test augmented reality scene image, utilize region class actual situation sorter and large scale detection window to detect, obtain the final detection result of virtual objects.
Make up augmented reality scene training dataset.Concentrate at training data, the augmented reality scene image that will comprise virtual objects is as positive sample, with the real scene image as negative sample.Utilize the physics imaging difference of virtual objects and true picture, choose the actual situation characteristic of division.The virtual class characteristic of choosing comprises: local statistic, surface graded, second fundamental form, Marco Beltrami stream.Can extract the above-mentioned actual situation characteristic of division that obtains this some correspondence at each pixel place of image.
On training dataset, utilize the actual situation characteristic of division, extract the regional unique characteristics of augmented reality scene, make up Pixel-level actual situation sorter.When making up the Pixel-level sorter,, only choose the virtual objects zone as positive sample areas to the augmented reality scene image; And to the real scene image, only choose with positive sample in the akin zone of virtual objects as the negative sample zone.For given image-region, calculate the actual situation characteristic of division (comprising: local statistic, surface graded, second fundamental form, Marco Beltrami stream) of every bit in the zone; Utilize the moment of inertia compression method that the actual situation characteristic of division of given area is compressed, obtain the corresponding regional unique characteristics in this zone.The regional unique characteristics set input support vector machine classifier of positive negative sample is trained, obtain Pixel-level actual situation sorter.
On training dataset, utilize the actual situation characteristic of division, extract the regional correlation characteristic of augmented reality scene, make up region class actual situation sorter.For positive and negative sample areas, with itself being regarded as subject area to be judged; And the homalographic rectangular area outside the regional bounding box is regarded as the residing background area of object; Extract the actual situation characteristic of division of every bit in subject area and the background area respectively; In objects of statistics zone and the background area somewhat corresponding actual situation characteristic of division, constitute the joint distribution histogram of subject area characteristic and the joint distribution histogram of background area characteristic respectively; Calculate the card side's distance between two histograms,, be called the regional correlation characteristic its characteristic that is regarded as weighing difference between object and its background of living in; The regional correlation characteristic set input support vector machine classifier of the positive negative sample that extracts is trained, obtain region class actual situation sorter.
Virtual shot chart makes up, and step is the augmented reality scene image for input, utilize small size detection window (detection window is of a size of [10,30] * [10,30] pixel) with less moving step length (as 1,2,3,4, the 5} pixel) the scanning entire image; Calculate the regional unique characteristics of the little image block in each small size detection window; The regional unique characteristics of all little image blocks is input in the Pixel-level actual situation sorter, obtains the regional unique characteristics score of each little image block, the high remarked pixel level of score sorter is high with the degree of certainty that this image block is categorized as virtual region; Because the relative entire image of size of detection window is very little and densely distributed, therefore can the regional unique characteristics score of each little image block be mapped to the center pixel of this image block, and with its virtual score as this central pixel point; Constituted the virtual shot chart of whole augmented reality scene image thus.This process can improve counting yield through two-dimensional integration figure.
Virtual degree of confidence figure makes up, and step is that the virtual shot chart for the augmented reality scene image carries out thresholding and handles, and writes down all and virtual is divided into positive point; A fixing number percent N% is set, writes down all virtual preceding N% and these point residing positions on original image that are divided into positive point, these points are called high virtual responsive point; A fixing and less relatively constant M (as making M ∈ [10,100]) is set, writes down all virtual preceding M point and these residing positions on original image that are divided into positive point, these points are called the highest virtual responsive point; Can guarantee that through parameter setting the highest virtual responsive point also is contained in the set at high virtual responsive point place simultaneously, promptly the highest virtual responsive point is the virtual the highest part of score value that gets in the high virtual responsive point.Positional information on comprehensive high virtual responsive point, the highest virtual responsive point and the place original image thereof constitutes virtual degree of confidence figure.
The rough shape of virtual objects bounding box and position reasoning step are following: to the virtual degree of confidence figure that obtains, with its be divided into five homalographics, can be overlapping subregion, try to achieve the distribution center of the high virtual responsive point in each subregion respectively; The subregion center is regarded as candidate's virtual objects central point; Obtain the densely distributed zone of high virtual responsive point from the outside respectively expanded search of each central point; For the densely distributed zone of high virtual responsive point; Be similar to and extrapolate the candidate target shape (being presented as candidate's virtual objects bounding box) in this zone,, constitute the preliminary candidate region of virtual objects in conjunction with this regional positional information; In the preliminary candidate region of a plurality of virtual objects; Each the self-contained high virtual responsive point and the number of high virtual responsive point according to it; Select maximum one of weighting number, as the virtual objects candidate region, this zone has promptly comprised virtual objects bounding box rough shape and positional information with it.
For the coarse localization of the virtual objects that obtains, further optimize, obtain the final detection result of virtual objects.Concrete steps are: get the zone that area is a virtual objects candidate region twice around in the virtual objects candidate region; (range of size of large scale detection window is generally [200 to the structure form size a plurality of overlapped large scale detection window identical with the virtual objects candidate region in this zone; 500] * [200; 500], the concrete value of its length and width equals the length and the width of virtual objects bounding box in the virtual objects candidate region); Get the interior image block of each large scale detection window and calculate its regional correlation characteristic; The regional correlation characteristic input area level actual situation sorter of image block in all large scale detection window is classified, select the final detection result of the highest detection window of reserved portion as virtual objects.
The present invention compared with prior art, its beneficial effect is:
(1) the present invention is a detected object with the virtual objects in the augmented reality scene, can the virtual objects in the augmented reality scene done as a whole differentiation and detects.
(2) the present invention has made up two-stage actual situation sorter, comprises Pixel-level actual situation sorter and region class actual situation sorter, satisfies the demand that degree of confidence figure makes up and virtual objects finally detects.
(3) the present invention has built a degree of confidence figure, based on virtual degree of confidence figure, can under the condition that does not have prior imformations such as virtual objects outward appearance, shape, position, draw virtual objects approximate location and shape in the augmented reality scene.
(4) the present invention does not need to know in advance any appearance information of virtual objects; Like prior imformations such as color, shape, sizes; Need not know virtual objects residing position in the augmented reality scene yet; Wider applicability is arranged, but widespread use is generalized to fields such as production of film and TV, digital entertainment, educational training.
Description of drawings
Fig. 1 is an overall design structure of the present invention;
Fig. 2 is that virtual degree of confidence figure of the present invention makes up process flow diagram;
Fig. 3 is virtual objects bounding box shape of the present invention, position reasoning process flow diagram;
Fig. 4 is the process flow diagram of the candidate's of obtaining central point of the present invention;
Fig. 5 is that expanded search of the present invention, the high virtual responsive of acquisition are put the process flow diagram in densely distributed zone.
Embodiment
As shown in Figure 1, key step of the present invention is following: make up augmented reality scene training dataset, and utilize the physics imaging difference of virtual objects and true picture, choose the actual situation characteristic of division; On training dataset, utilize the actual situation characteristic of division, extract the regional unique characteristics of augmented reality scene, make up Pixel-level actual situation sorter; On training dataset, utilize the actual situation characteristic of division, extract the regional correlation characteristic of augmented reality scene, make up region class actual situation sorter; Given test augmented reality scene is utilized Pixel-level actual situation sorter to carry out small scale and is detected, and obtains reflecting the virtual shot chart of each pixel actual situation classification results; Defining virtual degree of confidence figure, and on the basis of virtual shot chart, utilize thresholding to obtain testing the virtual degree of confidence figure of augmented reality scene; According to the distribution situation of high virtual responsive point among the virtual degree of confidence figure, obtain the rough shape and the position of virtual objects bounding box; On the basis of virtual objects coarse localization, in test augmented reality scene image, utilize region class actual situation sorter and large scale detection window to detect, obtain the final detection result of virtual objects.
The structure training dataset is used to train the actual situation sorter.Training dataset is made up of as negative sample as positive sample, real scene image the augmented reality scene image that comprises virtual objects.When training Pixel-level sorter,, only choose the virtual objects zone as positive sample to the augmented reality scene image; And to the real scene image, only choose with positive sample in the akin zone of virtual objects as negative sample.When training region class sorter, to the augmented reality scene image, choose virtual objects and on every side the image-region of homalographic as positive sample; And to the real scene image, choose with positive sample in the akin zone of virtual objects and on every side the image-region of homalographic as negative sample.
The extraction of zone unique characteristics.For given image-region, calculate the actual situation characteristic of division of every bit in the zone, comprising: local statistic, surface graded, second fundamental form, Marco Beltrami stream; Utilize the moment of inertia compression method that the actual situation characteristic of division of given area is compressed, obtain the corresponding regional unique characteristics in this zone.
Local statistic, surface graded, second fundamental form, Marco Beltrami stream physical significance and computing method thereof separately are respectively as follows:
What local statistic reflected is local small marginal texture.The computing method of local statistic are following: getting any 1 P on the gray-scale map of original image, is the little image block of one 3 * 3 pixel at center with the P point, and the pixel value of every bit wherein is arranged in 9 dimensional vector x=[x in order 1, x 2... x 9].The local statistic y that P is ordered is one 9 dimensional vector, and it is defined as:
y = x - x ‾ | | x - x ‾ | | D . Wherein, x ‾ = 1 9 Σ i = 1 9 x i ; And || || DIt is the operation of D norm.
The definition of D norm operation is:
Figure BDA0000095251690000063
wherein in i～j presentation video piece the point of all neighbours territories relations right.
The local statistic actual situation characteristic of division at p place, arbitrfary point is the 9 dimensional vector y at this some place.
Surface graded is the nonlinearities change characteristics that are used for measuring in the real scene imaging process.The surface graded S at any point place is defined as in the image:
Figure BDA0000095251690000064
Wherein,
Figure BDA0000095251690000065
Be the image gradient mould value at this some place, I x, I xThe local derviation of difference presentation video x direction (horizontal direction) and y direction (vertical direction).α is a constant, α=0.25.
The surface graded actual situation characteristic of division at p place, arbitrfary point is united by the image pixel value I at this some place and the surface graded S in this some place and is constituted.
Second fundamental form is to be used for describing the local concavo-convex degree of imaging surface.Two component λ of second fundamental form 1And λ 2Two eigenwerts of the corresponding matrix A of difference.
A = 1 1 + I x 2 + I y 2 I Xx I Xy I Xy I Yy , Wherein, I x, I xThe local derviation of difference presentation video x direction and y direction; I Xx, I Xy, I YyThe second order local derviation of difference presentation video xx direction, xy direction, yy direction; Can calculate the value of matrix A by this formula.Might as well A be designated as: A = a 11 a 12 a 21 a 22 , A wherein 11, a 12, α 21, α 22Four corresponding element values among the difference representing matrix A.Therefore, two of matrix A eigenvalue 1And λ 2Computing formula is following:
{ λ 1 , λ 2 } = a 11 + a 22 ± ( a 11 - a 22 ) 2 + 4 a 12 a 21 2 , λ 1 λ 2
The second fundamental form actual situation characteristic of division at p place, arbitrfary point is by the image gradient mould value at this some place
Figure BDA00000952516900000610
Two component λ with this some place second fundamental form 1, λ 2The associating formation.
Marco Beltrami stream can be used for describing the correlativity between the different color channels.(B}) corresponding Marco Beltrami flows Δ to Color Channel c for c={R, G gI cBe defined as:
Δ g I c = 1 | g | ( ∂ x ( | g | ( g xx ∂ x I c + g xy ∂ y I c ) ) ) + 1 | g | ( ∂ y ( | g | ( g yx ∂ x I c + g yy ∂ y I c ) ) )
Wherein, I cThe Color Channel c of expression original image (c={R, G, B}) image of correspondence; Operator
Figure BDA0000095251690000072
Represent to measure the local derviation of x direction and y direction respectively for effect;
Matrix g = 1 + ( I x R ) 2 + ( I x G ) 2 + ( I x B ) 2 I x R I y R + I x G I y G + I x B I y B I x R I y R + I x G I y G + I x B I y B 1 + ( I y R ) 2 + ( I y G ) 2 + ( I y B ) 2 ,
Figure BDA0000095251690000074
The local derviation of difference presentation video R passage (red channel) x direction and y direction;
Figure BDA0000095251690000075
The local derviation of difference presentation video G passage (green channel) x direction and y direction;
Figure BDA0000095251690000076
The local derviation of difference presentation video B passage (blue channel) x direction and y direction; | g| is the determinant of matrix g; And g Xx, g Xy, g Yy, g YxThen by g - 1 = g Xx g Xy g Yx g Yy Provide respectively, i.e. g Xx, g Xy, g Yy, g YxBe respectively four corresponding element values of inverse matrix of matrix g.
The Marco Beltrami stream actual situation characteristic of division at p place, arbitrfary point is by each Color Channel c (c={R, G, Marco Beltrami flow component Δ B}) at this some place gI cImage gradient mould value with each Color Channel
Figure BDA0000095251690000078
The associating formation.
Behind four groups of actual situation characteristic of divisions of every bit in calculating the zone (comprising: local statistic, surface graded, second fundamental form, Marco Beltrami flow), need utilize the moment of inertia compression method that the actual situation characteristic of division is compressed.Moment of inertia compression method step is following: consider earlier separately any one group (processing mode of each group actual situation characteristic of division is all identical) in local statistic, surface graded, second fundamental form, these four groups of actual situation characteristic of divisions of Marco Beltrami stream.If total N point in the given area, any point P i(i=1 ..., the total M dimension of one group of actual situation characteristic of division N) (value of M according in four groups of actual situation characteristic of divisions, get fixed wherein one group can confirm), will put P i(i=1 ..., one group of actual situation characteristic of division N) is designated as v i=(v I1..., v Im).At this moment, will put P iActual situation characteristic of division v i=(v I1..., v Im) be regarded as a particle in the M dimensional feature space, stipulate that the quality of this particle does , the position coordinates of this particle in the M dimensional feature space is v i=(v I1..., v Im), then can calculate the moment of inertia matrix J of the system of particles that all N particles constitute through the solid moment of inertia Matrix Formula.Moment of inertia matrix J is that a M * M ties up matrix, and matrix J can be write as following form:
Figure BDA00000952516900000710
Any element of matrix J is designated as J Jk(j, k=1 ..., M).J JkComputing method be:
Figure BDA00000952516900000711
(j, k=1 ..., M).M wherein iExpression particle P iQuality, v i=(v I1..., v Im) expression point P iPosition coordinates in feature space; | v i| expression particle P iTo the Euclidean distance of true origin, promptly
Figure BDA0000095251690000081
δ JkBe the Kronecker function, its computing method do δ Jk = 1 , If i = j 0 , If i ≠ j . Can confirm all elements J of moment of inertia matrix J thus Jk(j, k=1 ..., M).Symmetry by moment of inertia matrix J can be known J Jk=J Kj, therefore only get the principal diagonal and the above all elements J of principal diagonal of matrix J Jk(j, k=1 ..., M and j≤k), these elements can be represented all information of original matrix J.
Get all elements J in the moment of inertia matrix Jk(j, k=1 ..., M and j≤k); The centroid vector of uniting all particles
Figure BDA0000095251690000083
Unite all particles and true origin distance | v i| average, variance, the degree of bias, kurtosis; Constitute a proper vector.This proper vector is the compression expression result that this group actual situation characteristic of division of being had a few in this zone obtains through the moment of inertia compression method.Four groups of compression expression result associatings with obtaining respectively can obtain the corresponding regional unique characteristics in this zone.Because moment of inertia matrix can describe the distribution of a plurality of particles in feature space preferably, thus the moment of inertia matrix compression method can be when a plurality of dimensions strong point be compressed to the zone as far as possible low guaranteed keep the information that legacy data distributes largely.
The regional correlation Feature Extraction.For given image-region, the zone itself is regarded as subject area to be judged; And the homalographic rectangular area of next-door neighbour's bounding box outside the regional bounding box is regarded as the residing background area of object; Calculate the actual situation characteristic of division of every bit in subject area and the background area respectively; Respectively in the objects of statistics zone with the background area in institute have a few the actual situation characteristic of division of correspondence, the joint distribution histogram of the interior and background area characteristic of formation subject area; Card side's distance between the joint distribution histogram of calculating object provincial characteristics and the joint distribution histogram of background area characteristic with its characteristic that is regarded as weighing contrast between object and its background of living in or difference, is called the regional correlation characteristic.
Make up Pixel-level actual situation sorter and region class actual situation sorter, be used for dividing given zone whether to belong to the virtual objects region from the angle of regional unique characteristics and the angular area of regional correlation characteristic respectively.
Pixel-level actual situation sorter makes up, through the positive negative sample of input training set; Extract the regional unique characteristics of positive negative sample respectively; The regional unique characteristics set input support vector machine classifier of the positive negative sample that extracts is trained, obtain Pixel-level actual situation sorter.The characteristics of Pixel-level actual situation sorter are that the feature compression method that it adopted makes its classification results tool have the dimensions adaptability.Promptly when treating that the classified regions unique characteristics is that Pixel-level actual situation sorter had accuracy preferably for the classification results whether given area belongs to the virtual objects region when significantly different extracted region obtained by the area size with training set.Particularly: although the Pixel-level sorter is (to be of a size of [10 by virtual objects in the training set; 30] * [10; 30] regional unique characteristics set training pixel) obtains, but experimental result shows: this sorter (is of a size of [10,30] * [10 for relatively little a lot of zone; 30] provincial characteristics pixel), its classification results still has accuracy preferably.Since the classification of this sorter to as if to the small size zone, and these zonules are used for the approximate description regional center and put pairing pixel, therefore this sorter are become Pixel-level actual situation sorter.
Region class actual situation sorter makes up, through the positive negative sample of input training set; Extract the regional correlation characteristic of positive negative sample respectively; The regional correlation characteristic set input support vector machine classifier of the positive negative sample that extracts is trained, obtain region class actual situation sorter.Because the characteristic of division that region class actual situation sorter uses is the population distribution difference between conversion zone and the place background thereof, therefore can object to be detected be done as a wholely to differentiate preferably and detect.
Make up virtual shot chart.For the input the augmented reality scene image, utilize small size detection window (detection window is of a size of [10,30] * [10,30] pixel) with less moving step length (as 1,2,3,4, the 5} pixel) scanning entire image; Calculate the regional unique characteristics of the little image block in each small size detection window; The regional unique characteristics of all little image blocks is input in the Pixel-level actual situation sorter, obtains the regional unique characteristics score of each little image block, the high remarked pixel level of score sorter is high with the degree of certainty that this image block is categorized as virtual region; Because the relative entire image of size of detection window is very little and densely distributed, therefore can the regional unique characteristics score of each little image block be mapped to the center pixel of this image block, and with its virtual score as this central pixel point; Constituted the virtual shot chart of whole augmented reality scene image thus.Because the actual situation characteristic of division during zone self sign is calculated calculates and the feature compression operation is more consuming time; And need calculate its regional unique characteristics one by one for a large amount of overlapped image block that generates, therefore in this step, adopt integrogram method speed-up computation process.The virtual shot chart that obtains thus makes up the relation that the result can reflect preferably whether point and this point on the image belong to virtual objects.That is: experimental result shows: in the virtual shot chart, the point that virtual score is high generally all concentrates on the virtual objects region; Otherwise, the point of virtual objects region, the corresponding virtual score is all higher.
Make up virtual degree of confidence figure, its flow process is as shown in Figure 2.At first, carry out thresholding for the virtual shot chart that obtains and handle, select earlier and write down all virtual to be divided into positive point; A fixing number percent N% is set, selects and write down all virtual preceding N% and these point residing positions on original image that are divided into positive point.These points are called high virtual responsive point.A fixing and less relatively constant M (as making M ∈ [10,100]) is set, selects and write down all virtual preceding M point and these residing positions on original image that are divided into positive point.These points are called the highest virtual responsive point.The number of high virtual responsive point is much smaller than the number of high virtual responsive point.Positional information on comprehensive high virtual responsive point, the highest virtual responsive point and the place original image thereof promptly constitutes virtual degree of confidence figure.Described virtual degree of confidence figure makes up the relation that the result can reflect preferably whether point and this point on the image belong to virtual objects.That is: experimental result shows: among the virtual degree of confidence figure, high virtual responsive point generally all concentrates on the virtual objects region; Otherwise, the point of virtual objects region, corresponding high virtual responsive point distributes comparatively intensive.Similarly, the highest virtual responsive point generally only appears at the virtual objects region; Otherwise, in the virtual objects region, the highest more virtual responsive point generally can appear.For appearing at beyond the virtual objects zone in high virtual responsive point and the highest virtual responsive point, be referred to as noise spot.
The rough shape of virtual objects bounding box and position reasoning flow process are as shown in Figure 3, may further comprise the steps: divide subregion, obtain candidate's central point; Expanded search obtains high virtual responsive and puts densely distributed zone; The preliminary candidate region of virtual objects is confirmed; The virtual objects candidate region is confirmed.Wherein particularly, dividing subregion, is the virtual degree of confidence figure to obtaining, with its be divided into five homalographics, can be overlapping subregion.The process flow diagram that obtains candidate's central point is as shown in Figure 4, respectively according to the distribution of the high virtual responsive point in each subregion, utilizes the average drifting algorithm to try to achieve the high virtual responsive point center of distribution point in each subregion, and this central point is called candidate's central point.Candidate's central point number is k (k≤5, k less than 5 situation corresponding to there not being high virtual responsive point in some subregion), must be in above-mentioned k candidate's central point at this central point that might as well suppose virtual objects institute corresponding region.Expanded search; Obtain high virtual responsive and put densely distributed zone, this process is as shown in Figure 5: for each candidate's central point, be the center of circle with candidate's central point; Length to increase according to fixed step size is radius; Dynamically construct the circular region of search that increases successively, when the number of high virtual responsive point no longer increases in the current search zone, then can think to have searched the regional border of virtual objects; Under the ideal situation; The condition that expanded search stops is when search radius increases; The number increment of high virtual responsive point is zero in the region of search, but in order to eliminate the influence of the noise spot that exists among the virtual degree of confidence figure, a squelch parameter is set; The condition reinforcement that expanded search is stopped is: when search radius increased, the number increment of high virtual responsive point must be greater than the squelch parameter in the region of search.
The preliminary candidate region of virtual objects is confirmed to extrapolate the candidate target shape in this zone by the densely distributed zone of high virtual responsive point is approximate, in conjunction with this regional positional information, constitutes the preliminary candidate region of virtual objects.When expanded search stops, obtaining high virtual responsive and put densely distributed zone, can know the set P of high virtual responsive points all in this zone, can draw the candidate target shape in this zone based on P, promptly be presented as the shape of candidate target bounding box:
x min＝min({x|<x，y>∈P})；x max＝max({x|<x，y>∈P})；
y min＝min({y|<x，y>∈P})；y max＝max({y|<x，y>∈P})；
X wherein Min, x MaxRepresent x direction minimum value and maximal value in the correspondence image coordinate of candidate target bounding box region respectively; y Min, y MaxRepresent y direction minimum value and maximal value in the correspondence image coordinate of candidate target bounding box position respectively.Can confirm that thus the candidate target bounding box is with respect to position in the image and shape.
Candidate target shape in this zone in conjunction with this regional positional information (candidate's center position), constitutes the preliminary candidate region of virtual objects.
In the preliminary candidate region of the k that an obtains virtual objects; Each the self-contained high virtual responsive point and the number of high virtual responsive point according to it; Select maximum one of weighting number; As the virtual objects candidate region, this zone has promptly comprised the general shape of virtual objects bounding box and the information of position with it.
For the coarse localization of the virtual objects that obtains, further optimize, with the error that reduces possibly occur in the computation process of virtual objects candidate region, thereby obtain the final detection result of virtual objects.Concrete steps are: get the zone that area is a virtual objects candidate region twice around in the virtual objects candidate region; (range of size of large scale detection window is generally [200 to the structure form size a plurality of overlapped large scale detection window identical with the virtual objects candidate region in this zone; 500] * [200; 500], the concrete value of its length and width equals the length and the width of virtual objects bounding box in the virtual objects candidate region); Get the interior image block of each large scale detection window and calculate its regional correlation characteristic; The regional correlation characteristic input area level actual situation sorter of image block in all large scale detection window is classified, select the final detection result of the highest detection window of reserved portion as virtual objects.
The above is merely basic explanations more of the present invention, and any equivalent transformation according to technical scheme of the present invention is done all should belong to protection scope of the present invention.
The present invention does not set forth part in detail and belongs to techniques well known.

Claims (8)

1. differentiate and detection method based on the augmented reality scene virtual objects of degree of confidence figure, it is characterized in that performing step is following:
(1) with the augmented reality image that comprises virtual objects as positive sample, the real scene image makes up augmented reality scene training dataset as negative sample; And utilize the physics imaging difference of virtual objects and true picture, choose the actual situation characteristic of division;
(2) on training dataset, utilize the actual situation characteristic of division, extract the regional unique characteristics of augmented reality scene and real scene respectively, make up Pixel-level actual situation sorter;
(3) on training dataset, utilize the actual situation characteristic of division, extract the regional correlation characteristic of augmented reality scene and real scene respectively, make up region class actual situation sorter;
(4) given test augmented reality scene utilizes Pixel-level actual situation sorter and small size detection window to detect, and obtains reflecting the virtual shot chart of each pixel actual situation classification results;
(5) defining virtual degree of confidence figure, and on the basis of virtual shot chart, utilize thresholding to obtain testing the virtual degree of confidence figure of augmented reality scene;
(6) based on virtual degree of confidence figure, carry out the virtual objects coarse localization, obtain the rough shape and the position of virtual objects bounding box;
(7) on the basis of virtual objects coarse localization, in test augmented reality scene image, utilize region class actual situation sorter and large scale detection window to detect, obtain the final detection result of virtual objects.
2. the augmented reality scene virtual objects based on degree of confidence figure according to claim 1 is differentiated and detection method; It is characterized in that: the actual situation characteristic of division of choosing in the said step (1) comprises: local statistic, surface graded, second fundamental form and Marco Beltrami stream.Can extract the above-mentioned actual situation characteristic of division that obtains this some correspondence at each pixel place of image.
3. the augmented reality scene virtual objects based on degree of confidence figure according to claim 1 is differentiated and detection method; It is characterized in that: when making up the Pixel-level sorter in the said step (2); On training dataset; To the augmented reality scene image, only choose the virtual objects zone as positive sample areas; And to the real scene image, only choose with positive sample in the akin zone of virtual objects as the negative sample zone; For given image-region, calculate the actual situation characteristic of division of every bit in the zone; Utilize the moment of inertia compression method that the actual situation characteristic of division of given positive and negative sample areas is compressed, obtain the corresponding regional unique characteristics in this zone; The regional unique characteristics set input support vector machine classifier of positive negative sample is trained, obtain Pixel-level actual situation sorter.
4. the augmented reality scene virtual objects based on degree of confidence figure according to claim 1 is differentiated and detection method; It is characterized in that: when said step (3) makes up region class actual situation sorter; On training dataset, for positive and negative sample areas, with itself being regarded as subject area to be judged; And the homalographic rectangular area outside the regional bounding box is regarded as the residing background area of object; Extract the actual situation characteristic of division of every bit in subject area and the background area respectively; In objects of statistics zone and the background area somewhat corresponding actual situation characteristic of division, constitute the joint distribution histogram of subject area characteristic and the joint distribution histogram of background area characteristic respectively; Calculate the card side's distance between two histograms,, be called the regional correlation characteristic its characteristic that is regarded as weighing difference between object and its background of living in; The regional correlation characteristic set input support vector machine classifier of the positive negative sample that extracts is trained, obtain region class actual situation sorter.
5. the augmented reality scene virtual objects based on degree of confidence figure according to claim 1 is differentiated and detection method; It is characterized in that: the virtual shot chart construction step of said step (4) is: for the augmented reality scene image of input, utilize the small size detection window with less moving step length scanning entire image; Calculate the regional unique characteristics of the little image block in each small size detection window; The regional unique characteristics of all little image blocks is input in the Pixel-level actual situation sorter, obtains the regional unique characteristics score of each little image block, the high remarked pixel level of score sorter is high with the degree of certainty that this image block is categorized as virtual region; Because the relative entire image of size of detection window is very little and densely distributed, therefore can the regional unique characteristics score of each little image block be mapped to the center pixel of this image block, and with its virtual score as this central pixel point; Constituted the virtual shot chart of whole augmented reality scene image thus.
6. the augmented reality scene virtual objects based on degree of confidence figure according to claim 1 is differentiated and detection method; It is characterized in that: the virtual degree of confidence figure construction method of said step (5) is: carry out thresholding for the virtual shot chart of augmented reality scene image and handle, write down all and virtual be divided into positive point; A fixing number percent N% is set, writes down all virtual preceding N% and these point residing positions on original image that are divided into positive point, these points are called high virtual responsive point; A fixing and less relatively constant M is set, writes down all virtual preceding M point and these residing positions on original image that are divided into positive point, these points are called the highest virtual responsive point; Can guarantee that through parameter setting the highest virtual responsive point also is contained in the set at high virtual responsive point place simultaneously, promptly the highest virtual responsive point is the virtual the highest part of score value that gets in the high virtual responsive point; Positional information on comprehensive high virtual responsive point, the highest virtual responsive point and the place original image thereof constitutes virtual degree of confidence figure.
7. the augmented reality scene virtual objects based on degree of confidence figure according to claim 1 is differentiated and detection method; It is characterized in that: said step (6) obtains the rough shape of virtual objects bounding box and the method for position is: to the virtual degree of confidence figure that obtains with its be divided into five homalographics, can be overlapping subregion, try to achieve the distribution center of the high virtual responsive point in each subregion respectively; The subregion center is regarded as candidate's virtual objects central point,, obtains the densely distributed zone of high virtual responsive point from each central point outside expanded search respectively; For the densely distributed zone of high virtual responsive point, be similar to respectively and extrapolate the candidate target shape in this zone, in conjunction with this regional positional information, constitute the preliminary candidate region of virtual objects; In the preliminary candidate region of virtual objects; Each the self-contained high virtual responsive point and the number of high virtual responsive point according to it; Select maximum one of weighting number, as the virtual objects candidate region, this zone has promptly comprised virtual objects bounding box rough shape and positional information with it.
8. the augmented reality scene virtual objects based on degree of confidence figure according to claim 1 is differentiated and detection method; It is characterized in that: said step (7) virtual objects detection method is specially: intensive sampling around the virtual objects candidate region in test augmented reality scene; Construct a plurality of overlapped detection window; And use region class actual situation sorter to classify, choose the final detection result of the best detection window of score as virtual objects.
CN 201110299857 2011-09-30 2011-09-30 Confidence map-based method for distinguishing and detecting virtual object of augmented reality scene Expired - Fee Related CN102509104B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201110299857 CN102509104B (en) 2011-09-30 2011-09-30 Confidence map-based method for distinguishing and detecting virtual object of augmented reality scene

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201110299857 CN102509104B (en) 2011-09-30 2011-09-30 Confidence map-based method for distinguishing and detecting virtual object of augmented reality scene

Publications (2)

Publication Number Publication Date
CN102509104A true CN102509104A (en) 2012-06-20
CN102509104B CN102509104B (en) 2013-03-20

Family

ID=46221185

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201110299857 Expired - Fee Related CN102509104B (en) 2011-09-30 2011-09-30 Confidence map-based method for distinguishing and detecting virtual object of augmented reality scene

Country Status (1)

Country Link
CN (1) CN102509104B (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102798583A (en) * 2012-07-13 2012-11-28 长安大学 Ore rock block degree measurement method based on improved FERRET
WO2015062164A1 (en) * 2013-10-31 2015-05-07 The Chinese University Of Hong Kong Method for optimizing localization of augmented reality-based location system
CN104780180A (en) * 2015-05-12 2015-07-15 成都绿野起点科技有限公司 Virtual reality platform based on mobile terminals
CN104794754A (en) * 2015-05-12 2015-07-22 成都绿野起点科技有限公司 Distribution type virtual reality system
CN104869160A (en) * 2015-05-12 2015-08-26 成都绿野起点科技有限公司 Distributed virtual reality system based on cloud platform
CN105654504A (en) * 2014-11-13 2016-06-08 丁业兵 Adaptive bandwidth mean value drift object tracking method based on rotary inertia
US9858482B2 (en) 2013-05-28 2018-01-02 Ent. Services Development Corporation Lp Mobile augmented reality for managing enclosed areas
CN108492374A (en) * 2018-01-30 2018-09-04 青岛中兴智能交通有限公司 The application process and device of a kind of AR on traffic guidance
CN110555358A (en) * 2018-06-01 2019-12-10 苹果公司 method and apparatus for detecting and identifying features in an AR/VR scene
CN111739084A (en) * 2019-03-25 2020-10-02 上海幻电信息科技有限公司 Picture processing method, atlas processing method, computer device, and storage medium
CN112270063A (en) * 2020-08-07 2021-01-26 四川航天川南火工技术有限公司 Sensitive parameter hypothesis testing method for initiating explosive system
CN115346002A (en) * 2022-10-14 2022-11-15 佛山科学技术学院 Virtual scene construction method and rehabilitation training application thereof
CN117315375A (en) * 2023-11-20 2023-12-29 腾讯科技（深圳）有限公司 Virtual part classification method, device, electronic equipment and readable storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101520904A (en) * 2009-03-24 2009-09-02 上海水晶石信息技术有限公司 Reality augmenting method with real environment estimation and reality augmenting system
CN101893935A (en) * 2010-07-14 2010-11-24 北京航空航天大学 Cooperative construction method for enhancing realistic table-tennis system based on real rackets
WO2011084720A2 (en) * 2009-12-17 2011-07-14 Qderopateo, Llc A method and system for an augmented reality information engine and product monetization therefrom

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101520904A (en) * 2009-03-24 2009-09-02 上海水晶石信息技术有限公司 Reality augmenting method with real environment estimation and reality augmenting system
WO2011084720A2 (en) * 2009-12-17 2011-07-14 Qderopateo, Llc A method and system for an augmented reality information engine and product monetization therefrom
CN101893935A (en) * 2010-07-14 2010-11-24 北京航空航天大学 Cooperative construction method for enhancing realistic table-tennis system based on real rackets

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102798583B (en) * 2012-07-13 2014-07-30 长安大学 Ore rock block degree measurement method based on improved FERRET
CN102798583A (en) * 2012-07-13 2012-11-28 长安大学 Ore rock block degree measurement method based on improved FERRET
US9858482B2 (en) 2013-05-28 2018-01-02 Ent. Services Development Corporation Lp Mobile augmented reality for managing enclosed areas
WO2015062164A1 (en) * 2013-10-31 2015-05-07 The Chinese University Of Hong Kong Method for optimizing localization of augmented reality-based location system
CN105654504A (en) * 2014-11-13 2016-06-08 丁业兵 Adaptive bandwidth mean value drift object tracking method based on rotary inertia
CN104780180B (en) * 2015-05-12 2019-02-12 国电物资集团有限公司电子商务中心 A kind of Virtual Reality Platform based on mobile terminal
CN104780180A (en) * 2015-05-12 2015-07-15 成都绿野起点科技有限公司 Virtual reality platform based on mobile terminals
CN104794754A (en) * 2015-05-12 2015-07-22 成都绿野起点科技有限公司 Distribution type virtual reality system
CN104869160A (en) * 2015-05-12 2015-08-26 成都绿野起点科技有限公司 Distributed virtual reality system based on cloud platform
CN104794754B (en) * 2015-05-12 2018-04-20 成都绿野起点科技有限公司 A kind of Distributed Virtual Reality System
CN104869160B (en) * 2015-05-12 2018-07-31 成都绿野起点科技有限公司 A kind of Distributed Virtual Reality System based on cloud platform
CN108492374A (en) * 2018-01-30 2018-09-04 青岛中兴智能交通有限公司 The application process and device of a kind of AR on traffic guidance
CN108492374B (en) * 2018-01-30 2022-05-27 青岛中兴智能交通有限公司 Application method and device of AR (augmented reality) in traffic guidance
CN110555358A (en) * 2018-06-01 2019-12-10 苹果公司 method and apparatus for detecting and identifying features in an AR/VR scene
CN110555358B (en) * 2018-06-01 2023-09-12 苹果公司 Method and apparatus for detecting and identifying features in an AR/VR scene
CN111739084A (en) * 2019-03-25 2020-10-02 上海幻电信息科技有限公司 Picture processing method, atlas processing method, computer device, and storage medium
CN111739084B (en) * 2019-03-25 2023-12-05 上海幻电信息科技有限公司 Picture processing method, atlas processing method, computer device, and storage medium
CN112270063A (en) * 2020-08-07 2021-01-26 四川航天川南火工技术有限公司 Sensitive parameter hypothesis testing method for initiating explosive system
CN112270063B (en) * 2020-08-07 2023-03-28 四川航天川南火工技术有限公司 Sensitive parameter hypothesis testing method for initiating explosive system
CN115346002A (en) * 2022-10-14 2022-11-15 佛山科学技术学院 Virtual scene construction method and rehabilitation training application thereof
CN115346002B (en) * 2022-10-14 2023-01-17 佛山科学技术学院 Virtual scene construction method and rehabilitation training application thereof
CN117315375A (en) * 2023-11-20 2023-12-29 腾讯科技（深圳）有限公司 Virtual part classification method, device, electronic equipment and readable storage medium
CN117315375B (en) * 2023-11-20 2024-03-01 腾讯科技（深圳）有限公司 Virtual part classification method, device, electronic equipment and readable storage medium

Also Published As

Publication number Publication date
CN102509104B (en) 2013-03-20

Similar Documents

Publication Publication Date Title
CN102509104B (en) Confidence map-based method for distinguishing and detecting virtual object of augmented reality scene
CN105654021B (en) Method and apparatus of the detection crowd to target position attention rate
CN104063702B (en) Three-dimensional gait recognition based on shielding recovery and partial similarity matching
CN104536009B (en) Above ground structure identification that a kind of laser infrared is compound and air navigation aid
CN102932605B (en) Method for selecting camera combination in visual perception network
CN103927511B (en) image identification method based on difference feature description
CN101996401B (en) Target analysis method and apparatus based on intensity image and depth image
CN104166841A (en) Rapid detection identification method for specified pedestrian or vehicle in video monitoring network
CN102270308B (en) Facial feature location method based on five sense organs related AAM (Active Appearance Model)
CN106529499A (en) Fourier descriptor and gait energy image fusion feature-based gait identification method
Zia et al. Revisiting 3d geometric models for accurate object shape and pose
CN105488809A (en) Indoor scene meaning segmentation method based on RGBD descriptor
Wang et al. Window detection from mobile LiDAR data
CN106250895A (en) A kind of remote sensing image region of interest area detecting method
CN106780552B (en) Anti-shelter target tracking based on regional area joint tracing detection study
CN105160317A (en) Pedestrian gender identification method based on regional blocks
CN102663391A (en) Image multifeature extraction and fusion method and system
CN103186775A (en) Human body motion recognition method based on mixed descriptor
CN104182765A (en) Internet image driven automatic selection method of optimal view of three-dimensional model
CN104036284A (en) Adaboost algorithm based multi-scale pedestrian detection method
CN101655914A (en) Training device, training method and detection method
CN109902585A (en) A kind of three modality fusion recognition methods of finger based on graph model
CN103971106A (en) Multi-view human facial image gender identification method and device
CN105976376A (en) High resolution SAR image target detection method based on part model
McKeown et al. Performance evaluation for automatic feature extraction

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20130320

Termination date: 20150930

EXPY Termination of patent right or utility model