CN102509104B - Confidence map-based method for distinguishing and detecting virtual object of augmented reality scene - Google Patents

Confidence map-based method for distinguishing and detecting virtual object of augmented reality scene Download PDF

Info

Publication number
CN102509104B
CN102509104B CN 201110299857 CN201110299857A CN102509104B CN 102509104 B CN102509104 B CN 102509104B CN 201110299857 CN201110299857 CN 201110299857 CN 201110299857 A CN201110299857 A CN 201110299857A CN 102509104 B CN102509104 B CN 102509104B
Authority
CN
China
Prior art keywords
virtual
actual situation
augmented reality
point
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN 201110299857
Other languages
Chinese (zh)
Other versions
CN102509104A (en
Inventor
陈小武
赵沁平
穆珺
王哲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beihang University
Original Assignee
Beihang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beihang University filed Critical Beihang University
Priority to CN 201110299857 priority Critical patent/CN102509104B/en
Publication of CN102509104A publication Critical patent/CN102509104A/en
Application granted granted Critical
Publication of CN102509104B publication Critical patent/CN102509104B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Image Analysis (AREA)

Abstract

The invention relates to a confidence map-based method for distinguishing and detecting a virtual object of an augmented reality scene. The method comprises the following steps of: selecting vitality and reality classification features; constructing a pixel level vitality and reality classifier by means of the vitality and reality classification features; extracting regional comparison features of the augmented reality scene and a real scene respectively by means of the vitality and reality classification features, and constructing a region level vitality and reality classifier; giving a test augmented reality scene, detecting by means of the pixel level vitality and reality classifier and a small-size detection window to acquire a virtual score plot which reflects each pixel vitality and reality classification result; defining a virtual confidence map, and acquiring the virtual confidence map of the test augmented reality scene by thresholding; acquiring the rough shape and the position of a virtual object bounding box according to the distribution situation of high virtual response points in the virtual confidence map; and detecting by means of the region level vitality and reality classifier and a large-size detection window in the test augmented reality scene to acquire a final detection result of the virtual object. The method can be applied to the fields of film and television manufacturing, digital entertainment, education training and the like.

Description

Augmented reality scene virtual objects based on degree of confidence figure is differentiated and detection method
Technical field
The present invention relates to image processing, computer vision and augmented reality field, specifically a kind of augmented reality scene virtual objects based on degree of confidence figure is differentiated and detection method.
Background technology
Augmented reality is the further expansion of virtual reality, it coexists as in the same augmented reality system true environment of virtual objects that computing machine generates and outwardness by the equipment of necessity, presents the augmented reality environment that virtual objects and true environment combine together to the user on sense organ and the experience effect.Along with the development of augmented reality, have the appearance of the augmented reality scene of the higher image sense of reality, be badly in need of the standard and judgment of tolerance and evaluation augmented reality scene confidence level.How to judge whether augmented reality scene of a scene, and further the virtual objects in the augmented reality scene detected that an approach as augmented reality scene image trust evaluation has important Research Significance and application demand.
2011, the researchist of much of Italian special human relations proposed a kind of image forge discrimination method, and the computing machine generating component that the method can will incorporate in the real scene detects.This work be in the known work on hand unique one take the augmented reality scene as processing object.But the detection that this work is carried out is not take object as unit, but only detects the virtual composition in the augmented reality scene, and namely testing result may be a zone, may be the point of scattered distribution yet.
The researchist of U.S. Dartmouth University in 2005 has proposed based on the natural image statistical model of wavelet decomposition and has adopted support vector machine and the classify method of virtual image and true picture of linear discriminant analysis.This at first extracts after the coloured image wavelet decomposition Fourth feature (average, variance, the degree of bias, kurtosis) of coefficient of dissociation on each subband and direction; Consider simultaneously the Fourth Order Linear predicated error feature between the adjacent coefficient of dissociation after the wavelet decomposition, then utilize support vector machine and linear discriminant analysis method to train sorter, the sorter that again the test set input is trained obtains classification results.The actual situation classification of the method is carried out for whole image, and classification accuracy has than great fluctuation process along with the extraction area size difference of actual situation characteristic of division.
2007, the researchist of USA New York University of Science and Technology proposed to utilize the color filter array interpolation to detect the method that aberration consistance in characteristics and the image is distinguished virtual image and true picture.The method is at first extracted color-based filter array interpolation and is detected the conforming feature of aberration in characteristics and the image from the positive negative sample of training set, then will train sorter in the feature input support vector machine of extracting, the sorter that again the test set input is trained obtains classification results.
The researchist of Alberta, Canada university in 2009 has proposed to utilize the classify method of virtual image and true picture of the consistance of image block resampling parameter.The party's ratio juris is based on virtual image may use the operations such as rotation to texture image, convergent-divergent to the process of model surface texture in generating, and causes the parameter that each image block resamples in the virtual image inconsistent.So just can whether unanimously distinguish virtual image and true picture by the parameter that the detected image piece resamples.The parameter estimation that the image block of the method resamples is to carry out for whole image.
2004, the researchist of Compag Computer's Cambridge Research Laboratories proposed to utilize based on the Ha Er wave filter and has adopted the AdaBoost sorting algorithm to carry out the method that people's face detects.The method is at first extracted characteristic of division from training set, retraining goes out the sorter based on people's face and non-face statistical nature, the characteristic of division of the image to be detected that then will extract input sorter and by cascade classifier reduce need the detection window that calculates number to raise the efficiency, finally obtain testing result.The feature extraction of the method is based on the Ha Er wave filter, description be the region contrast that people's face inherent structure brings.
2005, the researchist of La Photographie computing machine and automation research institute proposed to utilize histograms of oriented gradients and linear SVM to carry out the method for person detecting.The method is divided at first the input picture is carried out color normalization, then calculate the gradient in the picture, statistics drops on the pixel between different directions and azimuthal bin, and overlapping space piece compared normalization, the histograms of oriented gradients of each detection window of regeneration, sort out personage/inhuman object area with the linear SVM sorter at last, obtain testing result.The method has higher detection effect than other detection methods, but requires personage in the picture will roughly keep the state of standing.What the method feature extraction was adopted is the image gradient histogram, description be the inherent characteristics of human body contour outline.
The method of above-mentioned differentiation virtual image and true picture, common ground are that actual situation characteristic of division that they extract all is not suitable for the actual situation classification for any given zone in the image.In addition, in the work of existing object detection, the general object of processing has the stronger appearance characteristics that is easy to describe as prior imformation.Comparatively speaking, the virtual objects in the augmented reality scene detects, and it detects target (being virtual objects) and does not have the explicit in appearance prior imformation that is easy to describe, and such as color, shape, size etc., therefore differentiation and detection difficulty are larger.
Summary of the invention
Technical solution of the present invention: overcome the deficiencies in the prior art, provide a kind of augmented reality scene virtual objects based on degree of confidence figure to differentiate and detection method, the method does not need to know in advance any appearance information of virtual objects, such as color, shape, size, do not need to know virtual objects residing position in the augmented reality scene yet, but utilize the physics imaging difference of distinguishing virtual objects and true picture, carrying out the actual situation characteristic of division extracts, regional unique characteristics and the regional correlation feature of the positive negative sample of difference calculation training collection, and construct Pixel-level actual situation sorter and region class actual situation sorter; On this basis, carry out tentatively formalize location and accurately detecting of virtual objects by differentiating based on the virtual objects of virtual degree of confidence figure with detection.
The technical solution used in the present invention: the augmented reality scene virtual objects based on degree of confidence figure is differentiated and detection method, step is as follows: make up augmented reality scene training dataset, and utilize the physics imaging difference of virtual objects and true picture, choose the actual situation characteristic of division; On training dataset, utilize the actual situation characteristic of division, extract respectively the regional unique characteristics of augmented reality scene and real scene, make up Pixel-level actual situation sorter; On training dataset, utilize the actual situation characteristic of division, extract respectively the regional correlation feature of augmented reality scene and real scene, make up region class actual situation sorter; Given test augmented reality scene utilizes Pixel-level actual situation sorter and small size detection window to detect, and obtains reflecting the virtual shot chart of each pixel actual situation classification results; Defining virtual degree of confidence figure, and on the basis of virtual shot chart, utilize thresholding to obtain testing the virtual degree of confidence figure of augmented reality scene; According to the distribution situation of high virtual responsive point among the virtual degree of confidence figure, obtain rough shape and the position of virtual objects bounding box; On the basis of virtual objects coarse localization, in test augmented reality scene image, utilize region class actual situation sorter and large scale detection window to detect, obtain the final detection result of virtual objects.
Make up augmented reality scene training dataset.Concentrate at training data, will comprise the augmented reality scene image of virtual objects as positive sample, with the real scene image as negative sample.Utilize the physics imaging difference of virtual objects and true picture, choose the actual situation characteristic of division.The virtual class feature of choosing comprises: local statistic, surface graded, second fundamental form, Marco Beltrami stream.Can extract the above-mentioned actual situation characteristic of division that obtains this some correspondence at each pixel place of image.
On training dataset, utilize the actual situation characteristic of division, extract the regional unique characteristics of augmented reality scene, make up Pixel-level actual situation sorter.When making up the Pixel-level sorter, to the augmented reality scene image, only choose the virtual objects zone as positive sample areas; And to the real scene image, only choose with positive sample in the akin zone of virtual objects as negative sample zone.For given image-region, calculate the actual situation characteristic of division (comprising: local statistic, surface graded, second fundamental form, Marco Beltrami stream) of every bit in the zone; Utilize the moment of inertia compression method that the actual situation characteristic of division of given area is compressed, obtain the regional unique characteristics of this zone correspondence.The regional unique characteristics set input support vector machine classifier of positive negative sample is trained, obtain Pixel-level actual situation sorter.
On training dataset, utilize the actual situation characteristic of division, extract the regional correlation feature of augmented reality scene, make up region class actual situation sorter.For positive and negative sample areas, subject area to be determined will itself be considered as; And the homalographic rectangular area outside the regional bounding box is considered as the residing background area of object; Extract respectively the actual situation characteristic of division of every bit in subject area and the background area; The actual situation characteristic of division that all-pair is answered in objects of statistics zone and the background area consists of respectively the joint distribution histogram of subject area feature and the joint distribution histogram of background area feature; Calculate the card side's distance between two histograms, with its feature that is considered as weighing object background difference of living in it, be called the regional correlation feature; The regional correlation characteristic set input support vector machine classifier of the positive negative sample that extracts is trained, obtain region class actual situation sorter.
Virtual shot chart makes up, and step is the augmented reality scene image for input, utilize small size detection window (detection window is of a size of [10,30] * [10,30] pixel) with less moving step length (as 1,2,3,4,5} pixel) the scanning entire image; Calculate the regional unique characteristics of the little image block in each small size detection window; The regional unique characteristics of all little image blocks is input in the Pixel-level actual situation sorter, obtains the regional unique characteristics score of each little image block, the high expression of score Pixel-level sorter is high with the degree of certainty that this image block is categorized as virtual region; Because the relative entire image of size of detection window is very little and densely distributed, therefore the regional unique characteristics score of each little image block can be mapped to the center pixel of this image block, and with its virtual score as this central pixel point; Consisted of thus the virtual shot chart of whole augmented reality scene image.This process can improve counting yield by two-dimensional integration figure.
Virtual degree of confidence figure makes up, and step is that the virtual shot chart for the augmented reality scene image carries out thresholding and processes, and records all and virtual is divided into positive point; A fixing number percent N% is set, records all virtual front N% and these point residing positions on original image that are divided into positive point, these points are called high virtual responsive point; The constant M (as making M ∈ [10,100]) of a fixing and less is set, records all virtual front M point and these residing positions on original image that are divided into positive point, these points are called the highest virtual responsive point; Can guarantee that by parameter setting the highest virtual responsive point also is contained in the set at high virtual responsive point place simultaneously, namely the highest virtual responsive point is the virtual the highest part of score value that gets in the high virtual responsive point.Positional information on comprehensive high virtual responsive point, the highest virtual responsive point and the place original image thereof consists of virtual degree of confidence figure.
The rough shape of virtual objects bounding box and position reasoning step are as follows: to the virtual degree of confidence figure that obtains, with its be divided into five homalographics, can be overlapping subregion, try to achieve respectively the distribution center of the high virtual responsive point in every sub regions; The subregion center is considered as candidate's virtual objects central point, from each central point respectively outwards expanded search obtain the densely distributed zone of high virtual responsive point, for the densely distributed zone of high virtual responsive point, Approximate Calculation goes out the candidate target shape (being presented as candidate's virtual objects bounding box) in this zone, in conjunction with this regional positional information, consist of the preliminary candidate region of virtual objects; In the preliminary candidate region of a plurality of virtual objects, each self-contained high virtual responsive point and the number of high virtual responsive point according to it, select maximum one of weighting number, as the virtual objects candidate region, this zone has namely comprised virtual objects bounding box rough shape and positional information with it.
For the coarse localization of the virtual objects that obtains, further optimize, obtain the final detection result of virtual objects.Concrete steps are: get the zone that area is virtual objects candidate region twice around in the virtual objects candidate region, (range of size of large scale detection window is generally [200 to the structure form size a plurality of overlapped large scale detection window identical with the virtual objects candidate region in this zone, 500] * [200,500], the concrete value of its length and width equals length and the width of virtual objects bounding box in the virtual objects candidate region); Get the interior image block of each large scale detection window and calculate its regional correlation feature; The regional correlation feature input area level actual situation sorter of image block in all large scale detection window is classified, select the final detection result of the highest detection window of reserved portion as virtual objects.
The present invention compared with prior art, its beneficial effect is:
(1) virtual objects of the present invention in the augmented reality scene be as detected object, the virtual objects in the augmented reality scene can be done as a whole differentiation and detect.
(2) the present invention has made up two-stage actual situation sorter, comprises Pixel-level actual situation sorter and region class actual situation sorter, satisfies the demand that degree of confidence figure makes up and virtual objects finally detects.
(3) the present invention has built a degree of confidence figure, based on virtual degree of confidence figure, can under the condition that does not have the prior imformations such as virtual objects outward appearance, shape, position, draw virtual objects approximate location and shape in the augmented reality scene.
(4) the present invention does not need to know in advance any appearance information of virtual objects, such as prior imformations such as color, shape, sizes, do not need to know virtual objects residing position in the augmented reality scene yet, wider applicability is arranged, but widespread use is generalized to the fields such as production of film and TV, digital entertainment, educational training.
Description of drawings
Fig. 1 is overall design structure of the present invention;
Fig. 2 is that virtual degree of confidence figure of the present invention makes up process flow diagram;
Fig. 3 is virtual objects bounding box shape of the present invention, position reasoning process flow diagram;
Fig. 4 is the process flow diagram of the candidate's of obtaining central point of the present invention;
Fig. 5 is that expanded search of the present invention, the high virtual responsive of acquisition are put the process flow diagram in densely distributed zone.
Embodiment
As shown in Figure 1, key step of the present invention is as follows: make up augmented reality scene training dataset, and utilize the physics imaging difference of virtual objects and true picture, choose the actual situation characteristic of division; On training dataset, utilize the actual situation characteristic of division, extract the regional unique characteristics of augmented reality scene, make up Pixel-level actual situation sorter; On training dataset, utilize the actual situation characteristic of division, extract the regional correlation feature of augmented reality scene, make up region class actual situation sorter; Given test augmented reality scene is utilized Pixel-level actual situation sorter to carry out small scale and is detected, and obtains reflecting the virtual shot chart of each pixel actual situation classification results; Defining virtual degree of confidence figure, and on the basis of virtual shot chart, utilize thresholding to obtain testing the virtual degree of confidence figure of augmented reality scene; According to the distribution situation of high virtual responsive point among the virtual degree of confidence figure, obtain rough shape and the position of virtual objects bounding box; On the basis of virtual objects coarse localization, in test augmented reality scene image, utilize region class actual situation sorter and large scale detection window to detect, obtain the final detection result of virtual objects.
The structure training dataset is used for training actual situation sorter.Training dataset is made of as negative sample as positive sample, real scene image the augmented reality scene image that comprises virtual objects.When training Pixel-level sorter, to the augmented reality scene image, only choose the virtual objects zone as positive sample; And to the real scene image, only choose with positive sample in the akin zone of virtual objects as negative sample.In training during regional fraction class device, to the augmented reality scene image, choose virtual objects and on every side the image-region of homalographic as positive sample; And to the real scene image, choose with positive sample in the akin zone of virtual objects and on every side the image-region of homalographic as negative sample.
The extraction of zone unique characteristics.For given image-region, calculate the actual situation characteristic of division of every bit in the zone, comprising: local statistic, surface graded, second fundamental form, Marco Beltrami stream; Utilize the moment of inertia compression method that the actual situation characteristic of division of given area is compressed, obtain the regional unique characteristics of this zone correspondence.
Local statistic, surface graded, second fundamental form, Marco Beltrami stream physical significance and computing method thereof separately are as follows respectively:
What local statistic reflected is local small marginal texture.The computing method of local statistic are as follows: get any point P on the gray-scale map of original image, the little image block of 3 * 3 pixels centered by the P point is arranged in 9 dimensional vector x=[x in order with the pixel value of every bit wherein 1, x 2... x 9].The local statistic y that P is ordered is 9 dimensional vectors, and it is defined as:
y = x - x ‾ | | x - x ‾ | | D . Wherein, x ‾ = 1 9 Σ i = 1 9 x i ; And || || DIt is the operation of D norm.
The definition of D norm operation is:
Figure BDA0000095251690000063
Wherein the point of all neighbours territory relations is right in i～j presentation video piece.
The local statistic actual situation characteristic of division at p place, arbitrfary point is the 9 dimensional vector y at this some place.
Surface graded is the nonlinearities change characteristics of measuring in the real scene imaging process.The surface graded S at any point place is defined as in the image:
Figure BDA0000095251690000064
Wherein,
Figure BDA0000095251690000065
Be the image gradient mould value at this some place,
Figure BDA0000095251690000066
I x, I xThe local derviation of difference presentation video x direction (horizontal direction) and y direction (vertical direction).α is constant, α=0.25.
The surface graded actual situation characteristic of division at p place, arbitrfary point is united by the image pixel value I at this some place and the surface graded S in this some place and is consisted of.
Second fundamental form is the concavo-convex degree of Description Image surface local.Two component λ of second fundamental form 1And λ 2Two eigenwerts of difference homography A.
A = 1 1 + I x 2 + I y 2 I xx I xy I xy I yy , Wherein, I x, I xThe local derviation of difference presentation video x direction and y direction; I Xx, I Xy, I YyThe second order local derviation of difference presentation video xx direction, xy direction, yy direction; Can be calculated the value of matrix A by this formula.A might as well be designated as: A = a 11 a 12 a 21 a 22 , A wherein 11, a 12, α 21, α 22Four corresponding element values among the difference representing matrix A.Therefore, two of matrix A eigenvalue λ 1And λ 2Computing formula is as follows:
{ λ 1 , λ 2 } = a 11 + a 22 ± ( a 11 - a 22 ) 2 + 4 a 12 a 21 2 , λ 1 λ 2
The second fundamental form actual situation characteristic of division at p place, arbitrfary point is by the image gradient mould value at this some place
Figure BDA00000952516900000610
Two component λ with this some place second fundamental form 1, λ 2The associating formation.
Marco Beltrami stream can be used for describing the correlativity between the different color channels.The Marco Beltrami stream Δ that Color Channel c (c={R, G, B}) is corresponding gI cBe defined as:
Δ g I c = 1 | g | ( ∂ x ( | g | ( g xx ∂ x I c + g xy ∂ y I c ) ) ) + 1 | g | ( ∂ y ( | g | ( g yx ∂ x I c + g yy ∂ y I c ) ) )
Wherein, I cThe image corresponding to Color Channel c (c={R, G, B}) of expression original image; Operator
Figure BDA0000095251690000072
Represent respectively to measure for effect the local derviation of x direction and y direction;
Matrix g = 1 + ( I x R ) 2 + ( I x G ) 2 + ( I x B ) 2 I x R I y R + I x G I y G + I x B I y B I x R I y R + I x G I y G + I x B I y B 1 + ( I y R ) 2 + ( I y G ) 2 + ( I y B ) 2 ,
Figure BDA0000095251690000074
The local derviation of difference presentation video R passage (red channel) x direction and y direction;
Figure BDA0000095251690000075
The local derviation of difference presentation video G passage (green channel) x direction and y direction;
Figure BDA0000095251690000076
The local derviation of difference presentation video B passage (blue channel) x direction and y direction; | g| is the determinant of matrix g; And g Xx, g Xy, g Yy, g YxThen by g - 1 = g xx g xy g yx g yy Provide respectively, i.e. g Xx, g Xy, g Yy, g YxBe respectively four element values corresponding to inverse matrix of matrix g.
The Marco Beltrami stream actual situation characteristic of division at p place, arbitrfary point is by the Marco Beltrami flow point amount Δ of each Color Channel c (c={R, G, B}) at this some place gI cImage gradient mould value with each Color Channel The associating formation.
Behind four groups of actual situation characteristic of divisions of every bit in calculating the zone (comprising: local statistic, surface graded, second fundamental form, Marco Beltrami flow), need to utilize the moment of inertia compression method that the actual situation characteristic of division is compressed.Moment of inertia compression method step is as follows: consider that first separately local statistic, surface graded, second fundamental form, Marco Beltrami flow any one group (processing mode of each group actual situation characteristic of division is all identical) in these four groups of actual situation characteristic of divisions.If total N point in the given area, any point P i(i=1 ..., the total M dimension of one group of actual situation characteristic of division N) (value of M according in four groups of actual situation characteristic of divisions, get fixed wherein one group can determine), will put P i(i=1 ..., one group of actual situation characteristic of division N) is designated as v i=(v I1..., v Im).At this moment, will put P iActual situation characteristic of division v i=(v I1..., v Im) be considered as a particle in the M dimensional feature space, stipulate that the quality of this particle is
Figure BDA0000095251690000079
, the position coordinates of this particle in the M dimensional feature space is v i=(v I1..., v Im), then can calculate by the solid moment of inertia Matrix Formula moment of inertia matrix J of the system of particles that all N particles consist of.Moment of inertia matrix J is that a M * M ties up matrix, and matrix J can be write as following form:
Figure BDA00000952516900000710
Any one element of matrix J is designated as J Jk(j, k=1 ..., M).J JkComputing method be:
Figure BDA00000952516900000711
(j, k=1 ..., M).M wherein iExpression particle P iQuality,
Figure BDA00000952516900000712
v i=(v I1..., v Im) expression point P iPosition coordinates in feature space; | v i| expression particle P iTo the Euclidean distance of true origin, namely
Figure BDA0000095251690000081
δ JkBe Kronecker function, its computing method are δ jk = 1 , if i = j 0 , if i ≠ j . Can determine thus all elements J of moment of inertia matrix J Jk(j, k=1 ..., M).By the symmetry of moment of inertia matrix J as can be known, J Jk=J Kj, therefore only get principal diagonal and the above all elements J of principal diagonal of matrix J Jk(j, k=1 ..., M and j≤k), these elements can represent all information of original matrix J.
Get all elements J in the moment of inertia matrix Jk(j, k=1 ..., M and j≤k); The centroid vector of uniting all particles
Figure BDA0000095251690000083
Unite all particles and true origin distance | v i| average, variance, the degree of bias, kurtosis; Constitute a proper vector.This proper vector is the compression expression result that this group actual situation characteristic of division of having a few in this zone obtains through the moment of inertia compression method.With four groups of compression expression result associatings that obtain respectively, can obtain the regional unique characteristics of this zone correspondence.Because moment of inertia matrix can describe the distribution of a plurality of particles in feature space preferably, thus the moment of inertia matrix compression method can be when a plurality of high dimensional data points compress to the zone as far as possible low guaranteed keep largely the information that legacy data distributes.
The extraction of regional correlation feature.For given image-region, the zone itself is considered as subject area to be determined; And the homalographic rectangular area of next-door neighbour's bounding box outside the regional bounding box is considered as the residing background area of object; Calculate respectively the actual situation characteristic of division of every bit in subject area and the background area; The actual situation characteristic of division that all-pair is answered in the objects of statistics zone and the background area respectively consists of in the subject area and the joint distribution histogram of background area feature; Card side's distance between the joint distribution histogram of the joint distribution histogram of calculating object provincial characteristics and background area feature with its feature that is considered as weighing contrast between object and its background of living in or difference, is called the regional correlation feature.
Make up Pixel-level actual situation sorter and region class actual situation sorter, be used for dividing given zone whether to belong to the virtual objects region from the angle of regional unique characteristics and the angular area of regional correlation feature respectively.
Pixel-level actual situation sorter makes up, by the positive negative sample of input training set; Extract respectively the regional unique characteristics of positive negative sample; The regional unique characteristics set input support vector machine classifier of the positive negative sample that extracts is trained, obtain Pixel-level actual situation sorter.The characteristics of Pixel-level actual situation sorter are that the Feature Compression method that it adopts is so that its classification results tool has the dimensions adaptability.Namely when regional unique characteristics to be sorted be when being obtained by the extracted region significantly different from the area size of training set, Pixel-level actual situation sorter has preferably accuracy for the classification results whether given area belongs to the virtual objects region.Particularly: although the Pixel-level sorter is (to be of a size of [10 by virtual objects in the training set, 30] * [10,30] regional unique characteristics set training pixel) obtains, but experimental result shows: this sorter (is of a size of [10 for relatively little a lot of zone, 30] * [10,30] provincial characteristics pixel), its classification results still has preferably accuracy.Since the classification of this sorter to as if for the small size zone, and these zonules are used for the approximate description regional center and put corresponding pixel, therefore this sorter are become Pixel-level actual situation sorter.
Region class actual situation sorter makes up, by the positive negative sample of input training set; Extract respectively the regional correlation feature of positive negative sample; The regional correlation characteristic set input support vector machine classifier of the positive negative sample that extracts is trained, obtain region class actual situation sorter.Because the characteristic of division that region class actual situation sorter uses is the population distribution difference between conversion zone and the place background thereof, therefore object to be detected can be done as a wholely to differentiate preferably and detect.
Make up virtual shot chart.For the input the augmented reality scene image, utilize small size detection window (detection window is of a size of [10,30] * [10,30] pixel) with less moving step length (as 1,2,3,4,5} pixel) scanning entire image; Calculate the regional unique characteristics of the little image block in each small size detection window; The regional unique characteristics of all little image blocks is input in the Pixel-level actual situation sorter, obtains the regional unique characteristics score of each little image block, the high expression of score Pixel-level sorter is high with the degree of certainty that this image block is categorized as virtual region; Because the relative entire image of size of detection window is very little and densely distributed, therefore the regional unique characteristics score of each little image block can be mapped to the center pixel of this image block, and with its virtual score as this central pixel point; Consisted of thus the virtual shot chart of whole augmented reality scene image.Because the actual situation characteristic of division during zone self sign is calculated calculates and the Feature Compression operation is more consuming time, and need to calculate one by one its regional unique characteristics for a large amount of overlapped image block that generates, therefore in this step, adopt integrogram method speed-up computation process.The virtual shot chart that obtains thus makes up point and this point relation that whether belong to virtual objects of result on can the reflected well image.That is: experimental result shows: in the virtual shot chart, the point that virtual score is high generally all concentrates on the virtual objects region; Otherwise, the point of virtual objects region, corresponding virtual score is all higher.
Make up virtual degree of confidence figure, its flow process as shown in Figure 2.At first, carry out thresholding for the virtual shot chart that obtains and process, select first and record all virtual to be divided into positive point; A fixing number percent N% is set, selects and record all virtual front N% and these point residing positions on original image that are divided into positive point.These points are called high virtual responsive point.The constant M (as making M ∈ [10,100]) of a fixing and less is set, selects and record all virtual front M point and these residing positions on original image that are divided into positive point.These points are called the highest virtual responsive point.The number of high virtual responsive point is much smaller than the number of high virtual responsive point.Positional information on comprehensive high virtual responsive point, the highest virtual responsive point and the place original image thereof namely consists of virtual degree of confidence figure.Described virtual degree of confidence figure makes up point and this point relation that whether belong to virtual objects of result on can the reflected well image.That is: experimental result shows: among the virtual degree of confidence figure, high virtual responsive point generally all concentrates on the virtual objects region; Otherwise, the point of virtual objects region, corresponding high virtual responsive point distributes comparatively intensive.Similarly, the highest virtual responsive point generally only appears at the virtual objects region; Otherwise, in the virtual objects region, the highest more virtual responsive point generally can appear.For appearing at the virtual objects zone in high virtual responsive point and the highest virtual responsive point in addition, be referred to as noise spot.
The rough shape of virtual objects bounding box and position reasoning flow process may further comprise the steps as shown in Figure 3: divide subregion, obtain candidate's central point; Expanded search obtains high virtual responsive and puts densely distributed zone; The preliminary candidate region of virtual objects is determined; The virtual objects candidate region is determined.Wherein particularly, dividing subregion, is the virtual degree of confidence figure to obtaining, with its be divided into five homalographics, can be overlapping subregion.Obtain the process flow diagram of candidate's central point as shown in Figure 4, respectively according to the distribution of the high virtual responsive point in every sub regions, utilize mean shift algorithm to try to achieve high virtual responsive point center of distribution point in every sub regions, this central point is called candidate's central point.Candidate's central point number is k (k≤5, k less than 5 situation corresponding to there not being high virtual responsive point in some subregion), must be in above-mentioned k candidate's central point at this central point that might as well suppose virtual objects institute corresponding region.Expanded search, obtain high virtual responsive and put densely distributed zone, this process is as shown in Figure 5: for each candidate's central point, take candidate's central point as the center of circle, take the length that increases according to fixed step size as radius, the border that has searched the virtual objects zone until when the number of high virtual responsive point no longer increases in the current search zone, then can be thought in the circular region of search that dynamic construction increases successively; Ideally, the condition that expanded search stops is when search radius increases, the number increment of high virtual responsive point is zero in the region of search, but in order to eliminate the impact of the noise spot that exists among the virtual degree of confidence figure, a squelch parameter is set, the condition reinforcement that expanded search is stopped is: when search radius increased, the number increment of high virtual responsive point must be greater than the squelch parameter in the region of search.
The preliminary candidate region of virtual objects is determined to go out candidate target shape in this zone by the densely distributed regional Approximate Calculation of high virtual responsive point, in conjunction with this regional positional information, consists of the preliminary candidate region of virtual objects.When expanded search stops, obtaining high virtual responsive and put densely distributed zone, the set P of all high virtual responsive points in as can be known should the zone can draw candidate target shape in this zone according to P, namely is presented as the shape of candidate target bounding box:
x min＝min({x|<x，y>∈P})；x max＝max({x|<x，y>∈P})；
y min＝min({y|<x，y>∈P})；y max＝max({y|<x，y>∈P})；
X wherein Min, x MaxRepresent respectively x direction minimum value and maximal value in the correspondence image coordinate of candidate target bounding box region; y Min, y MaxRepresent respectively y direction minimum value and maximal value in the correspondence image coordinate of candidate target bounding box position.Can determine that thus the candidate target bounding box is with respect to the position in the image and shape.
Candidate target shape in this zone in conjunction with this regional positional information (candidate's center position), consists of the preliminary candidate region of virtual objects.
In the preliminary candidate region of the k that an obtains virtual objects, each self-contained high virtual responsive point and the number of high virtual responsive point according to it, select maximum one of weighting number, as the virtual objects candidate region, this zone has namely comprised the general shape of virtual objects bounding box and the information of position with it.
For the coarse localization of the virtual objects that obtains, further optimize, with the error that reduces may occur in the computation process of virtual objects candidate region, thereby obtain the final detection result of virtual objects.Concrete steps are: get the zone that area is virtual objects candidate region twice around in the virtual objects candidate region; (range of size of large scale detection window is generally [200 to the structure form size a plurality of overlapped large scale detection window identical with the virtual objects candidate region in this zone, 500] * [200,500], the concrete value of its length and width equals length and the width of virtual objects bounding box in the virtual objects candidate region); Get the interior image block of each large scale detection window and calculate its regional correlation feature; The regional correlation feature input area level actual situation sorter of image block in all large scale detection window is classified, select the final detection result of the highest detection window of reserved portion as virtual objects.
The above only is basic explanations more of the present invention, and any equivalent transformation according to technical scheme of the present invention is done all should belong to protection scope of the present invention.
The non-elaborated part of the present invention belongs to techniques well known.

Claims (1)

1. differentiate and detection method based on the augmented reality scene virtual objects of degree of confidence figure, it is characterized in that performing step is as follows:
(1) with the augmented reality image that comprises virtual objects as positive sample, the real scene image makes up augmented reality scene training dataset as negative sample; And utilize the physics imaging difference of virtual objects and true picture, choose the actual situation characteristic of division;
(2) on training dataset, utilize the actual situation characteristic of division, extract respectively the regional unique characteristics of augmented reality scene and real scene, make up Pixel-level actual situation sorter;
(3) on training dataset, utilize the actual situation characteristic of division, extract respectively the regional correlation feature of augmented reality scene and real scene, make up region class actual situation sorter;
(4) given test augmented reality scene utilizes Pixel-level actual situation sorter and small size detection window to detect, and obtains reflecting the virtual shot chart of each pixel actual situation classification results;
(5) defining virtual degree of confidence figure, and on the basis of virtual shot chart, utilize thresholding to obtain testing the virtual degree of confidence figure of augmented reality scene;
(6) based on virtual degree of confidence figure, carry out the virtual objects coarse localization, obtain rough shape and the position of virtual objects bounding box;
(7) on the basis of virtual objects coarse localization, in test augmented reality scene image, utilize region class actual situation sorter and large scale detection window to detect, obtain the final detection result of virtual objects;
The actual situation characteristic of division of choosing in the described step (1) comprises: local statistic, surface graded, second fundamental form and Marco Beltrami stream, can both extract the above-mentioned actual situation characteristic of division that obtains this point correspondence at each pixel place of image;
When making up the Pixel-level sorter in the described step (2), on training dataset, to the augmented reality scene image, only choose the virtual objects zone as positive sample areas; And to the real scene image, only choose with positive sample in the akin zone of virtual objects as negative sample zone; For given image-region, calculate the actual situation characteristic of division of every bit in the zone; Utilize the moment of inertia compression method that the actual situation characteristic of division of given positive and negative sample areas is compressed, obtain the regional unique characteristics of this zone correspondence; The regional unique characteristics set input support vector machine classifier of positive negative sample is trained, obtain Pixel-level actual situation sorter;
When described step (3) makes up region class actual situation sorter, on training dataset, for positive and negative sample areas, subject area to be determined will itself be considered as; And the homalographic rectangular area outside the regional bounding box is considered as the residing background area of object; Extract respectively the actual situation characteristic of division of every bit in subject area and the background area; The actual situation characteristic of division that all-pair is answered in objects of statistics zone and the background area consists of respectively the joint distribution histogram of subject area feature and the joint distribution histogram of background area feature; Calculate the card side's distance between two histograms, with its feature that is considered as weighing object background difference of living in it, be called the regional correlation feature; The regional correlation characteristic set input support vector machine classifier of the positive negative sample that extracts is trained, obtain region class actual situation sorter;
The virtual shot chart construction step of described step (4) is: for the augmented reality scene image of input, utilize the small size detection window with less moving step length scanning entire image; Calculate the regional unique characteristics of the little image block in each small size detection window; The regional unique characteristics of all little image blocks is input in the Pixel-level actual situation sorter, obtains the regional unique characteristics score of each little image block, the high expression of score Pixel-level sorter is high with the degree of certainty that this image block is categorized as virtual region; Because the relative entire image of size of detection window is very little and densely distributed, therefore the regional unique characteristics score of each little image block can be mapped to the center pixel of this image block, and with its virtual score as this central pixel point; Consisted of thus the virtual shot chart of whole augmented reality scene image;
The virtual degree of confidence figure construction method of described step (5) is: carry out thresholding for the virtual shot chart of augmented reality scene image and process, record all and virtual be divided into positive point; A fixing number percent N% is set, records all virtual front N% and these point residing positions on original image that are divided into positive point, these points are called high virtual responsive point; The constant M of a fixing and less is set, records all virtual front M point and these residing positions on original image that are divided into positive point, these points are called the highest virtual responsive point; Can guarantee that by parameter setting the highest virtual responsive point also is contained in the set at high virtual responsive point place simultaneously, namely the highest virtual responsive point is the virtual the highest part of score value that gets in the high virtual responsive point; Positional information on comprehensive high virtual responsive point, the highest virtual responsive point and the place original image thereof consists of virtual degree of confidence figure;
Described step (6) obtains the rough shape of virtual objects bounding box and the method for position is: to the virtual degree of confidence figure that obtains with its be divided into five homalographics, can be overlapping subregion, try to achieve respectively the distribution center of the high virtual responsive point in every sub regions; The subregion center is considered as candidate's virtual objects central point, from each central point outside expanded search respectively, obtains the densely distributed zone of high virtual responsive point; For the densely distributed zone of high virtual responsive point, Approximate Calculation goes out the candidate target shape in this zone respectively, in conjunction with this regional positional information, consists of the preliminary candidate region of virtual objects; In the preliminary candidate region of virtual objects, each self-contained high virtual responsive point and the number of high virtual responsive point according to it, select maximum one of weighting number, as the virtual objects candidate region, this zone has namely comprised virtual objects bounding box rough shape and positional information with it;
Described step (7) virtual objects detection method is specially: intensive sampling around the virtual objects candidate region in test augmented reality scene, construct a plurality of overlapped detection window, and use region class actual situation sorter to classify, choose the best detection window of score as the final detection result of virtual objects.
CN 201110299857 2011-09-30 2011-09-30 Confidence map-based method for distinguishing and detecting virtual object of augmented reality scene Expired - Fee Related CN102509104B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201110299857 CN102509104B (en) 2011-09-30 2011-09-30 Confidence map-based method for distinguishing and detecting virtual object of augmented reality scene

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201110299857 CN102509104B (en) 2011-09-30 2011-09-30 Confidence map-based method for distinguishing and detecting virtual object of augmented reality scene

Publications (2)

Publication Number Publication Date
CN102509104A CN102509104A (en) 2012-06-20
CN102509104B true CN102509104B (en) 2013-03-20

Family

ID=46221185

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201110299857 Expired - Fee Related CN102509104B (en) 2011-09-30 2011-09-30 Confidence map-based method for distinguishing and detecting virtual object of augmented reality scene

Country Status (1)

Country Link
CN (1) CN102509104B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11830214B2 (en) * 2018-06-01 2023-11-28 Apple Inc. Methods and devices for detecting and identifying features in an AR/VR scene

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102798583B (en) * 2012-07-13 2014-07-30 长安大学 Ore rock block degree measurement method based on improved FERRET
EP2932708B1 (en) 2013-05-28 2017-07-19 Hewlett-Packard Enterprise Development LP Mobile augmented reality for managing enclosed areas
WO2015062164A1 (en) * 2013-10-31 2015-05-07 The Chinese University Of Hong Kong Method for optimizing localization of augmented reality-based location system
CN105654504A (en) * 2014-11-13 2016-06-08 丁业兵 Adaptive bandwidth mean value drift object tracking method based on rotary inertia
CN104794754B (en) * 2015-05-12 2018-04-20 成都绿野起点科技有限公司 A kind of Distributed Virtual Reality System
CN104869160B (en) * 2015-05-12 2018-07-31 成都绿野起点科技有限公司 A kind of Distributed Virtual Reality System based on cloud platform
CN104780180B (en) * 2015-05-12 2019-02-12 国电物资集团有限公司电子商务中心 A kind of Virtual Reality Platform based on mobile terminal
CN108492374B (en) * 2018-01-30 2022-05-27 青岛中兴智能交通有限公司 Application method and device of AR (augmented reality) in traffic guidance
CN111739084B (en) * 2019-03-25 2023-12-05 上海幻电信息科技有限公司 Picture processing method, atlas processing method, computer device, and storage medium
CN112270063B (en) * 2020-08-07 2023-03-28 四川航天川南火工技术有限公司 Sensitive parameter hypothesis testing method for initiating explosive system
CN115346002B (en) * 2022-10-14 2023-01-17 佛山科学技术学院 Virtual scene construction method and rehabilitation training application thereof
CN117315375B (en) * 2023-11-20 2024-03-01 腾讯科技（深圳）有限公司 Virtual part classification method, device, electronic equipment and readable storage medium

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101520904B (en) * 2009-03-24 2011-12-28 上海水晶石信息技术有限公司 Reality augmenting method with real environment estimation and reality augmenting system
WO2011084720A2 (en) * 2009-12-17 2011-07-14 Qderopateo, Llc A method and system for an augmented reality information engine and product monetization therefrom
CN101893935B (en) * 2010-07-14 2012-01-11 北京航空航天大学 Cooperative construction method for enhancing realistic table-tennis system based on real rackets

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11830214B2 (en) * 2018-06-01 2023-11-28 Apple Inc. Methods and devices for detecting and identifying features in an AR/VR scene

Also Published As

Publication number Publication date
CN102509104A (en) 2012-06-20

Similar Documents

Publication Publication Date Title
CN102509104B (en) Confidence map-based method for distinguishing and detecting virtual object of augmented reality scene
CN104134071B (en) A kind of deformable part model object detecting method based on color description
CN104063702B (en) Three-dimensional gait recognition based on shielding recovery and partial similarity matching
CN102629328B (en) Probabilistic latent semantic model object image recognition method with fusion of significant characteristic of color
CN104536009B (en) Above ground structure identification that a kind of laser infrared is compound and air navigation aid
CN104978567B (en) Vehicle checking method based on scene classification
CN101996401B (en) Target analysis method and apparatus based on intensity image and depth image
CN104166841A (en) Rapid detection identification method for specified pedestrian or vehicle in video monitoring network
CN106529499A (en) Fourier descriptor and gait energy image fusion feature-based gait identification method
CN102722891A (en) Method for detecting image significance
CN105528794A (en) Moving object detection method based on Gaussian mixture model and superpixel segmentation
CN106780552B (en) Anti-shelter target tracking based on regional area joint tracing detection study
CN107784663A (en) Correlation filtering tracking and device based on depth information
CN106250895A (en) A kind of remote sensing image region of interest area detecting method
CN101551852B (en) Training system, training method and detection method
CN105160317A (en) Pedestrian gender identification method based on regional blocks
CN105893946A (en) Front face image detection method
CN111915583B (en) Vehicle and pedestrian detection method based on vehicle-mounted thermal infrared imager in complex scene
CN102289948A (en) Multi-characteristic fusion multi-vehicle video tracking method under highway scene
CN104182765A (en) Internet image driven automatic selection method of optimal view of three-dimensional model
CN106023257A (en) Target tracking method based on rotor UAV platform
CN103186775A (en) Human body motion recognition method based on mixed descriptor
CN106682641A (en) Pedestrian identification method based on image with FHOG- LBPH feature
CN101655914A (en) Training device, training method and detection method
CN101477626A (en) Method for detecting human head and shoulder in video of complicated scene

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20130320

Termination date: 20150930

EXPY Termination of patent right or utility model