CN103310219B - The precision assessment method of registration with objects shape and equipment, the method and apparatus of registration - Google Patents

Abstract

The application provides the method and apparatus of precision of object shapes of assessment registration, the method and apparatus of registration with objects shape.The method of the precision of the object shapes of a kind of registration represented by multiple fixed points for assessing in image, the method includes: the first mark calculation procedure, uses the first model to calculate the first mark according to object shapes；Second mark calculation procedure, uses the second model to calculate the second mark according to each in the plurality of fixed point；And appraisal procedure, the precision of registration is calculated according to the first mark and the second mark.

Description

The precision assessment method of registration with objects shape and equipment, the method and apparatus of registration

Technical field

The present invention relates to image procossing, computer vision and pattern recognition.More particularly, the present invention relate to assessment figure The method and apparatus of the precision of the object shapes of the registration represented by multiple fixed points in Xiang and for registrating in image The method and apparatus of object shapes.

Background technology

At such as facial recognition, Expression analysis, 3D D facial modelling and many computer vision fields of face cartoon making In, automatically and accurately registrating the shape by the object (such as, face) of the set description of fixed point is important task.

Face is registrated, it has been proposed that many different types of methods.Among those methods, by Cootes et al. The active shape model (ASM) and the active apparent model (AAM) that propose are proved to be effective scheme.Compared with AAM, ASM exists Speed, precision and generalization properties have better performance.Therefore, in recent years for ASM have been proposed that a lot of improve and Change.

Active apparent model (ASM) is the object shapes developed in nineteen ninety-five by Tim Cootes and Chris Taylor Statistical model, it is iterated modification with the example matching with the object in new images.The shape of object is by (passing through shape Control) set expression of point.ASM algorithm is intended to mate this model with new images.It is by being alternately performed following steps Work:

Image around each is found the more good position for this point；

Update model parameter most preferably to mate with these newfound positions.

According to ASM, reality exists many algorithms for face portion registration.Coming in, face registration is the most expanded To multiple view registration.Have been carried out much work and extend these algorithms to process various visual angles face.

In general, the state of conventional multiview face method for registering is method based on visual angle, and wherein, visual angle is distributed The some sub-visual angle of such as front, half side-view and full side etc..Fig. 2 illustrates the example of various visual angles face registration, wherein visual angle quilt It is divided into seven sub-visual angles, and for every sub-visual angle, training shapes model and the set of local texture model.The method Deform by selecting correct model based on visual angle to tackle the non-linear shape to visual angle face.

The universal method of multiple view registration is that face visual angle is estimated.The shape at each visual angle can be stored.Such as Figure 1A Shown in, for input picture, first pass through visual angle method of estimation and estimate face visual angle, then by corresponding with the visual angle estimated Model perform face registration.Figure 1A illustrates the process of the universal method for being carried out various visual angles face registration by ASM method Sequence.

But, the visual angle of this universal method is estimated to be frequently not satisfactorily.Now, in order to more preferable visual angle is estimated, Some current multiple view registration methods have caused concern.

Method 1: the method estimated based on shape and view parameter

At L.Zhang, H.Ai, Multi-View Active Shape Model with Robust Parameter In Estimation, ICPR, 2006, it is proposed that the method for parameter estimation estimated for various visual angles active shape.First, the most right In each visual angle training shapes model and the set of local texture model.Secondly, estimate the initial visual angle of given image, and Perform to use the Local Search of local texture model.Then, parameter estimation is carried out by nonlinear optimization method.Use optimization side Method, in this optimization method, carries out dynamic weighting to each characteristic point so that only consistent with shape characteristic point will There is big weight, and the impact of exceptional value (outlier) will be eliminated.Finally, it is possible to obtain new shape is until this shape Till convergence.Figure 1B illustrates the flow chart of the various visual angles face method for registering utilizing parameter estimation.

Method 2: the method based on ASM and 3D mask based on mixing visual angle

At Yanchao Su, Haizhou Ai, Shihong Lao, Multi-View Face Alignment Using 3D Shape Model for View Estimation.The 3rd IAPR International Conference on Biometrics, in 2009, scans the simple 3D face shape mould of foundation on face to ASM based on visual angle with at 500 3D Type is combined, to set up fully automated various visual angles face registration arrangement.Initiated by various visual angles face detector, first by base Local texture model in visual angle carrys out the characteristic point around Local Search original shape, then uses 3D face shape model from this A little some reconstruct 3D face shape.3D shape according to reconstruct, it is possible to obtain its facial information, can refer to according to this facial information Show the point of self-enclosed (self-occluded), then use the 2D shape at this visual angle to come by Non-linear parameter estimation essence The non-close shape that refinement is observed.The whole registration process of method 1 and method 2 is essentially identical.Main difference is that and make in method 1 Parameter estimation is carried out with 3D face shape model.Fig. 1 C illustrates flow chart.Fig. 1 C illustrates the main flow chart of both methods 1 and 2 (method 2 has dotted box and method 1 does not has).For method 2 whole registration process shown in Fig. 1 D.Fig. 1 D is to registrate The illustration of journey.In (a), by the average shape of current visual angle, algorithm is initialized.In (b), use local grain Information obtains the shape observed by Local Search.In (c), utilize the shape observed, use 3D shape reconstruct 3D shape Shape, and estimate attitude.In (d), from the Shape Reconstruction 2D shape observed.In (e), obtain end form when iteration convergence Shape.

Method 1 and the problem of method 2:

One common issue of said method is that they need to estimate the sub-view of the object in each iteration.But, depending on That angular estimation itself is very difficult to and be prone to make mistakes.Although these visual angle methods of estimation are not to depend entirely on to be detected by face The initial visual angle that device obtains, but the inaccuracy that finally registrates is estimated to cause in visual angle.

Method 3: method based on Bayesian mixture models

At Y.Zhou, W.Zhang, X.Tang, and H.Shum.A bayesian mixture model for Multi-view face alignment.CVPR, in 2005, it is proposed that for the multi-mode Bayes of various visual angles face registration Framework.First, visual angle is initialized by various visual angles face detector.Secondly, according to initial visual angle, use mixed model to describe Distribution of shapes and some visuality, the posteriority being then led off model parameter when the observed value of given unknown validity feature point is general Rate.Specifically, the problem of multi-mode and alterable features point is expressed as unified Bayesian frame.Finally, EM algorithm is given Estimate the visuality of the point of model parameter, the shape of regularization and this shape.Fig. 1 E illustrates that this local updating processes.

The problem of method 3:

On the one hand, in method 3, the texture model used in the Local Search of each labelling point depends on initially regarding Angle, and obtain initial visual angle by face detector.But, it is very sensitive generally for the estimation of hiding view parameter. When not having correctly predicted initial visual angle, the result of Local Search becomes unreliable.If the estimation of form parameter does not process Potential exceptional value, then the method will failure.On the other hand, in local updating step, use two with different weights Model carries out local updating.One problem is the 3D shape that linear mixed model can not describe face fully, and this may lead Cause final result is inaccurate.

These methods process the non-linear shape modification of various visual angles face by selecting correct model based on visual angle. Texture owing to using in the local matching at each fixed point of given shape model depends on its visual angle classification, so these Method is very sensitive for the estimation of initial visual angle classification.But, accurately estimating of face visual angle itself is to solve the most very well Problem and still in development.Although the overlapping definition of angular field of view can alleviate due to visual angle not to a certain extent The mistake suitably initialized and cause, but if initial visual angle is not predicted correctly, then the result of local matching will become It is unreliable to obtain.

Thus, the most effectively selecting correct model is the committed step in various visual angles face based on visual angle registration.From Knowable to above-mentioned, it is extremely difficult for selecting optimal visual angle before models fitting.Therefore, after matching, Model Selection is performed It can be the effective scheme for this problem.According to fitting result select best model question essence on be how to assess from The problem of the precision of the fitting result of different models.

The several methods for assessing the precision that face registrates are there are.In one of these methods, at Face Alignment via Component-based Discriminative Search, ECCV, 2008, L.Liang, R.Xiao, In F.Wen, J.Sun, Boosted apparent model (BAM) is used to calculate each access shape { S₀..., S_kMark f (S_k), And choose the mark of maximum as final output.Posterior probability P (S | I) optimized for form parameter also can be considered shape The assessment of the wellness of registration.

In the case of the given face data collection with actual value demarcation, train grader based on boosting to learn (Fig. 3 A, from Xiaoming Liu.Generic face alignment using to practise deformation pattern based on actual value demarcation Boosted appearance model.CVPR, 2007) (positive class) and the change of fixed point based on random disturbance (perturb) Decision boundaries between shape image (negative class).One group of housebroken Weak Classifier of rectangular characteristic based on Haar-like determines Boosted apparent model.

Classification confidence score from last strong grader is considered to registrate the measurement of the wellness of shape.

Fig. 3 A illustrates the deformation pattern of the disturbance for BAM training.The disturbance of the negative sample in training is to the performance of BAM Affect bigger.Fig. 3 B is (from Xiaoming Liu.Generic face alignment using boosted Appearance model.CVPR, 2007) the boosted apparent model for face registration is shown.

Use correct registration and two kinds of apparent models trained in BAM of incorrect registration.Face considered as a whole and Positive and negative sample is deformed into 30 × 30 pixels.This works well, but throws away the information come in handy.It addition, in BAM Apparent model can not be applied to the assessment of multiple model.

Face registration is considered as classification problem by the method.Synthesized by the random disturbance of form parameter and train for BAM Negative shape.The selection of negative shape depend heavilys on the experience of user.If negative shape is not by suitable disturbance, then this will lead Cause excessively poor ground performance.If carrying out disturbance with substantial amounts of disturbance to bear shape, then this will cause good classification performance, and two classes The surplus of (that is, positive classification and negative classification) will be not big.It can be seen that the disturbance of negative sample in Xun Lian is for the property of BAM Can there is a more impact, and there are some surplus regions that sample is considered ambiguous.

Summary of the invention

Being excited by the problem of said method, the present invention concentrates on the fitting result of the model for different visual angles classification Select.

In one aspect of the invention, it is provided that a kind of for assessing the registration represented by multiple fixed points in image The method of the precision of object shapes.The method includes: the first mark calculation procedure, uses the first model to calculate according to object shapes First mark；Second mark calculation procedure, uses the second model to calculate second point according to each in the plurality of fixed point Number；And appraisal procedure, the precision of registration is calculated according to the first mark and the second mark.

In another aspect of this invention, it is provided that a kind of method of object shapes for registrating in image.The method includes: Shape step of registration, according to the multiple shape matching object shapes for object, each in the plurality of shape The individual image corresponding to a visual angle from object；Select step, pass through according to aspects of the present invention based on fitting result Appraisal procedure calculates the mark of the fitting result of each shape, and selects the fitting result corresponding with highest score to make Registration result for object shapes.

In still another aspect of the invention, it is provided that a kind of registration represented by multiple fixed points assessed in image to pictograph The equipment of the precision of shape.This equipment includes: the first mark calculates device, for using the first model to calculate the according to object shapes One mark；Second mark calculates device, for using the second model to calculate second according to each in the plurality of fixed point Mark；And apparatus for evaluating, for calculating the precision of registration according to the first mark and the second mark.

In still another aspect of the invention, it is provided that the equipment of a kind of object shapes for registrating in image.This equipment includes: Shape registration apparatus, is used for according to the multiple shape matching object shapes for object, in the plurality of shape Each is corresponding to the image at a visual angle from object；And selection device, for passing through according to this based on fitting result The equipment of the aspect of invention calculates the mark of the fitting result of each shape, and selects the matching corresponding with highest score Result is as the registration result of object shapes.

According to the present invention, find the best matching result after performing the registration of all shapes.In the present invention, nothing The sub-visual angle of iterative estimate and matching is needed not to rely on initial visual angle.The present invention for select correct visual angle shape and The shape removing incorrect visual angle is useful especially.

Utilize the method for the precision of the object shapes for assessing the registration represented by multiple fixed points in image and set Standby and for the method and apparatus registrating the object images in image, the precision of object registration in image can be improved, and And best fit shape can be selected with low calculating cost.

From below with reference to the accompanying drawing explanation to exemplary embodiment, further characteristic of the invention will be apparent from.

Accompanying drawing explanation

Figure 1A is shown through the process sequence that ASM method carries out the universal method of various visual angles face registration.

Figure 1B to 1E is shown respectively the flow chart utilizing parameter estimation to carry out the conventional method that various visual angles face registrates.

Fig. 2 illustrates the example of various visual angles face registration.

Fig. 3 A illustrates the deformation pattern of the disturbance for BAM training.

Fig. 3 B illustrates the boosted apparent model for face registration.

Fig. 4 illustrates that the block diagram of the hardware configuration of the computer system of the information output device that can realize the present invention.

Fig. 5 illustrates the block diagram of the equipment of the various visual angles object registration according to the present invention.

Fig. 6 illustrates the flow chart of the object registration according to the present invention.

Fig. 7 illustrates the figure of the selection of candidate shape model.

Fig. 8 illustrates the assessment equipment of the precision of the object shapes for assessing the registration in image.

Fig. 9 illustrates for registration by the flow chart of the precision assessment method of the object shapes of the set expression of fixed point.

Detailed description of the invention

To describe the preferred embodiments of the present invention in detail according to accompanying drawing now.Note, the relative cloth of the assembly in embodiment Put the shape with device and be merely described as example, and be not intended as limiting the scope of the invention to these examples.Additionally, phase As reference refer to similar item in the drawings with letter, thus, if definition one in one drawing, then without for rear Continuous figure discusses this.

Fig. 4 is the block diagram of the hardware configuration of the computer system 1000 illustrating the ability to realize embodiments of the invention.

As shown in Figure 4, computer system includes computer 1110.Computer 1110 includes connecting via system bus 1121 Processing unit 1120, system storage 1130, irremovable non-volatile memory interface 1140, removable non-volatile deposit Memory interface 1150, user's input interface 1160, network interface 1170, video interface 1190 and output peripheral interface 1195.

System storage 1130 includes ROM (read only memory) 1131 and RAM (random access memory) 1132.BIOS (basic input output system) 1133 resides in ROM 1131.Operating system 1134, application program 1135, other program unit 1136 reside in RAM 1132 with some routine data 1137.

The irremovable nonvolatile memory 1141 of such as hard disk etc is connected to irremovable nonvolatile memory Interface 1140.Irremovable nonvolatile memory 1141 such as can store operating system 1144, application program 1145, other Program unit 1146 and some routine data 1147.

Such as the removable non-volatile memory of floppy disk 1151 and CD-ROM drive 1155 etc is connected to Removable non-volatile memory interface 1150.Such as, diskette 1 152 can be inserted in floppy disk 1151, and CD (CD) 1156 can be inserted in CD-ROM drive 1155.

Such as the input equipment of mouse 1161 and keyboard 1162 etc is connected to user's input interface 1160.

Computer 1110 can be connected to remote computer 1180 by network interface 1170.Such as, network interface 1170 Remote computer 1180 can be connected to by LAN 1171.Alternatively, network interface 1170 is connectable to modulation /demodulation Device (modulator-demodulator) 1172, and modem 1172 is connected to remote computer 1180 via wide area network 1173.

Remote computer 1180 can include the memorizer 1181 of such as hard disk etc, and it stores remote application 1185。

Video interface 1190 is connected to monitor 1191.

Output peripheral interface 1195 is connected to printer 1196 and speaker 1197.

Computer system shown in Fig. 4 is merely illustrative and is never intended to enter invention, its application, or uses Any restriction of row.

Computer system shown in Fig. 4 can be implemented in any embodiment, it is possible to as stand-alone computer, it is also possible to makees For the processing system in equipment, it is possible to remove one or more unnecessary assembly, or be added to one or more additional Assembly.

Equipment for object registration according to the present invention and method be will now be described.

Fig. 5 is the block diagram of the equipment for various visual angles object registration according to the present invention.

As it is shown in figure 5, the equipment for object registration includes: model storage 501, for the object that storage is to be registrated Multiple shapes, each shape is corresponding to the sub-view of object；Shape registration apparatus 502, for solely according to Multiple shape matching object shapes, and export the fitting result of each model；And registration result selects device 503, For after a while the matching that the apparatus for evaluating of the present invention described calculates each shape being tied by basis based on fitting result The mark of fruit, and from all results of the fitting result as object shapes, select optimum.

Utilize this for the equipment of object registration, once input face image, face-image is placed for every height The corresponding average shape of view, described sub-view includes positron view, left half son's view, left complete sub-view, right half son's view, the right side Complete sub-view etc..Average shape is obtained from the correspondingly-shaped model being stored in model storage.As example, from for front The shape of sub-view obtains the average shape for front sub-view.Certainly, according to the requirement of image procossing, the number of sub-view Amount n is variable.Correspondingly, quantity n of shape is also variable.

For every sub-view, shape registration apparatus 502 performs shape matching.More specifically, on image Place the initial average shape of the every individual sub-view for corresponding model, and the set of selected shape parameter.Check from flat All the region of each image around of shape is to find the optimal proximity matching for this point.Update form parameter with most preferably The new-found point of matching.It is to say, perform shape distortion to obtain for object images matching from corresponding shape Shape.Afterwards, fitting result, i.e. the shape of matching be transfused to fitting result select device 503 to select best fit result, This will be later described.Best fit result is output as the last registration of object images.

Think, the process of the method for the equipment execution for object registration will be passed through discussion with reference to Fig. 6.Fig. 6 is according to this The flow chart for object registration of invention.

In step 610, for input picture, it is possible to use face detector detection face or object.It is then possible to from face The framing mask in portion estimates original shape and size.According to the present invention, use based on boosting nesting cascade detectors regarding more Angle face detector.Face detector may provide for the initial facial visual angle of face registration from various visual angles.Alternatively, from various visual angles Face detector can not provide initial facial visual angle.

In step 620, according to initial visual angle and the shape that obtained by face detector, select suitably from the model of training The set of shape.These models selected corresponding to initial visual angle itself and are adjacent visual angles.Such as, face visual angle It is divided into five visual angle classifications, i.e. just (F), left half side (LHP), left side (LP), right half side (RHP), right side (RP).Each regard Angle classification is considered passage, and the model trained has five models, including LP, LHP, F, RHP and RP shape.

The main process of the example of selected shape model shown in Fig. 7.Fig. 7 is for selecting showing of candidate shape model Figure.As it is shown in fig. 7, select the shape set corresponding to the visual angle adjacent with initial visual angle as candidate family.Specifically, For passage F, selected shape can include F, LHP and RHP model；For passage LP, selected shape LP and LHP model can be included；For passage RP, selected shape can include RP and RHP model；For passage LHP, Selected shape can include F, LHP and LP model；For passage RHP, selected shape can include F, RHP and RP model.

As it is shown in fig. 7, select multiple shape rather than single model, this makes the dependency for initial visual angle Reduce, and also realize preferably registrating performance than single model.Selection can also be performed the most in any other way. For example, it is possible to select four models adjacent with the model at initial visual angle.

Alternatively, if the most not providing initial visual angle, then skilled shape can be selected.

In step S630, once it is determined that the set of shape, then these models can be used to come relative to original shape Registration face.In this step, standard ASM scheme is used to perform matching.The program is mainly made up of following steps: 1) obtain Monitored initial visual angle, position and the size of face；2) Local Search is to find the best match position of each characteristic point；And 3) select main points and update form parameter by shape.Many face registration sides can be used in models fitting step Method, described models fitting step such as uses the face registration of statistical model and wavelet character, based on local grain grader Face registration and the face registration via difference search based on composition.

In step 640, after by using these shapes selected for face to perform matching respectively, build The vertical appraisal procedure finding best fit result.The details of appraisal procedure explained below.

Following presentation goes out the difference between the present invention and prior art.

Table 1. present invention and the comparison of method of the prior art

In Table 1, √ represents and is, × represent no.As it can be seen from table 1 compared with conventional method, right according to the present invention Estimate or renewal, any mixing shape and 3D shape as registration apparatus and method need not visual angle.Additionally, the present invention Multiple fitting result can be produced, and conventional method only produces a fitting result.

Object registration scheme according to the present invention compares with using the scheme corresponding to the single model at sub-visual angle.

When using, for the face registration in universal method, the sub-visual angle of only one obtained by face detector, only select The shape corresponding with sub-visual angle performs matching.Without being correctly predicted initial visual angle, then registration result becomes not Reliably.In the object registration equipment according to the present invention and method, the set of selected shape model from the shape of training； Each in selected model is used to carry out the characteristic point around the average shape that Local Search is corresponding；And select optimal plan Close result.This is not only avoided initial visual angle for the impact of whole registration process, and realizes high accuracy and the Shandong of face registration Rod.

Now, the method and apparatus being used for description assessing the precision of registration shape.How to assess the precision of registration shape Problem be divided into and calculate two marks corresponding from different information: the first mark is used for according to spatial prior (spatial Prior) wellness of the face shape of assessment registration, described spatial prior represents likely deforming of object shapes；Second Mark assesses the wellness of registration shape for using the image evidence (evidence) provided by object images.By two The model of off-line training, i.e. spatial prior model and standard likelihood model, be modeled both information.

The present invention concentrated with shape registration accuracy evaluation problem be also expressed as Bayes's framework and be represented as Posterior probability p (V | I).P (V | I) represents the probability of the V under conditions of image texture I.V=(x₁, x₂... x_n, y₁, y₂..., y_n) general general formula analysis (GPA, Generalized Procrustes Analysis, a kind of Statistical Shape analysis) is described after Face shape, this is general removes similarity conversion attitude and by zero barycenter and size normalization (unit with general formula analysis Norm) it is normalized to remove skew and size variation.I represent fixed point number.(x_i, y_i) represent i-th fixed point Coordinate.

Based on beta functionBy following formula (1) represent above posterior probability p (V | I)。

$p\left(V\right|I)=\frac{p\left(V\right)p\left(I\right|V)}{p\left(I\right)}\∝p\left(V\right)p\left(\text{I|V}\right)---\left(1\right)$

P (V) is the prior probability of face shape V.P (I | V) it is the face of this position in image when to shaped V Apparent likelihood probability.Here, picture material is arbitrary, i.e. the apparent Probability p (I) of any image is equal.By This, the apparent Probability p (I) of face is considered as steady state value.Therefore,It is directly proportional to p (V) p (I | V).

Now, the mode of the prior probability p (V) obtaining face shape V will be described.By the shape sample to hand labeled Carry out principal component analysis (PCA), total shape is diminished and is divided into many independent compositions, and every in these independent compositions One is encoded by corresponding main constituent.PCA is to use orthogonal transformation the set of the observed value of the variable that may be correlated with to be converted to It is referred to as the mathematical procedure of the set of the value of the uncorrelated variables of main constituent.The number of main constituent is less than or equal to the number of source variable Mesh, to reduce the dimension of converted data.

It is zero and the corresponding basis of principal component model that the shape distortion encoded by each main constituent is modeled as average True value is the Gauss distribution of variance.These distributions can calculate from the face portion shape of hand labeled.

By current face shape V is mapped on main constituent axle, prior probability calculated as below.

When given face shape V, face shape V is mapped as the linear combination V=V of K main constituent₀+p₁V₁+…+ p_kV_k+…+p_KV_K。V_kIt is kth main constituent, and is the vector derived from the sample statistics of hand labeled.p_kCorresponding with V Mapping coefficient, and be scalar.V is by p₁..., p_k..., p_KDetermine uniquely.Therefore, p (V) can be represented as mapping coefficient p₁..., p_k..., p_KJoin probability distribution p (p₁..., p_k..., p_K)。

P (V)=p (p₁..., p_k..., p_K) (2)

According to the distribution of PCA, main constituent V_kOrthogonal, the most independently of one another.Therefore, each main constituent the data represented It it is Gauss distribution.Thus, the prior probability of face shape V is calculated by following formula (3).

$p\left(V\right)=p(p_1,\·\·\·,p_k,\·\·\·,p_K)$

$=\underset{k=1}{\overset{K}{\mathrm{\Π}}}\left(p_k\right)$

$=\underset{k=1}{\overset{K}{\mathrm{\Π}}}\left[\frac{1}{\sqrt{{2\mathrm{\π\λ}}_k}}\exp (-\frac{p_k^2}{{2\mathrm{\λ}}_k})\right]---\left(3\right)$

$\∝\exp (-\underset{k=1}{\overset{K}{\mathrm{\Σ}}}\frac{p_k^2}{{\mathrm{\λ}}_k})\underset{k=1}{\overset{K}{\mathrm{\Π}}}\frac{1}{\sqrt{{\mathrm{\λ}}_k}}$

Wherein, λ_kIt is PCA coefficient p_kStatistical variance.

Now, the mode of likelihood probability p (I | V) obtaining face shape V will be described.P (I | V) it is when to shaped V The likelihood probability that the face of the position in image I is apparent.Position v_iPlace sees that the image evidence of i-th fixed point is assumed to be For the image evidence independent of other fixed point when the locus of other fixed point given.Therefore,

Locally likelihood probability is modeled as Gauss distribution.It is to say, likelihood probability p (I | v_n) be modeled as Gauss and divide Cloth.

In a word, it is possible to use following formula (4) calculates overall likelihood probability:

$\left(I\right|V)\left(I\right|v_1,v_2,\·\·\·{,v}_N)$

$\∝\underset{n=1}{\overset{N}{\mathrm{\Π}}}p\left(I\right|v_n)$

$\∝\underset{n=1}{\overset{N}{\mathrm{\Π}}}\frac{1}{\sqrt{{2\mathrm{\π\σ}}_n^2}}\exp (-\frac{{(v_{n,\mathrm{current}}-v_{n,\mathrm{true}})}^2}{{2\mathrm{\σ}}_n^2})---\left(4\right)$

$\∝\exp (-\underset{n=1}{\overset{N}{\mathrm{\Σ}}}\frac{{\left|\right|{\mathrm{\Δv}}_n\left|\right|}^2}{{\mathrm{\σ}}_n^2})\underset{n=1}{\overset{N}{\mathrm{\Π}}}\frac{1}{{\mathrm{\σ}}_n}$

Wherein,It is at given position v_nTime the statistical variance of local likelihood probability of image I.Δv_nRepresent n-th The current location v of fixed point_{N, current}With true value position v_{N, true}Between displacement.The n-th fixed point when carrying out shape registration True value position v_{N, true}It is unknown.But, apparent likelihood model is designed to use increment based on gradient tree Boost to return Return model direct estimation displacement v_n, this be discussed below.

Being approximated by two above, posterior probability can be rewritten as:

$p\left(V\right|I)\∝p\left(V\right)p\left(I\right|V)$

$\∝\exp (-\underset{k=1}{\overset{K}{\mathrm{\Σ}}}\frac{p_k^2}{{\mathrm{\λ}}_k})\underset{k=1}{\overset{K}{\mathrm{\Π}}}\frac{1}{\sqrt{{\mathrm{\λ}}_k}}\·\exp (-\underset{n=1}{\overset{N}{\mathrm{\Σ}}}\frac{{\left|\right|{\mathrm{\Δv}}_n\left|\right|}^2}{{\mathrm{\σ}}_n^2})\underset{n=1}{\overset{N}{\mathrm{\Π}}}\frac{1}{{\mathrm{\σ}}_n}]$

$\∝\exp (\mathrm{\α}\·\underset{k=1}{\overset{K}{\mathrm{\Σ}}}\frac{p_k^2}{{\mathrm{\λ}}_k}+\mathrm{\β}\·\underset{n=1}{\overset{N}{\mathrm{\Σ}}}\frac{{\left|\right|{\mathrm{\Δv}}_n\left|\right|}^2}{{\mathrm{\σ}}_n^2})\underset{k=1}{\overset{K}{\mathrm{\Π}}}\frac{1}{\sqrt{{\mathrm{\λ}}_k}}\underset{n=1}{\overset{N}{\mathrm{\Π}}}\frac{1}{{\mathrm{\σ}}_n}---\left(5\right)$

$\∝c_1{c}_2\·\exp (\mathrm{\α}\·s_1+\mathrm{\β}\·s_2)$

$\∝\exp (\mathrm{\α}\·s_1+\mathrm{\β}\·s_2)$

$s_1=\underset{k}{\overset{K}{\mathrm{\Σ}}}\frac{p_k^2}{{\mathrm{\λ}}_k}---\left(6\right)$

$s_2=\underset{n=1}{\overset{N}{\mathrm{\Σ}}}\frac{{\left|\right|{\mathrm{\Δv}}_n\left|\right|}^2}{{\mathrm{\σ}}_n^2}---\left(7\right)$

Wherein, s₁It is the first mark representing prior probability, s₂It it is the second mark representing likelihood probability.Parameter alpha and β divide It is not s₁And s₂Weight.The purpose introducing the two parameter is so that s₁And s₂Scope roughly the same, with for posterior probability Balance s₁And s₂Distribution.The value of the two parameter can adjust according to demand.As example,And

Only relevant to pre-determined model and apparent independent of current image.In an experiment, The inventors discovered that and replace c with constant₁c₂Impact for the Model Selection of the present invention is negligible.Then, c₁c₂· exp(α·s₁+β·s₂) be further approximated as exp (α s₁+β·s₂) to simplify calculating.

When given registration shape and object images, can be easily with little calculating cost calculation s=α s₁+β·s₂.Should Mark is for assessing the wellness of shape registration.

Then, the mark of corresponding with multiple shapes registration shape is compared with one another.There is the registration of highest score Shape is selected as final result.

The experimental result that table 2 registrates according to the various visual angles face of the present invention

As shown in table 2, first row represents the number of experiment, and the first row represents shape to be used.Corresponding shape Row under model represent the mark of the model in corresponding experiment.Such as, the mark "-0.472332 " in the 2nd row the 2nd row means Mark for the model " LP " of experiment 1 is-0.472332.

In the experiment 1 of table 2, the shape with highest score-0.183356 is selected as final result.At table 2 In experiment 2, the shape with highest score-0.164278 is selected as final result.In the experiment 3 of table 2, have the highest The shape of mark-0.187144 is selected as final result.In the experiment 4 of table 2, there is highest score-0.193906 Shape is selected as final result.Experiment shows that the mark calculated by the present invention can be with high robust difference registration shape Wellness.

In any case, above expression formula s=α s₁+β·s₂It it is only a kind of method calculating final mark.Alternatively, may be used With by by s₃、s₄S is calculated with multiplication.s₃It iss₄It isIndeed, it is possible to S is calculated based on any step in above-mentioned derivation.

In a word, the principle of the present invention is as follows.For the face shape of each registration, calculate according to differently information independently Two marks, and two mark linear combinations are produced the final mark wellness with the face shape of assessment registration.Root Calculate the first mark according to the deformation of current face shape, and calculate second according to the apparent information of the face provided by object images Mark.

The method and apparatus of above-mentioned assessment of precision for realize face registration (object registration) be will now be described.

Fig. 8 illustrates the object shapes for assessing the registration represented by multiple fixed points in image according to the present invention The assessment equipment of precision.

This assessment equipment includes: storage device 1201, for storage in advance from the spatial prior of hand labeled sample learning Model and object apparent model；First mark calculates device 1202, is used for according to current shape relative to spatial prior model Deformation calculates the first mark；Second mark calculates device 1203, for using object apparent model according to the mark by all registrations The image evidence of the position in the object images that fixed point determines, calculates the second mark；And apparatus for evaluating 1204, it is used for passing through The weighting summation of the two mark calculates the final mark of the object shapes of registration.

In storage device 1201, by utilizing general general formula analysis (GPA) registration training shapes vector to construct space Prior model, to remove similarity conversion attitude and to be normalized to remove skew by zero barycenter and size normalization And size variation, and the shape vector of registration is performed principal component analysis (PCA) so that whole shape distortion is divided into many These compositions are also encoded by independent composition by main constituent.By according to fixed point by apparent for total object be divided into some Independent local is apparent, for each fixed point learn displacement prediction device with estimate current location and ground connection be really position it Between displacement, and each in the apparent likelihood score in local be modeled as centered by the actual position of fixed point, average be Zero and the Gauss distribution that residual is variance of displacement prediction device that learns in advance, construct object apparent model.

In the first mark calculates device 1202, by shape map will be registrated to main constituent axle to obtain shape distortion Parameter, uses the Gaussian distribution model corresponding with each one-tenth split axle to calculate the mark of each parameter, and sues for peace all marks To form the first mark, calculate the first mark.

In the second mark calculates device 1203, by estimating between current location and actual position for each fixed point Displacement, use the displacement of Gauss distribution local apparent model to calculate the mark of each fixed point, and all marks asked With to form the second mark, calculate the second mark.

Now, the method being used for description assessing the precision of the registration face shape of the set expression by fixed point.Fig. 9 Illustrate for the registration flow chart by the precision assessment method of the object shapes of the set expression of fixed point.

In step 1310, the registration shape of the set expression of fixed point is entered through.By vector V=(x₁, x₂... x_n, y₁, y₂..., y_n) representing shape, this vector comprises the coordinate of the fixed point in image.v_i=(x_i, y_i) represent in image The coordinate of i-th fixed point.

In step 1320, the method establishment off-line space prior model represents likely deforming of object shapes. In order to learn the distribution likely deformed of shape, select the shape of N (such as 300 or 1000) individual hand labeled as training Sample.Preferably, N is big to create more accurate model as far as possible.But, the quantity of required training sample depends on shape Complexity and can flexible degree.

Selecting after all training samples, use general general formula analysis by shape vector and reference figuration vector alignment, It is normalized to remove skew and size change with removal similarity conversion attitude and by zero barycenter and size normalization Change.

Procedure below is for being modeled shape distortion.By the classification of the graphical model of referred to as k-fan, by allowing Shape is distributed the Gauss Markov Random Field Mixture model carrying out deforming or by Gauss distribution according to multivariate Gaussian, to space Shape prior is modeled.For such as face, hands and the special object to be modeled of human body, the degree of shape distortion is not With, and the model with different complexity can be used to be modeled.Model to be used depends on the complexity of shape Degree with possible shape distortion.

Face shape is registrated, proves in an experiment the shape distortion encoded by each main constituent is modeled as Gauss Distribution is enough.

In step 1330, calculate the first mark V=(x of a shape according to spatial prior model₁, x₂... x_n, y₁, y₂..., y_n).The shape distortion using Gauss distribution pair corresponding with each main constituent of PCA shape is modeled.This mark It is calculated as follows:

$s_1=\underset{k}{\overset{K}{\mathrm{\Σ}}}\frac{p_k^2}{{\mathrm{\λ}}_k}---\left(6\right).$

Wherein, p_kIt is the mapping coefficient corresponding with the kth main constituent of PCA shape.λ_kIt is PCA coefficient p_kStatistics Variance.K is the quantity of the main constituent retained.

K-fans or Gauss Markov Random Field Mixture model can be used or particularly simple use the current of object Distance between shape and average shape, calculates the first mark.

In step 1340, the method creates the apparent likelihood model of shape and represents the image when given current shape Evidence.Object is apparent is divided into the some Part portions corresponding with fixed point.At position v_iSee the image card of i-th fixed point According to the image evidence being assumed independent of other point when the locus of other point given.For each Part portions, Apparent likelihood score is modeled as Gauss distribution, and this Gauss distribution is centered by the actual position of fixed point and has particular variance.

$p\left(I\right|v_n)\∝\exp (-\frac{{(v_{n,\mathrm{current}}-v_{n,\mathrm{true}})}^2}{{2\mathrm{\σ}}_n^2})---\left(4a\right)$

In order to calculate local likelihood probability, set up model and use increment regression model based on gradient tree Boost direct Predictive displacement Δ v_n=v_{N, current}-v_{N, true}, and use training residual as variance

Here other methods many can also be used.It is, for example possible to use with haar-like feature combination GentleBoost logistic regression method learns image evidence.Use Haar-like feature next as the basis of weak regression function Local grain is mapped as size characteristic value h (I).And to calculate displacement from eigenvalue h (I) by weak regression function.

In step 1350, calculate the second mark according to the apparent likelihood model in local.

$s_2=\underset{n=1}{\overset{N}{\mathrm{\Σ}}}\frac{{\left|\right|{\mathrm{\Δv}}_n\left|\right|}^2}{{\mathrm{\σ}}_n^2}---\left(7\right).$

Wherein Δ v_nRepresent in object images is corresponding with the n-th fixed point pre-by the n-th apparent likelihood model in local Displacement between current location and the actual position surveyed.N is the number of fixed point.

In step 1360, calculated the accurate measurement of current shape by the weighted sum of two marks calculated.

$\mathrm{Score}=\mathrm{\α}\·\underset{k}{\overset{K}{\mathrm{\Σ}}}\frac{p_k^2}{{\mathrm{\λ}}_k}+\mathrm{\β}\·\underset{i=1}{\overset{N}{\mathrm{\Σ}}}\frac{{\left|\right|{\mathrm{\Δv}}_i\left|\right|}^2}{{\mathrm{\σ}}_i^2}---\left(5a\right)$

$=\mathrm{\α}\·s_1+\mathrm{\β}\·s_2$

Parameter alpha and β are the weights of respective items, they (0,1] in the range of and need to select according to experiment.This In,And

In any case, above-mentioned expression formula Score=α s₁+βs₂It it is only a kind of mode calculating final mark.Alternatively, logical Cross s₃, s₄Score is calculated with multiplication.s₃It iss₄It isIndeed, it is possible to Score is calculated based on any step in above-mentioned derivation.

Compared with the method for L.Liang et al., in the present invention, face is divided into some regional areas according to face points. At the Local map photo of 30 × 30 pixels, an apparent model is learnt independently for each face points.With for 30 × 30 pictures Whole face-image one apparent model of study of element is compared, and can represent more information by apparent model, and this will compare The method of L.Liang et al. more robust.Additionally, in the present invention, increment homing method structure apparent model is used.By from The random file of actual position creates sample, and uses displacement marker samples.Regression training can use obtainable whole instruction Practice data.So it is more accurate than BAM.

Additionally, in the appraisal procedure of the present invention,Δv_n=v_{I, current}- v_{I, groundtruth}It it is the displacement currently between some position and actual position.Apparent likelihood model is designed to demarcate for n-th Point direct estimation displacement v_n.Thus, it is possible to by little calculating cost calculation likelihood probability.

The above-mentioned appraisal procedure according to the present invention can be used in numerous applications.One of these examples are as in Fig. 5 and 6 The shown Model Selection in various visual angles face shape registration.

Can variously perform the method and system of the present invention.For example, it is possible to by software, hardware, firmware or Combinations thereof performs the method and apparatus of the present invention.The said sequence of the step of described method is only intended to illustrate, and this The step of bright method is not limited to order particularly described above, unless expressly stated otherwise.Additionally, in certain embodiments, The present invention can also be embodied as the program recorded in the recording medium, including for realize the machine of the method according to the invention can Reading instruction.Thus, the present invention also covers the record medium of the storage program for realizing the method according to the invention.

Although passing through example particular instantiation some specific embodiments of the present invention, but those skilled in the art should managing Solve example above to be only intended to illustrate rather than limit the scope of the present invention.Skilled artisan would appreciate that can without departing from Above example is revised in the case of scope and spirit of the present invention.The scope of the present invention is defined by the following claims.

Although describing the present invention with reference to exemplary embodiment, it should be understood that the invention is not restricted to disclosed exemplary enforcement Example.The scope of following claims should be given the widest explaining with the 26S Proteasome Structure and Function comprising all modifications and equivalent.

Claims (19)

1. for the method assessing the precision of the object shapes of the registration represented by multiple fixed points in image, the method Including:

First mark calculation procedure, uses the first model to calculate the first mark according to the object shapes of registration；

Second mark calculation procedure, uses the second model to calculate the second mark according to each in the plurality of fixed point；With And

Appraisal procedure, calculates the precision of registration according to the first mark and the second mark；

Wherein, described first mark is for the wellness of the object shapes according to spatial prior assessment registration, described spatial prior Representing the deformation of the object shapes of registration, described second mark is joined for using the image evidence provided by described image to assess The wellness of accurate object shapes, described first model is the spatial prior model of the deformation of the object shapes representing registration, institute Stating the second model is the object apparent model representing the image evidence that described image provides.

Method the most according to claim 1, wherein,

In described appraisal procedure, calculated the precision of registration by the weighted sum of the first mark and the second mark.

Method the most according to claim 1, wherein,

In the first mark calculation procedure, calculate the first mark according to object shapes relative to the deformation of spatial prior model.

Method the most according to claim 3, wherein,

In the first mark calculation procedure:

Object shapes is mapped to main constituent axle to obtain shape morphing parameter；

The Gaussian distribution model corresponding with each main constituent is used to calculate the mark of each shape morphing parameter；And

Sue for peace to obtain the first mark to the mark corresponding with each main constituent.

Method the most according to claim 4, wherein,

In the first mark calculation procedure, calculate the first mark by following formula:

Wherein, s₁Representing the first mark, K represents the number of main constituent, and k represents kth main constituent, p_kRepresent kth main constituent Form parameter, enters_kRepresent form parameter p_kStatistical variance.

Method the most according to claim 1, wherein,

In the second mark calculation procedure:

Displacement between current location and actual position is estimated for each fixed point；

Use the mark being displaced through the Gaussian distribution model calculating each fixed point corresponding with described object apparent model；And

The mark of the calculating corresponding with each fixed point is sued for peace, to obtain the second mark.

Method the most according to claim 6, wherein,

In the second mark calculation procedure, calculate the second mark by following formula,

Wherein, s₂Representing the second mark, N represents the number of fixed point, Δ v_iRepresent the current location corresponding with i-th fixed point And the assessed value of the displacement between actual position,Represent the assessed value Δ v of displacement_iStatistical variance.

8. according to the method described in claim 6 or 7, wherein,

The estimation for the displacement between current location and the actual position of each fixed point is performed by first order recursive assessment.

9. for the method registrating the object shapes in image, including:

Shape step of registration, according to the multiple shape matching object shapes for object；Wherein, the plurality of shape In each corresponding to the image at a visual angle from object, and each fitting result is represented by multiple fixed points；

Select step, calculated the mark of the fitting result of each shape based on fitting result by appraisal procedure, and select Select the fitting result corresponding with the highest score registration result as object shapes；

Wherein, for a fitting result, described appraisal procedure includes:

First mark calculation procedure, uses the first model to calculate the first mark according to fitting result；

Second mark calculation procedure, uses the second model to calculate the second mark according to each in the plurality of fixed point；With And

Appraisal procedure, according to the first mark and the precision of the second mark digital simulation result.

10. for the method registrating the object shapes in image, including:

Shape step of registration, according to the multiple shape matching object shapes for object, in the plurality of shape Each is corresponding to the image at a visual angle from object；

Select step, calculate each shape based on fitting result by the appraisal procedure according to any one in claim 1-8 The mark of the fitting result of model, and select the fitting result corresponding with highest score as the registration result of object shapes.

The equipment of the precision of the object shapes of 11. 1 kinds of registrations represented by multiple fixed points assessed in image, this equipment bag Include:

First mark calculates device, for using the first model to calculate the first mark according to the object shapes of registration；

Second mark calculates device, for using the second model to calculate second point according to each in the plurality of fixed point Number；And

Apparatus for evaluating, for calculating the precision of registration according to the first mark and the second mark；

Wherein, described first mark is for the wellness of the object shapes according to spatial prior assessment registration, described spatial prior Representing the deformation of the object shapes of registration, described second mark is joined for using the image evidence provided by described image to assess The wellness of accurate object shapes, described first model is the spatial prior model of the deformation of the object shapes representing registration, institute Stating the second model is the object apparent model representing the image evidence that described image provides.

12. equipment according to claim 11, wherein,

Described apparatus for evaluating calculates the precision of registration by the weighted sum of the first mark and the second mark.

13. equipment according to claim 12, wherein,

First mark calculates device and calculates the first mark according to object shapes relative to the deformation of spatial prior model.

14. equipment according to claim 13, wherein,

First mark calculates device and object shapes is mapped to main constituent axle to obtain shape morphing parameter；Use with each main become Corresponding Gaussian distribution model is divided to calculate the mark of each shape morphing parameter；And the mark corresponding with each main constituent is entered Row summation is to obtain the first mark.

15. equipment according to claim 14, wherein,

First mark calculates device and calculates the first mark by following formula:

Wherein, s₁Representing the first mark, K represents the number of main constituent, and k represents kth main constituent, p_kRepresent kth main constituent Form parameter, enters_kRepresent form parameter p_kStatistical variance.

16. equipment according to claim 11, wherein,

Second mark calculates device and estimates the displacement between current location and actual position for each fixed point；Use displacement is led to Cross the Gaussian distribution model corresponding with described object apparent model and calculate the mark of each fixed point；And to each fixed point The mark of corresponding calculating is sued for peace, to obtain the second mark.

17. equipment according to claim 16, wherein,

Second mark calculates device and calculates the second mark by following formula,

Wherein, s₂Representing the second mark, N represents the number of fixed point, Δ v_iRepresent the current location corresponding with i-th fixed point And the assessed value of the displacement between actual position,Represent the assessed value Δ v of displacement_iStatistical variance.

18. according to the equipment described in claim 16 or 17, wherein,

The estimation for the displacement between current location and the actual position of each fixed point is performed by first order recursive assessment.

19. 1 kinds of equipment being used for registrating the object shapes in image, this equipment includes:

Shape registration apparatus, for according to the multiple shape matching object shapes for object, the plurality of shape In each corresponding to the image at a visual angle from object；And

Select device, for calculating each shape based on fitting result by the equipment according to any one in claim 11 to 18 The mark of the fitting result of shape model, and select the fitting result corresponding with highest score as the registration knot of object shapes Really.