CN105205827A - Auxiliary feature point labeling method for statistical shape model - Google Patents

Abstract

The invention discloses an auxiliary feature point labeling method for a statistical shape model. The method comprises the steps that 1, all feature points on the edge of an object contour are connected through a CCR curve, and a general template for the object shape is built; 2, the general template is mapped to an image to be labeled; 3, rigid adjustment is carried out on the general template, so that the general template is matched with the object contour on the image to be labeled; 4, non-rigid adjustment is carried out on the general template, so that all the features fall to the real edge of the image to be labeled, and the positions are correct; 5, the general template obtained after the positions of the feature points are adjusted is stored. The problem of labeling the objects with the unconspicuous feature points can be effectively solved, and the time for manually drawing points can be saved. A normal assisting method is adopted, manual feature point adjustment becomes simple and is easy to operate, and labeling efficiency is improved. The CCR curve crossing control points, namely, the features points are used for expressing the contour, and contour description is more scientific and accurate.

Description

Unique point for statistical shape model assists mask method

Technical field

The invention belongs to technical field of image processing, be specifically related to a kind of unique point for statistical shape model and assist mask method.

Background technology

Statistical shape model such as ASM model and AAM model are widely used in the segmentation field of the object such as face, vehicle, and the iterative search procedures of two kinds of models all be unable to do without the unique point set up in advance.Therefore, the quality of unique point mark, directly has influence on the precision of statistical shape model.

Unique point also claims key point.Namely so-called unique point mark is the edge marking out body form on image to be marked.

Existing unique point mask method great majority are all for face, and such as unique point is marked on two corners of the mouths of face, four canthus place etc., as shown in Fig. 1 (a).Many auxiliary mask methods also all for face, as Yu Shiqi etc. proposes a kind of face key point localization method based on three-dimensional face images in patent [CN104899563A]; The people such as Han Junyu propose a kind of auxiliary face key point mask method of predicted characteristics point reposition in patent [CN102880612A].But the precondition that these methods can be proved effective still needs the unique point marked in advance.Secondly, the unique point mask method for face might not be used for marking other objects.For example, face has some features obviously position, such as canthus, the corners of the mouth, nose etc.Therefore when marking, these positions can be found rapidly, and the quality of mark is higher.But for the unconspicuous object of feature on some profiles, the method effect of this direct searching obvious characteristic point is unsatisfactory.In two-dimensional ultrasonic image as shown in Fig. 1 (b) left ventricle endocardial contours line on significantly feature only have valve and the apex of the heart, other local curvature are without significant change, smoother.Therefore for this type objects, the mask method often not competent work of face, marks second-rate.Meanwhile, the profile of familiar object is level and smooth mostly, if with means conventional at present, goes to express profile by the mode that straight line connects by each unique point, and profile is too coarse comparatively large with original difference first, and two is that impact is attractive in appearance, image not.As shown in Figure 2, visible CCR curve representation more accurately image is contrasted, especially when unique point is less.

Summary of the invention

In order to overcome the above-mentioned shortcoming of prior art, the invention provides a kind of unique point for statistical shape model and assisting mask method.

The technical solution adopted for the present invention to solve the technical problems is: a kind of unique point for statistical shape model assists mask method, comprises the steps:

Each unique point on contour of object edge is carried out line by step one, employing CCR curve, sets up the common template of a body form;

Step 2, common template is mapped on image to be marked;

Step 3, common template done to rigidity adjustment, make the contour of object on common template and image to be marked identical;

Step 4, non-rigid adjustment is done to common template, each unique point is dropped on image true edge to be marked and position is correct;

Common template after step 5, preservation adjustment characteristic point position.

Compared with prior art, good effect of the present invention is: more existing artificial mask methods, can only mark the obvious objects of feature such as such as face, for unique point unconspicuous object mark difficulty.The auxiliary mask method for statistical shape model that the present invention proposes can solve this problem very effectively, and following effect can be reached: the method that is, common template is set up in employing, template is mapped directly on image to be marked, saves the artificial time of drawing point.The methods two being, adopting normal auxiliary, make manually to adjust unique point and become simple and easy to operate, improve annotating efficiency.Three be, used the CCR curve representation profiles of reference mark also i.e. unique point, the description making profile science, accurately more.

Accompanying drawing explanation

Examples of the present invention will be described by way of reference to the accompanying drawings, wherein:

In Fig. 1: (a) is the width face figure marked; B () is two-dimensional ultrasonic image left ventricle picture;

In Fig. 2: (a) uses the CCR curve representation profile through unique point; B () expresses profile for using straight line to connect adjacent feature point; C () is CCR and straight line two kinds of expression way contrasts;

Fig. 3 is the process flow diagram of the inventive method;

Fig. 4 is unique point and normal corresponding relation figure, and wherein: black round dot represents the reference mark in template, the curve of connection control point represents CCR curve; Normal m was some P _i, and be adjacent the vertical straight line of 2 lines; N is actual profile edge on image to be marked; P ' _ifor the intersection point of normal m and curve n;

In Fig. 5: (a) is the common template set up; B () is for unique point Template Map is on image to be marked; C () is for doing rigidity adjustment to template; D () is for explicit representation is to boost line; E () does non-rigid adjustment for template, the reposition of unique point is with reference to the intersection point of normal direction boost line and endocardium of left ventricle real border; F () is the endocardium of left ventricle profile after adjustment.

Embodiment

The unique point that the invention provides body form in a kind of two dimensional image assists mask method, is intended to solve the problem that cannot mark unique point when feature is not obvious on contour of object.

The mask method that we propose is based on CCR (CentripetalCatmull-Rom) curve and ASM (ActiveSplineModel) model theory.First an introduction is done to both below.

CCR curve can come tendency and the shape of controlling curve by one group of point.CCR curve has some advantage following compared to other curve.First, it not requirement itself be a closed curve, so just can use the body form of this curve representation profile non-closed; Secondly, CCR curve was the curve at reference mark, was more easy to operation compared with the Bezier etc. without reference mark; Moreover it allows reference mark more sparse, use less reference mark just can reach the object accurately expressing shape profile like this.

The concrete method for expressing of CCR curve is:

With P=[x, y] ^trepresent a point.Reference mark P _i-1, P _i, P _i+1, P _i+2can be used for representing a segment of curve Q of CCR curve _i.T _i-1, t _i, t _i+1, t _i+2represent the knot (knot) on this segment of curve.This segment of curve can with as shown in the formula subrepresentation:

$Q_i=\frac{t_{i+1}-t}{t_{i+1}-t_i}{L}_{012}+\frac{t_i-t}{t_{i+1}-t_i}{L}_{123}---\left(1\right)$

Here,

$L_{012}=\frac{t_{i+1}-t}{t_{i+1}-t_{i-1}}{L}_{01}+\frac{t-t_{i-1}}{t_{i+1}-t_{i-1}}{L}_{12}---\left(2\right)$

$L_{123}=\frac{t_{i+2}-t}{t_{i+2}-t_i}{L}_{12}+\frac{t-t_i}{t_{i+2}-t_i}{L}_{23}---\left(3\right)$

$L_{01}=\frac{t_i-t}{t_i-t_{i-1}}{P}_{i-1}+\frac{t-t_{i-1}}{t_i-t_{i-1}}{P}_i---\left(4\right)$

$L_{12}=\frac{t_{i+1}-t}{t_{i+1}-t_i}{P}_i+\frac{t-t_i}{t_{i+1}-t_i}{P}_{i+1}---\left(5\right)$

$L_{23}=\frac{t_{i+2}-t}{t_{i+2}-t_{i+1}}{P}_{i+1}+\frac{t-t_{i+1}}{t_{i+2}-t_{i+1}}{P}_{i+2}---\left(6\right)$

t _i+1＝|P _i+1-P _i| ^α+t _i(7)

(7) in formula, α span is [0,1], by its value 0.5 in the present invention.

Suppose that CCR curve is through a P ₁, P ₂, P ₃..., P _m, for making CCR curve pass through this m point completely, also need curve end to end respectively increase a reference mark P ₀and P _m+1, these two reference mark do not show on curve.The computing method of 2 are:

$\left\{\begin{array}{c}{P}_0=P_1-\mathrm{\ρ}(P_2-P_1)\\ P_{m+1}=P_m+\mathrm{\ρ}(P_m-P_{m-1})\end{array}\right.---\left(8\right)$

(8) in formula, the span of ρ is [0,0.5], here value 0.1.

ASM model is the model of Corpus--based Method shape.The method carries out statistical study to shape instance a large amount of in training set, sets up the shape Statistics model of reflection target shape Changing Pattern and the local gray level model of reflection intensity profile rule.In search object process, the target first treated in detected image is searched for, and obtains initial position.By training average shape Model Mapping out on detected image.This process is also referred to as rigid deformation.Utilize the feature around each unique point of training out afterwards, drive average shape to move to more desirable position; Its rationality is judged simultaneously, the rationality of shape in statistical significance is ensured to irrational shape adjustment.So by loop iteration, obtain desirable matching result.This process is called as non-rigid deformation.Therefore, mask method in this paper develops based on above two tools.

Introduce the concrete steps that unique point described in the present invention assists mask method below, as shown in Figure 3:

Step one, set up the common template of a body form:

Common template refers to the coordinate of each unique point on contour of object edge, is designated as { (x ₁, y ₁), (x ₂, y ₂), (x ₃, y ₃) ..., (x _n, y _n).During indicating template, we adopted the CCR curve of these unique points to represent.Here we provide the method for building up of two kinds of common templates: one is the unique point rule of thumb drawn out by expert on this body form, then using these unique points as reference mark, connect with CCR curve, give expression to body form.Finally preserve template, namely preserve the coordinate at each reference mark in template.Two is by selecting multiple images, directly draws contour of object with CCR curve, then marks out the unique point on shape profile, then these shapes alignd, and calculates mean profile, on mean profile, finally select a stack features point to preserve, as common template.Here we provide a kind of alignment schemes based on ProcrustesMethod.

Given two similar shape X ₁(x ₁₁, y ₁₁, x ₁₂, y ₁₂..., x _1n, y _1n) and X ₂(x ₂₁, y ₂₁, x ₂₂, y ₂₂..., x _2n, y _2n).For by X ₂snap to X ₁, namely to find an anglec of rotation θ, zoom factor s and translational movement t (t _x, t _y), make following formula minimum:

E＝(X ₁-M(s,θ)[x ₂]-t) ^TW(X ₁-M(s,θ)[x ₂]-t)

Here

$M(s,\mathrm{\θ})\left[\begin{array}{c}{x}_{jk}\\ y_{jk}\end{array}\right]=\left(\begin{array}{c}(s\·cos\mathrm{\θ})x_{jk}-(s\·sin\mathrm{\θ})y_{jk}\\ (s\·sin\mathrm{\θ})x_{jk}+(s\·\cos \mathrm{\θ})y_{jk}\end{array}\right)$

t＝(t _x,t _y,...,t _x,t _y) ^T

W is a diagonal matrix, and on diagonal line, each element represents the weight of each unique point.Its computing method are: suppose known one group of similar shape, kth (k=1,2 ..., n) the weight w of individual Feature point correspondence _kcomputing method be:

$w_k={\left(\underset{l=1}{\overset{n}{\Σ}}{V}_{R_{k1}}\right)}^{-1},1=1,2,3,\mathrm{...},n$

Wherein, R _klrepresent the distance between a kth point and l point in a shape; represent R in this group shape _klstatistical variance.

According to the alignment schemes of above two similar shapes, for one group of similar shape X ₁, X ₂..., X _n, its alignment schemes is:

(1) in this group shape, selected first shape as a reference, is designated as remaining shape is alignd with first shape (by rotating, convergent-divergent, translation come).

(2) shape X ' is obtained after alignment ₁, X ' ₂..., X ' _n, calculate average shape and by average shape normalization.

(3) by X ' ₁, X ' ₂..., X ' _nsnap to

(4) check whether convergence, as convergence, then stop; Otherwise, repeat (2) (3) step until convergence.

Step 2, by Template Map on image to be marked:

The method mapped is the center barycenter of template being positioned over image to be marked.Template barycenter in this method formula of asking for be:

$\stackrel{\‾}{x}=\frac{\underset{i=1}{\overset{n}{\Σ}}{x}_i}{n},\stackrel{\‾}{y}=\frac{\underset{i=1}{\overset{n}{\Σ}}{y}_i}{n}$

Step 3, template done to rigidity adjustment:

System of the present invention provides action button, can amplify, reduce, to anticlockwise, to right rotation, up and down translation to template.Mark person (expert of mark, as doctor) rule of thumb, does the operation of the rigidity such as corresponding convergent-divergent, translation, rotation to template, until the contour of object on template and image to be marked is roughly identical.During adjustment, the coordinate integral translation at all reference mark, convergent-divergent, rotation, method is as follows:

(1) suppose that integral translation distance is for (d _x, d _y), then the new coordinate in template after each point translation is:

$\{\begin{array}{c}{x}_i^{\′}=x_i+d_x\\ y_i^{\′}=y_i+d_y\end{array},i=1,2,\mathrm{...},n$

(2) suppose that the barycenter of template is (c _x, c _y), the factor of convergent-divergent is α, then the coordinate in template after each point convergent-divergent is:

$\left\{\begin{array}{c}{x}_i^{\′}=x_i\·\mathrm{\α}+c_x\·(1-\mathrm{\α})\\ y_i^{\′}=y_i\·\mathrm{\α}+c_y\·(1-\mathrm{\α})\end{array}\right.,i=1,2,3,\mathrm{...},n$

(3) suppose that the barycenter of template is (c _x, c _y), the anglec of rotation is θ, then in template, each point rotation recoil is designated as:

$\{\begin{array}{c}{x}_i^{\′}=(x-c_x)\·\cos \mathrm{\θ}-(y-c_y)\·\sin \mathrm{\θ}+c_x\\ y_i^{\′}=(x-c_x)\·\sin \mathrm{\θ}-(y-c_y)\·\cos \mathrm{\θ}+c_y\end{array},i=1,2,\mathrm{...},n$

Step 4, non-rigid adjustment is done to template:

Mark person finely tunes Individual features point, and target each unique point is dropped on image true edge and position is correct.The CCR curve deformation that during adjustment, the while of system, indicating characteristic point shift in position causes.In addition, when finely tuning, we provide normal householder method, make adjustment more easily simple.Concrete operation method is as shown in Figure 4: to the unique point P for adjustment _i, we calculate through a P _iand be adjacent two unique point P _i-1and P _i+1the normal m that line direction is vertical.User can control system show or hide normal.The local P ' that this normal is crossing with being marked contour of object on image _i, be exactly can be for reference P _ireposition.The length of normal section m can be adjusted by user, and user adjusts method line segment direction in the whole process of each unique point no longer changes (just line segment length change), to provide reference point for user.

Template after step 5, preservation adjustment characteristic point position:

Template is after adjustment, and its characteristic point position matches with the profile of image to be marked, and also namely the mark work of image completes, and now needs the position of preserving each unique point in template.In this step, the unique point quantity of current template may be less, and cannot meet the demand of user, therefore system allows the user to choose whether to continue to get supplemental characteristic point between adjacent two unique points.The quantity augmented features point that system is specified according to user, when being such as 2, at any P _iand P _i+1the segment of curve of point-to-point transmission is got again two points.

Below in conjunction with concrete application scenarios, namely in two-dimensional ultrasonic image left ventricle inner membrance unique point mark the present invention will be described (Fig. 5 is shown in by schematic diagram).

1) standard form of left ventricle is set up, as shown in Fig. 5 (a).Method is the representative left ventricle two-dimensional ultrasonic image of selection one.The unique point on left ventricle inner membrance is marked out by expert.Unique point normalization is saved as unique point template.

2) image to be marked is opened, by unique point Template Map on this image, as shown in Fig. 5 (b).The method mapped is placed on the central point of image to be marked by the barycenter of unique point template.

3) use convergent-divergent, rotation, shift method, do rigidity adjustment to template, final result makes the profile on template and image to be marked roughly identical.Effect after adjustment is shown in Fig. 5 (c).

4) explicit representation is to boost line, and as shown in Fig. 5 (d), according to the guiding of boost line, expert starts to adjust Individual features point, makes it to move on image outline to be marked, as shown in Fig. 5 (e).

5) preserve unique point, simultaneously if necessary, user can specify in the unique point of getting some between often pair of adjacent feature point again, and forming one has more multi-characteristic points also shape profile more accurately simultaneously, as shown in Fig. 5 (f).

Although we are labeled as example with endocardium of left ventricle in ultrasonoscopy carry out explanation of the present invention and explanation, those skilled in the art should be understood that: the present invention is equally applicable to the unique point mark of the unconspicuous body form of other contour features.

Claims (9)

1. the unique point for statistical shape model assists a mask method, it is characterized in that: comprise the steps:

Each unique point on contour of object edge is carried out line by step one, employing CCR curve, sets up the common template of a body form;

Step 2, common template is mapped on image to be marked;

Step 3, common template done to rigidity adjustment, make the contour of object on common template and image to be marked identical;

Step 4, non-rigid adjustment is done to common template, each unique point is dropped on image true edge to be marked and position is correct;

Common template after step 5, preservation adjustment characteristic point position.

2. the unique point for statistical shape model according to claim 1 assists mask method, it is characterized in that: the method for building up of common template is: select the subject image that representative, rule of thumb draw out the unique point on this body form, then using these unique points as reference mark, preserve the coordinate at each reference mark after being connected to form body form with CCR curve, obtain common template.

3. the unique point for statistical shape model according to claim 1 assists mask method, it is characterized in that: the method for building up of common template is: select multiple images, direct CCR curve draws contour of object, then the unique point on shape profile is marked out, again these shapes are alignd, calculate mean profile, finally the unique point on mean profile is preserved, obtain common template.

4. the unique point for statistical shape model according to claim 3 assists mask method, it is characterized in that: described alignment schemes is:

Given two similar shape X ₁(x ₁₁, y ₁₁, x ₁₂, y _12,..., x _1n, y _1n) and X ₂(x ₂₁, y _21,x ₂₂, y _22,..., x _2n, y _2n), if will by X ₂snap to X ₁, then need to find an anglec of rotation θ, zoom factor s and translational movement t (t _x, t _y), make following formula minimum:

E＝(X ₁-M(s,θ)[x ₂]-t) ^TW(X ₁-M(s,θ)[x ₂]-t)

In formula:

$M(s,\mathrm{\θ})\left[\begin{array}{c}{x}_{\mathrm{jk}}\\ y_{\mathrm{jk}}\end{array}\right]=\left[\begin{array}{c}(s\·\cos \mathrm{\θ})x_{\mathrm{jk}}-(s\·\sin \mathrm{\θ})y_{\mathrm{jk}}\\ (s\·\sin \mathrm{\θ})x_{\mathrm{jk}}+(s\·\cos \mathrm{\θ})y_{\mathrm{jk}}\end{array}\right],j,k=\mathrm{1,2,3},...,n$

t＝(t _x,t _y,...,t _x,t _y) ^T

W is a diagonal matrix, each element w on diagonal line _krepresent the weight of each unique point; Suppose known one group of similar shape, kth (k=1,2 ..., n) the weight w of individual Feature point correspondence _kcomputing method be:

$w_k={\left(\underset{1=1}{\overset{n}{\Σ}}{V}_{R_{k1}}\right)}^{-1},l=\mathrm{1,2,3},...,n$

Wherein, R _klrepresent the distance between a kth point and l point in a shape; represent R in this group shape _klstatistical variance;

According to the alignment schemes of above two similar shapes, for one group of similar shape X ₁, X ₂..., X _n, its alignment schemes is:

(1) in this group shape, selected first shape as a reference, is designated as remaining shape is alignd with first shape;

(2) shape X ' is obtained after alignment ₁, X ' ₂..., X ' _n, calculate average shape and by average shape normalization;

(3) by X ' ₁, X ' ₂..., X ' _nsnap to

(4) check whether convergence, as convergence, then stop; Otherwise, repeat (2) (3) step until convergence.

5. the unique point for statistical shape model according to claim 1 assists mask method, it is characterized in that: described rigidity adjustment comprises translation, convergent-divergent and rotation.

6. the unique point for statistical shape model according to claim 5 assists mask method, it is characterized in that: described shift method is as follows:

Suppose that integral translation distance is for (d _x, d _y), then the new coordinate on common template after each point translation is:

$\{\begin{array}{c}{x}_i^{\′}=x_i+d_x\\ y_i^{\′}=y_i+d_y\end{array},i=1,2,...,n.$

7. the unique point for statistical shape model according to claim 5 assists mask method, it is characterized in that: described Zoom method is as follows:

Suppose that the barycenter of common template is (c _x, c _y), the factor of convergent-divergent is α, then the coordinate on common template after each point convergent-divergent is:

$\{\begin{array}{c}{x}_i^{\′}=x_i\·\mathrm{\α}+c_x\·(1-\mathrm{\α})\\ y_i^{\′}=y_i\·\mathrm{\α}+c_y\·(1-\mathrm{\α})\end{array},i=1,2,3,...,n.$

8. the unique point for statistical shape model according to claim 5 assists mask method, it is characterized in that: described spinning solution is as follows:

Suppose that the barycenter of common template is (c _x, c _y), the anglec of rotation is θ, then on common template, each point rotation recoil is designated as:

$\{\begin{array}{c}{x}_i^{\′}=(x-c_x)\·\cos \mathrm{\θ}-(y-c_y)\·\sin \mathrm{\θ}+c_x\\ y_i^{\′}=(x-c_x)\·\sin \mathrm{\θ}-(y-c_y)\·\cos \mathrm{\θ}+c_y\end{array},i=1,2,...,n.$

9. the unique point for statistical shape model according to claim 1 assists mask method, it is characterized in that: the method for described non-rigid adjustment adopts normal householder method: to the unique point P for adjustment _i, calculate through some P _iand be adjacent two unique point P _i-1and P _i+1the point P ' that the normal m that line direction is vertical, normal m are crossing with being marked contour of object on image _i, be P _ireposition reference point.