CN100507947C - System and method for detecting and matching anatomical structures using appearance and shape - Google Patents

Abstract

A detection framework that matches anatomical structures using appearance and shape is disclosed. A training set of images are used in which object shapes or structures are annotated in the images. A second training set of images represents negative examples for such shapes and structures, i.e., images containing no such objects or structures. A classification algorithm trained on the training sets is used to detect a structure at its location. The structure is matched to a counterpart in the training set that can provide details about the structure's shape and appearance.

Description

Utilize outward appearance and shape to detect and mate the system and method for anatomical structure

The cross reference of related application

The sequence number that the application requires to submit on November 19th, 2003 is No.60/523,382 U.S. Provisional Application, the sequence number of submitting on March 9th, 2004 are No.60/551,585 U.S. Provisional Application, the sequence number of submitting on April 27th, 2004 are No.60/565,786 U.S. Provisional Application and the sequence number of submitting on June 21st, 2004 are No.60/581, the rights and interests of 535 U.S. Provisional Application are incorporated herein by reference these application integral body.

Technical field

The present invention relates to be used to utilize outward appearance and shape to detect system and method with match objects, and relate more specifically to be used to utilize off-line training, online detection and outward appearance and form fit to detect and mate the system and method for anatomical structure.

Background technology

The unusual detection and the diagnosis that medical image system (for example ultrasonic image-forming system) are used for being associated with the anatomical structure organ of heart (for example, such as) in the medical inspection process are very general.Usually come the evaluation map picture by trained medical expert (for example doctor or medical science technician), with the feature in the recognition image, described feature can be indicated the anatomical structure of the unusual or indication health that is associated with anatomical structure.

Because improvement of computer science, most computers can easily be handled lot of data and carry out large-scale calculating, described large-scale calculating can improve the quality of the image that is obtained.In addition, Flame Image Process can be used as the instrument of the analysis of assistant images.Effective detection of interested anatomical structure or object is important instrument in the further analysis of this structure in the image.Usually, the variation in the elapsed time of the unusual or this shape of the shape of anatomical structure (for example, pulsatile heart the is dirty or lung breathed) indication tumour or various diseases (for example Xin Ji expansion or ischaemic).

Such Flame Image Process can be used to other and use, such as the detection of people's face in the image.Because the variable relevant with different facial characteristics (for example hair color and length, eye color, face shape etc.), the facial detection is not footy task.Facial detection can be used for multiple application, such as User Recognition, supervision or Secure Application.

Various types of methods have been used to detect interested object (for example anatomical structure or face).Can handle the big variation of posture and illumination aspect based on the object detector (eye detector and face detecting device etc.) of part, and block with different variance noise under more sane.For example, in the ultrasonic cardiography map analysis, the local appearance of same anatomical (for example barrier film) is similar between patient, and the structure of heart or shape can be owing to for example visual angle or disease situations and significantly different.Similarly, in face detects, general spatial relationship between the facial characteristics is quite consistent (for example eyes is with respect to the general position of nose and mouth), and the structure of various facial characteristics and shape (for example expression of shape of eyes, mouth, and the relative distance between them) can marked changes.

Change in order to catch local appearance, many solutions rely on Gauss (Gaussian) to suppose.Recently, by (Support Vector Machine SVM) or promote the use of the non-linear learning machine of (boosting) and so on, has relaxed this hypothesis such as support vector machine.Some the most successful real-time object detection methods are based on the cascade of the lifting of simple feature.Simple classification device by making up selected quantity is via the response that promotes, and resulting strong classifier can be realized the high detection rate and can handle image in real time.Yet, existing under the situation of occlusion objects, existing method does not solve the detection problem.Simple or Weak Classifier will influence testing result negatively owing to block the errored response of (occlusion).

Use for most of vision track, measurement data is uncertain and loses sometimes: image has noise and distortion, blocks the part that may make objects simultaneously and becomes invisible.Uncertainty can be uniform on the whole; But in most of real world conditions, it is actually different variance, just anisotropy and uneven.Good example is echocardiogram (ultrasonic heart data).Ultrasonicly tend to reflect pseudo-shadow, specular reflector for example is such as those specular reflectors from barrier film.Because single " direction of observation ", the vertical surface of mirror surface structure produces strong echo, but tilt or " from axle " surface can produce weak echo, or not produce echo (acoustics " signal drop-out (drop out) ").For echocardiogram, signal drop-out may appear in the heart area place that is parallel to ultrasonic beam at tissue surface.

Because its availability, low relatively cost and Noninvasive, cardiac ultrasound images is widely used in assess cardiac function.Particularly, the analysis of ventricle motion is the effective way in order to the degree of assessment ischaemic and infraction formation.Cutting apart or detecting of intracardiac wall is to realize the elasticity of left ventricle and the first step of inotropic quantification.The example of some existing methods comprises based on the cutting apart of pixel/clustering method (for example, colored locular wall is (Color Kinesis) dynamically), the variation of light stream, deformable template and markov stochastic process/field and effective contour/dynamic outline (active snake).In two dimension, three-dimensional or four-dimensional (3D+ time) space, adopt certain methods.

Yet, most of existing cut apart or detection method does not attempt recovering the accurate regional movement of intracardiac wall, and in most applications, ignore component motion along wall.Only also adopt the processing of this simplification along the profile tracker of the method line search of working as front profile.This is not suitable for the zone walls abnormality detection, because the regional movement of unusual left ventricle leaves the normal of profile probably, more must say global motion, such as translation or rotation (because the hands movement of sonographer or respiratory movement of patient), also cause abnormal local motion on the profile.For the detection of zone walls dyskinesia, the global shape of intracardiac wall and its local motion are followed the tracks of in expectation.This information can be used to the further diagnosis of ischaemic and infraction formation.Existence is mated the needs of the detection framework of anatomical structure to utilizing outward appearance and shape.

Summary of the invention

The present invention relates to utilize outward appearance and shape to mate the detection framework of anatomical structure.Use training set of images, in this training group in image annotation object shape or structure.The second training group of image is represented the negative sample of this shape and structure, does not just comprise the image of this object or structure.The sorting algorithm of being trained according to this training group is used to detect the structure in its position.Make this structure and can provide about the counter pair coupling in the training group of the details of the shape of structure and outward appearance.

Another aspect of the present invention relates to a kind of method that is used for detecting the object of the image that comprises the invalid data zone.Being identified for the data mask (mask) of this image, is effective with which pixel in the indicating image.This data mask is represented as the integration mask, and each pixel has corresponding on this pixel and the value of the sum of the valid pixel in the image on this pixel left side in described integration mask.Rectangular characteristic is applied to this image, and described rectangular characteristic has a positive region and a negative region at least.Utilize the integration mask that those effective pixels in the rectangular characteristic are determined.Average brightness value to the zone that comprises inactive pixels asks approximate.By the weighted difference between the summation of brightness value in the positive and negative zone of calculating rectangular characteristic, determine the eigenwert of rectangular characteristic.Utilize this eigenwert to determine whether to detect object.

Another aspect of the present invention relates to a kind of method that is used for the object of detected image.Be the classifier calculated eigenwert in the window of image.Determine whether this eigenwert surpasses predetermined threshold.If this eigenwert surpasses threshold value, then be the eigenwert subsequently of classifier calculated subsequently in the window of image.Make up the value of this eigenwert and eigenwert subsequently.Determine whether the assemblage characteristic value surpasses the combined threshold value of current combination.If the assemblage characteristic value surpasses combined threshold value, then calculate further assemblage characteristic value, comprise further sorter subsequently, until not having subsequently sorter or assemblage characteristic value to be no more than combined threshold value.Last assemblage characteristic value is used to determine whether detected object.

Another aspect of the present invention relates to and a kind ofly is used for the anatomical structure of detected image and makes anatomical structure in the image and system and method that the one or more anatomical structures in the training set of images are complementary.Receive candidate image, and from candidate image, extract eigenwert.The application class function is to detect anatomical structure.If detect anatomical structure, the eigenwert of eigenwert of being extracted by making candidate image and the pairing image in the training group one or more pairing images in the training group of coming recognition image that are complementary then.Come the shape of the anatomical structure in the coupling pairing image of self-training group to be used to determine the shape of the anatomical structure in the candidate image.

Another aspect of the present invention relates to the method that the anatomical structure of the one or more similar shapings in a kind of anatomical structure that is used for making image and the training set of images is complementary.Receive the image of candidate's anatomical structure, and from this image, extract feature.The feature relevant with the anatomical structure of similar shaping compared with candidate's anatomical structure.The shape of at least one the nearest neighbor by being used to the self-training group is determined the shape of candidate's anatomical structure.

Another aspect of the present invention relates to a kind of system and method for deformable shape of the candidate target that is used for the detection and tracking image.This shape is represented at reference mark by a plurality of institutes mark.At least one reference mark of deformable shape in the detected image frame.At each reference mark relevant with candidate target, the uncertain matrix of calculating location.Produce shape, with deformable shape in the expression picture frame subsequently dynamically, wherein this shape comprises the statistical information from the training data group of the image of typical subject.This shape is aimed at the deformable shape of candidate target.Merge this shape and deformable shape, and the current shape of assessment candidate target.

Description of drawings

Describe the preferred embodiments of the present invention below with reference to the accompanying drawings in more detail, wherein similar reference number is represented similar element:

Fig. 1 illustrates the exemplary architecture of ultrasonic cardiography drawing system, and described ultrasonic cardiography drawing system uses the method according to the shape of the intracardiac wall that is used for the detection and tracking left ventricle of the present invention;

Fig. 2 illustrates the typical echocardiogram image of heart;

Fig. 3 a-3d illustrates the example according to the rectangular characteristic of expression Weak Classifier of the present invention;

Fig. 4 illustrate according to the brightness that is used to utilize integral image to determine given window of the present invention and method;

Fig. 5 illustrates according to of the present invention by the rectangular characteristic in the integral image of partial occlusion;

Fig. 6 illustrates the block masks according to the integral image of the Fig. 5 of being used for of the present invention;

Fig. 7 a and 7b illustrate according to H of the present invention _I-1And H _I-1 ^*Relation;

Fig. 8 illustrates the synoptic diagram according to the cascade of lifting of the present invention, as to utilize memory technology;

Fig. 9 illustrates according to the framework that is used for the endocardium of left ventricle Boundary Detection of the present invention;

Figure 10 illustrates the framework that is used for detecting the tumour of three-dimensional data volume according to of the present invention;

Figure 11 illustrates according to the invariant manifold that is used for shape alignment of the present invention;

Figure 12 a and 12b illustrate according to shape alignment of the present invention; And

Figure 13 illustrates according to the uncertainty of the present invention in SHAPE DETECTION and tracing process and propagates.

Embodiment

The present invention relates to be used to detect and mate the method for anatomical structure.To adopt an example of this method to be: by detecting via machine learning or classification and cut apart endocardium of ventricle and epicardial border, and to carry the similar situation of the database of note, detect the zone walls dyskinesia in the heart by identification.Those of ordinary skill in the art it should be understood that the present invention can be used to other and use, and wherein SHAPE DETECTION and coupling are useful, such as but be not limited to identification people's feature, such as facial characteristics or other physical trait.The present invention also can be used to two dimension, the three peacekeeping four-dimension (3D+ time) data analysis, for example such as past that can be in time and the medical analysis of the anatomical structure the heart, lung or the tumour that develop.

In order to describe the present invention, will example be described at the intracardiac wall of the left ventricle that detects human heart.Fig. 1 illustrates the exemplary architecture of ultrasonic cardiography drawing system, and wherein said ultrasonic cardiography drawing system uses according to of the present invention and is used to utilize shape and outward appearance to detect the method for the intracardiac wall of left ventricle.Medical sensor 102 such as ultrasonic transducer is used to patient is carried out inspection.Sensor 102 is used to obtain and the concrete consistent medical measurement of medical inspection.For example, the patient who stands cardiac problems can have the echocardiogram of carrying out for the concrete heart disease of assisted diagnosis.Ultrasonic system provides two, three and four (3D+ times) the dimension image of heart according to various skeleton views.

The information that obtains by sensor 102 is transferred into processor 104, and described processor can be workstation or personal computer.Processor 104 converts sensing data to the image that is transferred into display 108.Display 108 also can transmit other graphical information that relates to image or the table of information.According to the present invention, also provide the data of the initial profile of the intracardiac wall of expression for processor 104.These data can be provided by the user artificially such as doctor or sonographer, or are automatically provided by processor 104.This profile comprises a series of independent points, is illustrated on the display 108 by the motion of processor 104 these points of tracking and with it.

Except the data from medical sensor 102, processor 104 also can receive other data input.For example, processor can receive the data from the database 106 that is associated with processor 104.This data can comprise the subspace model, the potential contour shape of the intracardiac wall of described subspace model representation.These subspace models can be the images of a plurality of patients' of expression left ventricle, perhaps can be the contour shape models based on statistical information that computing machine produces.Utilize known method, such as Bayes nuclear (Bayesian kernel) coupling or based on the method for light stream, processor 104 is followed the tracks of the independent point of contour shapes.Make error accumulation in tracing process up by utilizing multi-template Adaptive matching framework.The form of sentencing covariance matrix at each point is represented the uncertainty of following the tracks of, and wherein uses the subspace shape constraining of nonopiate projection to adopt described covariance matrix subsequently fully.

Fig. 2 illustrates the typical echocardiogram image of heart.Come label that the part of intracardiac wall of the left ventricle of acoustics signal drop-out is arranged with solid line ellipse 208.With dashed lines ellipse 202,204 is represented the estimation of local wall motion.Because acoustic signal is lost, intracardiac wall is the strongest edge in image always not.The feature of echocardiogram image is image fan-shaped of with dashed lines 210,212 expression.The data that do not include usefulness in the zone of fan-shaped outside.

Many detection methods utilize the lifting of Weak Classifier or feature to come object in the detected image.Weak Classifier by making up selected quantity is via the response that promotes, and resulting strong classifier can be realized the high detection rate.Yet known method does not solve the problem of detected object when having other occlusion objects (data outside for example fan-shaped).Weak Classifier is because the errored response of blocking influences the detection of object negatively.

According to one aspect of the present invention, use description in the object detection process, eliminate the method for known influence of blocking now.For example, can handle the echocardiogram image in the mode of not considering the view data (just useless or invalid data) outside fan-shaped.In other words, the data in fan-shaped outside are used as and block processing.

The simple feature that is associated with the image of object is identified as Weak Classifier.The example of this feature is in the rectangular characteristic shown in Fig. 3 a-3d.The value of each rectangular characteristic is poor between the summation of white (the also just being known as) zone of each rectangle and the pixel intensity in grey (also the being known as negative) zone.For the rectangular characteristic shown in Fig. 3 a, negative region is 302, and positive region is 304.For the rectangular characteristic shown in Fig. 3 b, negative region is 308, and positive region is 306.For the rectangular characteristic shown in Fig. 3 c, negative region is 312 and 314, and positive region is 310 and 316.For the rectangular characteristic shown in Fig. 3 d, negative region is 320, and positive region is 318 and 322.

Rectangular characteristic provided complete basis for the fundamental region.For example, if rectangle is 24 * 24 pixels dimensionally, then the quantity of feature is 180000.One of advantage of rectangular characteristic is a computing velocity.By utilizing the intermediate representation that is called integral image (II) as shown in Figure 4, can utilize the operation of little fixed qty to come computation of characteristic values.

Before the calculating of rectangular characteristic, the II of calculating input image (for example echocardiogram image of left ventricle) in advance.(x y), determines brightness value at each pixel among the II.These brightness values are stored in (Fig. 1) in the database 106.In case, just simplify the calculating in all futures greatly for input picture has calculated II.At the position (x in the input picture ₀, y ₀) each pixel of locating, can be by determining at position (x ₀, y ₀) on and at position (x ₀, y ₀) the brightness sum of all pixels on the left side calculate brightness value.In other words, at II (x ₀, y ₀) locate can following definite II subclass:

$\mathrm{II}(x_0,y_0)=\underset{x\≤x_0,y\≤y_0}{\mathrm{\Σ}}I(x,y),---\left(1\right)$

Wherein (x y) is in the position (x, the brightness of the pixel of y) locating to I.

Fig. 3 illustrates and how to determine at rectangular characteristic R _fThe calculating of the brightness value of the II of place.The II at calculating location 408 places, it equals the zone within the solid line 410.The another way that is used to limit the II at 408 places, position is rectangle (A+B+C+R _f) the summation of brightness value.In order to obtain R _fSummation, must carry out additional calculations.The II of position 406 provides the summation in the zone that limits by line 412, and it equals the summation of the brightness value of rectangle (A+C).The II that deducts position 406 from the II of position 408 causes rectangle (B+R _f) II ＇.Then, the II of calculating location 404, it provides the summation in the zone that limits by (A+B).The II that deducts position 404 from II ＇ causes rectangle (A+R _f) II ".At last, " addition, this provides R with the II and the II of position 402 _fSummation.

Yet, at R _fIn pixel comprise that in the situation of blocking, the brightness value of those pixels provides invalid value, this will finally produce the incorrect estimation to rectangular characteristic.Fig. 5 illustrates and comprises the example that blocks 504 integral image 502.Rectangular characteristic 506 is placed in and comprises the position of blocking a part of 504.

According to the present invention, block masks is used to eliminate the effect that is included in the pixel that is blocked in the rectangular characteristic.Figure 6 illustrates the example of the block masks of the II that is used for Fig. 5.When in check environment, obtaining image, block masks can be used, perhaps block masks can be from data, inferred.For example, in surveillance application, known quiescent state background (for example position of door, wall, furniture etc.).Can determine to cause the likelihood of the object in the background of blocking, and use it for the establishment block masks.Another example is a ultrasonoscopy.In ultrasonoscopy, provide fan-shaped position by ultrasound machine, maybe can calculate fan-shaped position, for example the analysis of time variation can produce static inactive area.In case identify fan-shapedly, just can create block masks, in II calculates, to get rid of effectively or to cancel fan-shaped existing.

By will be blocked or otherwise the brightness value of invalid pixel is made as zero, the summation of the brightness value of rectangle will be influenced by incorrect value no longer.Yet, because there are " losing " data now, so summation will be unbalanced.When not existing when losing value, the average brightness value of rectangle summation and rectangle is proportional.Therefore, for the compensating missing value, when existence was blocked, the quantity that has the pixel of effective brightness value by utilization was come approximate average.Can be by at first calculating the quantity that isoboles or block masks obtain valid pixel.

Block masks M comprises Boolean, and wherein the valid pixel value of being endowed 1, and pixel value of being endowed 0 invalid or that be blocked.Can following utilization at current location (x ₀, y ₀) on and at current location (x ₀, y ₀) the quantity of valid pixel on the left side calculate the integration mask:

$\mathrm{IM}(x_0,y_0)=\underset{x\≤x_0,y\≤y_0}{\mathrm{\Σ}}M(x,y).---\left(2\right)$

Be similar to the II of equation (1), can calculate the quantity of valid pixel in the rectangle according to the integration mask with the operation of aforesaid equal number.

Weighted difference between the summation of the brightness in the positive and negative image-region will be provided for the equivalent features value of rectangular characteristic 506.If R ₊Remarked pixel brightness is with on the occasion of zone of making contributions and R _-The zone that remarked pixel brightness is made contributions with negative value, then eigenwert f is as follows:

$f=\frac{n_{-}}{N}\underset{(x,y)\∈R_{+}}{\mathrm{\Σ}}I(x,y)-\frac{n_{+}}{N}\underset{(x,y)\∈R_{-}}{\mathrm{\Σ}}I(x,y),---\left(3\right)$

N wherein _-, n ₊The quantity of representing the valid pixel in negative, positive zone respectively, each zone comprise N pixel.If n _-And n ₊All non-zero then passes through N/ (n _-n ₊) with final eigenwert standardization.By utilizing block masks to calculate the integral image of rectangular characteristic, obtain more accurate result, it causes better object detection.

Particularly under the situation such as the complex object of face or anatomical structure, owing to the detection for object needs calculated big measure feature or part, instrument is used to reduce needed calculated amount, still produces accurate result simultaneously.Generally a kind of such instrument of Shi Yonging is to promote.Generally, promote a plurality of Weak Classifiers of identification or feature.At each Weak Classifier, can calculate a value, described value compares with predetermined threshold then.If the value of Weak Classifier surpasses threshold value, then keep this sorter.If the value of Weak Classifier is lower than threshold value, then refuse this sorter.By asking weighted sum to surpassing all values threshold value, Weak Classifier, can produce strong classifier, it can be used in the object detection.

The modification that promotes is the cascade that promotes.In this technology, sorter is distinguished priority ranking.Calculate first sorter at window, and if it does not satisfy threshold value, window is moved into the another location so.Only keep those positions that the sorter that is calculated surpasses threshold value.Threshold value typically is set at the level place of appropriateness, to allow roomy margin for error.The quantity of the position by reducing to carry out calculating, this method is effective.Yet these methods abandon the output from before sorter.The training of next stage begins with the homogeneous weighting to new sample group.

According to the present invention, keep the value of each sorter that calculates, and use it for the calculating of following sorter so that strengthen after the classifier calculated in stage.By on the new training group of the current generation as shown in Fig. 7 a and 7b, new threshold value T being set _i ^*And relevant parity p _i ^*, the stage is directly used centre strong classifier H before the cascade _I-1Then, before training, with new sorter H based on primitive character _I-1 ^*Error training sample is weighted.The strong classifier of current generation is H _I-1 ^*Weighted sum with selected single tagsort device.In testing process, replace throwing away from the sorter output in stage before, it is utilized T _i ^*With relevant parity p _i ^*Threshold value is set, and is weighted, and be added in the weighted sum from the output of single feature Weak Classifier of current generation according to its error.

" utilizing the cascade of the lifting of storer " can describing as getting off be training algorithm (BCM):

In the superincumbent algorithm, suppose if use H _I-1 ^*, then it will be first.H _I-1 ^*Also can be used in the centre of other single feature Weak Classifier.What all need change is, replaces when the end in stage before, but in the training process of current generation, learns new threshold value and parity.

Can utilize current training group or complete representational checking to organize and carry out H _iAssessment.In last situation, only use H _i, and target is to satisfy the performance objective of current generation (that is per stage 95%); And in one situation of back, should use total cascade classifier, and target is the Comprehensive Performance Objective (that is, 0.95 ') until current generation i.

Another example of the cascade algorithm of the lifting that utilizes storer is shown below.

In the BCM framework, testing process is used for strong classifier in the middle of following each stage of cascade:

Because in the stage before, assessed H _i, additional calculating only is subtraction and the multiplication at each additional phase, wherein current as shown in Figure 8 specimen will be by described additional phase.For each additional category device of being considered, consider the relevant value of sorter with previous calculating, and it is combined with value at added value.Resulting value compares with threshold value then.Net result provides the value of the more accurate indication of object detection.

In case detected object potentially, just can utilize further treatment technology to obtain additional information about image.According to another aspect of the present invention, can be based on the common use of one group of training image application appearance and shape, with object shapes or the anatomical structure in coupling and the detection test pattern.Outward appearance is used to the object in the test pattern or the location of structure.Adopt matching technique then, find similar situation and to provide shape or CONSTRUCTED SPECIFICATION for the material standed for that is detected to concentrate from positive training data.Can detect, locate and mate with hierarchical approaches, to realize more accurate result.The feature that the matching technique utilization is learnt in testing process is to save computing time and to improve matching performance.

According to the present invention, be used for the general framework that object or anatomical structure detect and shape is recovered and comprise three phases: off-line training step, online detection-phase and matching stage.

In off-line training step, positive training sample is stored as the training data group.For example, under echocardiographic situation, the training group will comprise the image of the left ventricle of human heart.The training group will comprise comprehensive group of sample of the sample of differing formed left ventricle and unusual left ventricle.Preferably, (for example left ventricle occupy in the middle of the image in top condition; Image by standardization with the influence of eliminating size and rotation etc.) under image in the training group is shown.In the training stage process, boosting algorithm is applied to image.Use the data that obtained from these and be stored, and can comprise the proper vector of indication Weak Classifier output with training data.

Handle all positive training samples, with the characteristic that remains unchanged.For example, integral translation, rotation and convergent-divergent are the constant conversion (invariant transform) to the left ventricle of human heart.The aligning of correction data directly influences the design of detecting device, that is, each aiming axis need be expanded in testing process.In other words, for example, if cancellation rotation in training data, then detecting device has to search for a plurality of rotations in testing process.The training data that utilization is aligned, learning algorithm are exported selected feature and just are being used for/are bearing the corresponding decision function of classification.According to the present invention, can all training datas of conversion (for example convergent-divergent and rotation), with the detecting device of training institute conversion.

All features of training data (comprising the training data that is transformed) are stored in (Fig. 1) in the database 106.In addition, be each correction data image calculation proper vector, described correction data image comprises Weak Classifier output and their associated weight.Each Weak Classifier is represented the part relevant with object.The proper vector of correction data subsequently can with compare for proper vector that test pattern calculated, have the correction data of similar features to help identification.The location point corresponding to along the point of object outline of each correction data image also is stored, and is used to the form fit stage.

In online detection-phase, by window in test pattern or the data volume or cube are carried out translation, rotation and/or convergent-divergent, be used for the sweeping scheme of image or data volume, to produce candidate data sticking patch (patch).At each material standed for, during whole scanning process or reach object's position, convergent-divergent and/or rotation afterwards.In some cases, a plurality of detecting devices that are transformed of application are faster than conversion material standed for.Can be used to object or material standed for position in the indicating image based on the detecting device that promotes.

In stage, identification appears to comprise those material standed fors (material standed for of winning victory) of object in form fit.The similarity matching algorithm is applied to these material standed fors,, and their associated shape is applied to material standed for the nearest neighbor of retrieval from corresponding training data group.Proper vector is used to form fit, and described form fit is based on Weak Classifier output h/s and their associated weight α/s.Utilize the distance metric in the following space to mate: { α ₁h ₁, α ₂h ₂..., α _Kh _K, wherein K is the quantity of Weak Classifier feature.Also can use the further feature space." pairing (counterpart) " proper vector that the form fit algorithm search is associated with one or more images in the training data.In case obtained coupling, come the data of the relevant matches image in the self-training group can be used to provide details about object shapes and structure.

To utilize two dimension (2D) data to describe example now.Those skilled in the art will appreciate that the present invention also can handle three-dimensional (3D) data and the four-dimension (3D+ time) data.The exemplary framework that Fig. 9 illustrates the data set that utilizes the original image and the endocardial boundary of being followed the tracks of, use the band note of LV frame, detects based on left ventricle (LV) endocardial boundary of study and coupling.

This framework is at first constructed the database of the LV sticking patch orderly group of form, that be aligned (modulus translation, convergent-divergent and rotation) at boundary mark for example or reference mark, wherein its boundary strip note.Utilize this group sample and other negative sticking patch to design and train detection algorithm.For example, can use based on the feature selecting and the sorter construction algorithm that promote.Be all sample calculation proper vectors.

Then, this framework is the detection algorithm of the location employing study of LV.The material standed for that is detected is to comprise local sticking patch LV, that have its correct size and rotation.(by determining this size and rotation with a plurality of possible convergent-divergents and rotation sweep image.)

At last, from the material standed for that is detected, extract proper vector, and compare, to find one or more nearest neighbor with database.For example utilize weighted sum to make up their profile then, to form the profile of the candidate's sticking patch detected, wherein the matching distance of the sticking patch that extremely detected with them of weight is proportional.A possible characteristic type will be the brightness of image (having or do not have double sampling) in the sticking patch.Further feature comprises Weak Classifier output (having or do not have weighting factor).Also can utilize principal component analysis (PCA) (Principal ComponentAnalysis), to select the subclass of this feature.

Figure 10 illustrates the example of the lesion detection in the three-dimensional data volume.This method is similar to method recited above, except 3D neighborhood, 3D feature and 3D scanning are used to detect coupling.

In case detecting aspect shape and the outward appearance and mating object, then just can be along with the past of time the shape of tracing object.Because the rhythmic motion of cardiac muscle, this tracking is important in echocardiogram.In the shape tracing process, measuring uncertainty plays an important role.According to the present invention, with separately the training the part detector applies in image, to develop metastable local appearance, utilize the global shape model to come the constraint portions merging process simultaneously.Use description in automatic shape detection and tracking process, merge best from local detection now, move dynamically and probabilistic Unified frame of subspace shape modeling.The part detecting device that promotes is used to the left ventricle boundary alignment in the ultrasonic cardiography graphic sequence.

Suppose (x, C with N _x), promptly have mean value x and a covariance C _xThe candidate that represents of multidimensional Gaussian distribution before the detection of shape, first step be the situation of the constant conversion of the best following before sample shape x ⁰Among find and have (x, C by N _x), shape N (m, C _m) and from the predicting shape N (x of previous time step _-, Cx _-) shape before the sample of the common PRML that produces.The equivalence formula is to find x ^*, with the summation of the Mahalanobis distance in the shape space of shape space before minimizing and institute's conversion, just,

$x^{*}=\underset{\{T,x^0\}}{\arg \min }{\mathrm{<figure-callout\; id="2"\; label="d"\; filenames="C200480040666E00253.png"\; state="\{\{state\}\}">d</figure-callout>}}^2,---\left(4\right)$

${\mathrm{<figure-callout\; id="2"\; label="d"\; filenames="C200480040666E00253.png"\; state="\{\{state\}\}">d</figure-callout>}}^2={(x^{0\&prime;}-m)}^T{C_m}^{-1}(x^{0\&prime;}-m)+{(x^0-x)}^T{C_x}^{-1}$

$(x^0-x)+{(x^0-x_{-})}^T{C_{x\_}}^{-1}(x^0-x_{-}),---\left(5\right)$

X wherein ^{0 ＇}=T (x ⁰), wherein T is constant conversion.

Under the situation of shape before a plurality of candidates, produce the highest likelihood, also consider to detect the candidate of the likelihood value among the figure before shape win in the judgement time.Equation (5) need be in position and conversion in optimization, and even do not have separating of closing form for simple conversion, such as the similarity conversion that only allows translation, rotation and convergent-divergent yet.Can numerically seek whole the best by iteration, may be too expensive but calculate.

Difficulty comes from the following fact, and promptly the stream shape (shape just) of being crossed over through all possible conversion by shape before arbitrarily generally, particularly the group space dimensionality did not intersect with the shape subspace in relative hour.In the present invention, the shape subspace has from 6 to 12 dimension, and full Euclid (Euclidean) space has 〉=and 34 dimension.Figure 11 illustrates the invariant manifold of the shape alignment that is used for illustrating conceptually this relation, the stream shape that the one dimension Gaussian distribution 1106 of shape vector X and sloping shaft 1104 and expression subspace model was crossed over before wherein bold curve 1102 was described.Generally, stream shape will not intersected (just sloping shaft 1104 comprises model barycenter M) with the shape subspace.Omit prediction at this, perhaps can regard X as detect and predict amalgamation result.The present invention relates to two step optimization approach of solution as a whole, have separating at the closing form of two steps.Can easily explain this scheme with reference to Figure 11: first step is to utilize C _xIn information go to X from X ^*, or in other words, find optimal mapping from X to M.Second step is to be used to from C ^MAdditional information from X ^*Go to X ^MFirst step is known as alignment procedures, and second step is known as the constraint step.

The target of alignment procedures is to consider in the component uncertainty in the process of model transferring of shape and its covariance matrix before.The at first following d that minimizes ²:

${\mathrm{<figure-callout\; id="2"\; label="d"\; filenames="C200480040666E00253.png"\; state="\{\{state\}\}">d</figure-callout>}}^2={(m-x\&prime;)}^T{C}_x^{\&prime;-1}(m-x\&prime;)---\left(6\right)$

Wherein x '=T (x), and C ' _X=T (C _X).For contracted notation, suppose prediction N (x _-, C _X-) be merged into N (x, C _X).

When T is the similarity conversion, have:

$x\&prime;=\mathrm{Rx}+t,---\left(7\right)$

Wherein t is the translation vector with two free parameters, and R is block diagonal matrix, and wherein each piece is

$R_i=\left(\begin{array}{cc}a& -b\\ b& a\end{array}\right)---\left(8\right)$

Utilize pure algebraically (straight algebra), can following rewrite equation (6):

d ²＝(R ^-1(m-t)-x) ^TC _x ^-1(R ^-1(m-t)-x)

＝(T ^-1(m)-x) ^TC _x ^-1(T ^-1(m)-x)(9)

By to four free parameter differentiations among R and the t, can obtain separating of closing form.Figure 12 a and 12b are illustrated in the shape alignment under probabilistic situation of considering and not considering the some position.Figure 12 a is illustrated in the shape alignment under probabilistic situation of not considering the location.Figure 12 b illustrates and utilizes the probabilistic shape alignment of different variance.Oval 1202-1212 describes the covariance about a position, expression piece diagonal angle C _xIn information.Intuition is more to believe to have the more point of high confidence level.

In case shape is aimed at model before, the shape with PRML that is produced by two competition information sources (just, the detection/prediction of aligning is relative with (subspace) model) is determined.Under the situation of total space model, formula directly relates to the information of utilizing Gaussian source or BLUE (best linear unbiased estimator (Best Linear Unbiased Estimator)) and merges.

Suppose two noisy measurements of identical n dimension variable x, each measurement is characterised in that the multidimensional Gaussian distribution, i.e. N (x ₁, C ₁) and N (x ₂, C ₂), the maximal possibility estimation of x is the Mahalanobis distance D with revision ₂(x, x ₂, C ₂) the point of minimum summation.Suppose C with being without loss of generality ₂Be unusual.Utilize C ₂=U Λ U ^TSvd, U=[u wherein ₁, u ₂..., u _n], while u _iBe orthonormal, and Λ=diag{ λ ₁, λ ₂..., λ _p, 0 ..., 0} is to x ₂Mahalanobis distance as follows:

$D^2(x,{\mathrm{<figure-callout\; id="2"\; label="x"\; filenames="C200480040666E00253.png"\; state="\{\{state\}\}">x</figure-callout>}}_2,C_2)={(x-x_2)}^T{C}_2^{-1}(x-x_2)$

$=\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{\mathrm{\&lambda;}}_i^{-1}{\left[U^T(x-x_2)\right]}^2$

(10)

Work as λ _iBe tending towards at 0 o'clock, D ₂(x ₁, x ₂, C ₂) the trend infinity, unless U ^T ₀X=0, wherein U ₀=[u _P+1, u _P+2..., u _n].Suppose the initial point of subspace here by luv space with being without loss of generality.Because x ₂Occupy in the subspace, so U ₀ ^Tx ₂=0.

Because U ₀ ^TSo x=0 is d ²Become now:

${\mathrm{<figure-callout\; id="2"\; label="d"\; filenames="C200480040666E00253.png"\; state="\{\{state\}\}">d</figure-callout>}}^2={(U_{p}y-x_1)}^T{C}_1^{-1}(U_{p}y-x_1)+$

${(U_{p}y-x_2)}^{\mathrm{<figure-callout\; id="2"\; label="T\; C"\; filenames="C200480040666E00253.png"\; state="\{\{state\}\}">T</figure-callout>}}{C}_{}^{}{}_2^{+}(U_{p}y-x_2)$

(11)

Wherein y is 1 * p vector.

To the y differentiation, produce the merging estimator of subspace:

$y^{*}=C_{y*}{U}_p^T(C_1^{-1}{\mathrm{<figure-callout\; id="1"\; label="x"\; filenames="C200480040666E00251.png,C200480040666E00252.png"\; state="\{\{state\}\}">x</figure-callout>}}_1+{\mathrm{<figure-callout\; id="2"\; label="C"\; filenames="C200480040666E00253.png"\; state="\{\{state\}\}">C</figure-callout>}}_2^{+}{x}_2),$

(12)

$C_{y*}={\left[U_p^T(C_1^{-1}<figure-callout\; id="2"\; label="+\; C"\; filenames="C200480040666E00253.png"\; state="\{\{state\}\}">+</figure-callout>C_{}^{}{}_2^{+})U_p\right]}^{-1},$

(13)

Utilize the equivalent expression in the luv space:

$x^{*}=U_p{y}^{*}=C_{x*}(C_1^{-1}{\mathrm{<figure-callout\; id="1"\; label="x"\; filenames="C200480040666E00251.png,C200480040666E00252.png"\; state="\{\{state\}\}">x</figure-callout>}}_1+{\mathrm{<figure-callout\; id="2"\; label="C"\; filenames="C200480040666E00253.png"\; state="\{\{state\}\}">C</figure-callout>}}_2^{+}{x}_2)$

(14)

$C_{x*}=U_p{C}_{y*}{U}_p^T$

(15)

Can illustrate With

Be x ^*And y ^*Corresponding covariance matrix.

Alternatively, can write equation (12) and equation (13) as getting off:

$y^{*}={(U_p^T{C}_1^{-1}{U}_p+{\mathrm{\&Lambda;}}_p^{-1})}^{-1}(U_p^T{C}_1^{-1}{\mathrm{<figure-callout\; id="1"\; label="x"\; filenames="C200480040666E00251.png,C200480040666E00252.png"\; state="\{\{state\}\}">x</figure-callout>}}_1+{\mathrm{\&Lambda;}}_p^{-1}{y}_2)$

(16)

Here, y ₂Be x ₂By U _pCoordinate transforming in the subspace of being crossed over, and Λ _p=diag{ λ ₁, λ ₂..., λ _p.The BLUE that equation (16) can be counted as in the subspace of two Gaussian distribution merges, and one of them Gaussian distribution is N (y ₂, Λ _p), and another Gaussian distribution is the N (x in the subspace ₁, C ₁), N ((U ^T _pC ₁ ^-1U _p) ^-1U ^T _pC ₁ ^-1x _-1, (U ^T _pC ₁ ^-1U _p) ^-1) common factor.

Above-mentioned subspace merge into (subspace) model constrained, will (it is probabilistic to have different variance) shape measure and principal component analysis (PCA) (PCA) shape be considered as two information sources general formula be provided.Below, add the 3rd source, its expression is according to the performance prediction of following the tracks of.The vital benefit that obtains from tracking except that detecting is additional information and the information merging in time from the system dynamics of domination prediction.Based on top analysis, separating of equation (4) has following form:

$x_{+}=C_{x_{+}}(T\{{(C_{x\_}+{C_x}^{-1})}^{-1}$

$(C_{x_{-}}{x}_{-}+{C_x}^{-1}x)\}+{C_m}^{+}m),$

(17)

This is separated the information from detection, shape and performance prediction is placed in the united frame.When also limiting predicting shape in the subspace, subspace recited above BLUE formula can be applied in the conversion T with nested mode.Prediction N (x _-, C _X-) comprise information from system dynamics.This information is used to global motion trend, such as expansion with shrink and translation slowly and rotation are encoded.Utilize traditional method, the predictive filter in being provided with such as Kalman, can obtain N (x _-, C _X-):

$C_{x_{-}}={\mathrm{SC}}_{x_{+},\mathrm{prev}}{S}^T+Q,$

(19)

Wherein the system dynamics equation is

x _-＝Sx _+，prev+q，

(20)

And Q is the covariance of q, and " prev " expression is from the information of previous time step.

Figure 13 illustrates the synoptic diagram of analytical procedure, wherein propagates the uncertainty of detection in steps by institute.At each frame place,, a plurality of detection material standed fors are assessed by the likelihood of comparison a plurality of detection material standed fors in the context of two shapes and based on the prediction of system dynamics according to former frame.Ellipse such as 1302-1316 illustrates the position uncertainty.In alignment procedures, utilize shape that uncertainty is carried out conversion, in likelihood estimation and tracing process, utilize the prior imformation of model and prediction to merge uncertainty.

Described and be used to utilize outward appearance and shape to detect and mate the embodiment of the method for anatomical structure, but it may be noted that according to top instruction, those skilled in the art can make amendment and change.Therefore it should be understood that and in disclosed specific embodiments of the invention in the scope and spirit of the present invention as defined by the appended claims, to change.Therefore described the present invention, set forth the content of and expectation protection claimed in the appended claims by patent with the desired details of Patent Law and characteristic.

Claims (8)

1, a kind of method that is used for detecting the object of the image that comprises the invalid data zone, this method may further comprise the steps:

Being identified for the data mask of this image, is effective to indicate which pixel in this image;

This data mask is expressed as the integration mask, and each pixel has corresponding on this pixel and the value of the sum of the valid pixel in the image on this pixel left side in described integration mask;

Rectangular characteristic is applied to this image, and described rectangular characteristic has a positive region and a negative region at least;

Utilize this integration mask to determine the quantity of effective pixel in this rectangular characteristic;

Average brightness value to the zone that comprises inactive pixels asks approximate;

By the weighted difference between the summation of the brightness value in the positive and negative zone of calculating this rectangular characteristic, determine the eigenwert of this rectangular characteristic; And

Utilize this eigenwert to determine whether to detect object.

The process of claim 1 wherein that 2, described data mask is given those invalid pixel values 0.

3, the method for claim 2, wherein, described data mask is given those effective pixel values 1.

The process of claim 1 wherein that 4, described brightness value is the gray-scale value of pixel.

The process of claim 1 wherein that 5, described image is a ultrasonoscopy.

6, the method for claim 5, wherein, described to liking left ventricle.

7, the process of claim 1 wherein, described to liking face.

The process of claim 1 wherein that 8, described invalid data zone is partly to hinder blocking of object.

Images