WO2001067960A2

WO2001067960A2 - System and method for processing an image sequence

Info

Publication number: WO2001067960A2
Application number: PCT/CA2001/000282
Authority: WO
Inventors: Wido Menhardt
Original assignee: Cedara Software Corp.
Priority date: 2000-03-16
Filing date: 2001-03-07
Publication date: 2001-09-20
Also published as: WO2001067960A3; AU2001242127A1

Abstract

A method for determining a frequency of motion of an object from a time sequence of images of said object, said method comprising the steps of applying an autocorrelation function to individual pixels in across said image sequence; computing an average autocorrelation for each image to produce an amplitude pattern of said average autocorrelation; applying a filter to said amplitude pattern; applying a peak detector to said filtered amplitude; and comparing said peak to its time of occurrence in said image sequence to thereby determine frequency of motion of said object.

Description

SYSTEM AND METHOD FOR PROCESSING AN IMAGE SEQUENCE

This present invention relates to the field of imaging and in particular to a system and method which utilizes medical imaging for detecting, monitoring and measuring the movement of a body or portions of the body such as the heart and lungs.

BACKGROUND OF THE INVENTION

Various methods have been implemented in the art for detecting and monitoring the periodic movement of organs within a patient's body. For example, the ubiquitous stethoscope is used for detecting the audible acoustic impulses generated by a beating heart. The acoustic impulses are tallied in a predetermined time period, the tally corresponding to an average pulse rate that in turn may be correlated to the heart beat frequency of the patient. Other methods make use of more advance technology for monitoring movement of the heart or lung. These systems may be divided into several generally categories namely: stethoscopes and acoustic monitors; ultrasound monitors and magnetic resonance systems; optical imaging technology and non-acoustic pulse echo radar monitors. By way of background the reader is referred to United States Patent No. 5,766,208 which provides a comprehensive discussion of each of the above types of systems.

While the above methods and apparatus is suitable in certain applications for obtaining cardiac frequency and breathing rate, the use of such technology may not be feasible or efficient in other applications. In MRI imaging or X-ray angiography, the use of the above methods may introduce unnecessary features in the image sequence such as; for example, an ECG electrode may in fact obscure critical areas of a patient's pathology being investigated.

On the other hand, certain cardiac imaging modalities such as X-ray angiography and echocardiography create dynamic image sequences of the heart or lung. For certain applications, it is useful or necessary to have information about the cardiac frequency in these image sequences. Traditional methods as described above are normally used to obtain cardiac frequency during an imaging application.

It has also been recognized that in certain imaging applications, such as magnetic resonance imaging (MRI), the measurement of MR signals exhibit smaller errors if the measurement of the amount of signals is synchronized to a cyclic movement of the body. One such method described in United States Patent No. 5,987,348 that describes an ECG- triggered magnetic resonance imaging method and apparatus. The patent discloses a method of imaging a portion of a body placed in a static magnetic field by means for magnetic resonance and determines an acquisition period from ECG data in order to synchronize measurement of MR signals to a cyclic movement of the body. A drawback of this patent is that it requires a separate ECG measurement instrument. The acquisition sensors of which must be carefully placed so as not to obstruct the MR imaging at the location of interest.

Accordingly, there is a need for a method and system for determining these periodic movements of the patient's organs, such as cardiac frequency, directly from the image sequence without the need for external equipment such as an ECG and such like and which mitigates at least some of the above disadvantages.

SUMMARY OF THE INVENTION

The present invention seeks to provide a solution to the problem of determining periodic movement of selected portions of a body directly from an image sequence without the need for external measuring equipment such as ECGs.

In accordance with this invention, there is provided a method for measuring a periodicity of a recurring pattern contained in a time sequence of images comprising the steps of:

(a) processing a plurality of said images for detecting a pattern according to a first criterion; and

(b) processing the detected pattern according to a second criterion to determine a measure of periodicity of said recurring pattern.

In accordance with a further aspect the first criterion is an autocorrelation function. In accordance with a further aspect the first criterion is a Fourier analysis.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features of the preferred embodiments of the invention will become more apparent in the following detailed description in which reference is made to the appended drawings wherein:

Figure 1 is a schematic diagram depicting the components of a frequency monitoring and detecting system according to an embodiment of the present invention;

Figure 2 is a schematic diagram showing a sequence of images; Figure 3 is a schematic diagram showing an arrangement of pixels in a single image;

Figure 4 is a generalized flow chart showing the determination of a heartbeat frequency according to an embodiment of the present invention;

Figure 5 is a graphical plot showing an autocorrelation coefficient versus image number;

Figure 6 is a section of the graphical plot of figure 5; and

Figure 7 is a graphical plot showing an output of a highpass filter applied to the graphical plot of figure 5.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following description, like numerals refer to like structures in the drawings. Referring to figure 1, a system for detecting, monitoring and displaying periodic motion of a subject or portions of a subject, from a sequence of images, in accordance with one embodiment of the present invention is indicated generally at numeral 10. The present embodiment will be described with reference to a specific cardiac imaging modality such as x-ray angiography. Other imaging modalities may equally well be used to create the dynamic image sequences. The embodiment is also described with reference to the determination of a heartbeat frequency. The invention may be applied to detecting motion in the lungs, breathing or one or more combinations of motions present in a sequence of images.

Turning back to figure 1, the system 10 comprises an x-ray source 12, a timebase 14 for triggering the x-ray source 12 at a predetermined rate, an image plane 16 for receiving two dimensional x-ray image of the irradiated subject, a computer 20, a program 30 for running on the computer 20, a database 22, user interface 24, and a display 26. Usually, during an imaging procedure, x-rays are passed through the patient 18 and an image is projected onto the plane 16. The image is converted to a two dimensional array of pixels and forwarded to the computer 20 for viewing as a displayed image on display 26 or stored as a sequence of images 28 in a data-set in the database 22. The program 30 may process the data-set 28 or process the images in real time to determine motion of the portion of the body being irradiated. The user input 24 provides facility for an operator to interact with the system, and more particularly, for selecting areas of the displayed image 26 for isolating areas of the image to be processed or to set various parameters of the system. The methods and systems for acquiring, displaying and generating image sequences is well known in the art is described in various patents and patent applications owned by the current assignee and incorporated herein by reference.

The computer 20 is programmed in a manner, which is described in more detail below, to operate in accordance with the following description of the preferred embodiment.

As shown in figure 3(a) and (b), typically each image is comprised of a two dimensional array of individual pixels 304.

The present invention is based on the observation that motion may be reflected in successive sequences of images. The individual pixels 304 across the sequence of images may show variation in intensity in synchronization with the motion being monitored. For example, in order to detect cardiac motion, images will be taken of the heart and its surrounding areas. Thus, image areas, which are covered by the heart, will show changes in brightness in synchronism with the heartbeat. It may be noted that some pixels in the areas covered by the heart may not show any variation in a sequence of images, such as a pixel, which is contained within the area of heart during acquisition of the image sequence. On the other hand pixels in regions immediately outside of the heart may not show any variation, as these regions may also not move. Finally, all pixels may show variation synchronized with breathing. However breathing motion has in general a different frequency than the heartbeat. Thus if the heartbeat frequency is to be monitored some filtering may be employed to select the motion or frequency of the cycle of interest.

Turning now to figure 4, there is shown a flow chart that depicts the operation of the program 30 for controlling the computer 20. The sequence of steps performed by the program is indicated by the sequence of boxes 402 to 414. In box 402, the program inputs a sequence of images 28 for processing along with a value indicative of the image generation frequency f. Depending on the imaging technique and the frequency of the cycle being measured, a new image should be generated several dozen times per cycle or at a rate f. As seen in figure 3(b) the rows are represented along an X-axis and the columns of the image are represented along a Y-axis. Each image for different times is referenced through a time parameter t and the value p(x,y,t) corresponds to the intensity of an pixel at location x,y for the image at time t.

At block 404, the program computes a periodicity measurement for the sequence of images for each frequency of the cycle of interest. The expectation would be that for the frequency of the heartbeat, the periodicity measure would be strong, and that it would also be strong for breathing motion (if present). Various mathematical techniques can be used, including Fourier analysis and autocorrelation in time for extracting this periodicicty. The invention simply relies on some periodicity measure, regardless of how it has been obtained. In a preferred embodiment the autocorrelation is computed over time t for each pixel position x,y. Those pixels whose intensities change periodically will have large autocorrelations at the frequencies corresponding to the period motion. Other pixels may have low autocorrelations. Autocorrelation coefficients measure the degree of correlation between neighboring data observations in a time series. Assuming the time series is y_t = p(x,y,t) t=l,2, ..., the autocorrelation coefficient is estimated from sampling observations as follows:

where r_& describes the autocorrelation of pixels andy_t+k . These coefficients r* are stored in memory.

At block 406, the program computes the average autocorrelation for all pixel positions x,y for each image in the sequence. The average autocorrelation number r_t /where t = 1,2,3 .., corresponding to image t = 1,2,3 .., is stored in memory for further processing

At block 408, the program computes those frequencies, which are within the expected range

by applying a high-pass filter over the average autocorrelation ' . The high-pass filter over the autocorrelation will retain those frequencies which are within the expected range of the periodic motion of interest, but eliminate frequencies that are lower, such as breathing. In practical applications, heartrates, for example that are outside of a well-defined range will lead to an interruption of the clinical procedure.

The filter corner frequency is chosen to be somewhere between the two frequencies at issue, heart beat and breathing. They are fairly far apart (60 beats/minute and 15 breaths/minute for humans) and is typically implemented in software as a digital filter.

At block 410 the program computes the peaks in the high-pass filtered autocorrelation. The first peak will represent frequency of the motion of interest. Thus since the image rate f ( or frame rate) and the image number t_p at the first peak is known, the frequency will be t_pf Hz. Subsequent peaks may be used to obtain an average frequency. Referring now to figures 5, 6 and 7, there is shown a graphical representation of the subject invention as applied to an X-ray angiography sequence of a dog's heart. The frame rate fused is 15 frames per second. As can be seen, in figure 5 a plot is made of the correlation on the Y-axis versus the fame number represented on the x-axis. There is a strong correlation for approximately 60 frames, or .25 Hz (base frequency). There are also strong correlation for multiples of 60 frames - these are simply harmonics of the base frequency. In this example, the .25 Hz motion represents breathing motion (15 intakes per minute).

An enlargement of the autocorrelation as shown on figure 6 shows an overlying periodicity in the range of the heartbeat (approximately frame 7, which corresponds to 2 Hz, or 120 beats per minute).

A high-pass filter matched to the expected heart frequency range yields a filtered autocorrelation as shown in figure 7. The first peak in the high-pass filtered autocorrelation is at frame 8, corresponding to approximately 2Hz, or 120 heartbeats per minute, which is within the expected range of a dog's heart.

Although the present embodiment has been described with respect to X-ray angiograms, the method of the present invention is not limited thereto. Furthermore, the method is not limited to Cardiology - breathing motion could also be detected with this technique in other applications.

Periodicity is determined by using autocorrelation as described above, however other techniques may be used equally well, such as Fourier analysis. Also subsequent selection of the highest peak within a given frequency window may be used.

For performance reasons, the method can be applied to subregions of the image. If the heart outline is known (as is the case in certain applications), then the calculations can be limited to this region. Alternatively, only every nth row and column (describing a grid over the image) can be used.

The method can be applied to any given image sequence. For real-time applications, it may be more useful to apply it to a small retrospective window (1 to 2 seconds), and use it as a predictive measurement for the next 1 to 2 second window, on a continuous basis.

Although the invention has been described with reference to certain specific embodiments, various modifications thereof will be apparent to those skilled in the art without departing from the spirit and scope of the invention as outlined in the claims appended hereto.

Claims

THE EMBODIMENTS OF THE INVENTION IN WHICH AN EXCLUSIVE PROPERTY OR PRIVILEGE IS CLAIMED ARE DEFINED AS FOLLOWS:

1. A method for measuring a periodicity of a recurring pattern contained in a time sequence of images comprising the steps of:

2. A method as defined in claim 1, each image in said time sequence being a two dimensional array of pixels.

3. A method as defined in claim 2, said first criterion includes the steps of: applying an autocorrelation function over time t for each pixel; and computing an average autocorrelation for each image to produce an amplitude pattern of said average autocorrelation.

4. A method as defined in claim 3, including the step of applying a filter to said amplitude pattern.

5. A method as defined in claim 4, said second criterion including the steps of detecting a peak value in said amplitude pattern; and comparing said peak to its time of occurrence in said image sequence to thereby determine said measure of periodicity.

6. A method for determining a periodicity measure from a sequence of images of a body having a plurality of motions therein, comprising the steps of:

(b) selecting from said detected pattern at least one of said motions according to a second criterion to determine a measure of periodicity of said selected motion.

7. A method as defined in claim 6, said body including breathing and heart motion and said selected motion being said heart motion.

8. A method for determining a heart rate from a sequence of images of said heart, comprising the step of:

(a) processing a plurality of said images to extract a periodicity measure therefrom.

9. A method for determining a frequency of motion of an object from a time sequence of images of said object, said method comprising the steps of:

(a) applying an autocorrelation function to individual pixels across said image sequence;

(b) computing an average autocorrelation for each image to produce an amplitude pattern of said average autocorrelation;

(c) applying a filter to said amplitude pattern;

(d) applying a peak detector to said filtered amplitude; and

(e) comparing said peak to its time of occurrence in said image sequence to thereby determine frequency of motion of said object.