US20120078098A1 - Collagen Density and Structural Change Measurement And Mapping in Tissue - Google Patents

Collagen Density and Structural Change Measurement And Mapping in Tissue Download PDF

Info

Publication number
US20120078098A1
US20120078098A1 US13/285,635 US201113285635A US2012078098A1 US 20120078098 A1 US20120078098 A1 US 20120078098A1 US 201113285635 A US201113285635 A US 201113285635A US 2012078098 A1 US2012078098 A1 US 2012078098A1
Authority
US
United States
Prior art keywords
collagen
tissue
pixel
changes
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/285,635
Inventor
Everette Clifton Burdette
Scott P. Huntley
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to US13/285,635 priority Critical patent/US20120078098A1/en
Publication of US20120078098A1 publication Critical patent/US20120078098A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/52Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S15/00
    • G01S7/52017Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S15/00 particularly adapted to short-range imaging
    • G01S7/52053Display arrangements
    • G01S7/52057Cathode ray tube displays
    • G01S7/52071Multicolour displays; using colour coding; Optimising colour or information content in displays, e.g. parametric imaging
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/08Detecting organic movements or changes, e.g. tumours, cysts, swellings
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10132Ultrasound image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30024Cell structures in vitro; Tissue sections in vitro

Definitions

  • This invention relates to tissue monitoring and mapping equipment and methods, and more particularly, to methods and systems for monitoring and mapping changes in the collagen content and/or structures in tissue.
  • Collagen is the major insoluble fibrous protein in the extracellular matrix and in connective tissue. There are at least nineteen types of collagen, but 80% to 90% of the collagen in the body is composed of the conventional well-known Type I, Type II, and Type III forms.
  • the collagen molecule is a triple helix, each helical coil a biological polypeptide polymer constructed from glycine (C 2 H 5 NO 2 ), alanine (C 3 H 7 NO 2 ), proline (C 5 H 9 NO 2 ), and hydroproline (C 5 H 9 NO 3 ). As the three helices wrap around each other, hydrogen bonds form between each helix to maintain the structure. Collagen molecules pack together to form long, thin fibrils.
  • the most prominent collagen types are:
  • Type I The primary component of tendons, ligaments, and bones.
  • Type II The primary protein constituent (more than 50%) in cartilage.
  • Type III Strengthens the walls of hollow structures like arteries, the intestine, and the uterus.
  • Type IV Forms the basal lamina (also called the basement membrane) of epithelia.
  • collagen fibrils are found in many environments within the body.
  • the strength and relative elasticity of the collagen structure allow tendon and ligament structures to function in their role which requires great strength to manipulate skeletal structures.
  • Up to 90% of the dry weight of tendons is collagen (at least 30% of the total weight), up to 50% of the dry weight of articular cartilage and synovial tissue is collagen (5% to 30% of total weight).
  • Collagen comprises 75% of the dry weight of skin.
  • the helical structure and the nature of the hydrogen bond cross-linking of the collagen molecule cause it to react when heated.
  • the cross-linking bonds break and the helix, which is a type of spring structure, collapses somewhat.
  • the result is a shrinkage, not unlike the process which occurs when a woolen garment is washed in hot water and heated in a dryer.
  • the amount of shrinkage is a function of time and temperature [see FIG. 1 ] and is well known.
  • Some therapy systems measure tissue temperature during treatment using invasive probes and predict associated tissue modification, while others estimate the temperature based on power-time-temperature parametric curves established from imperical measurements. But, because thermal dose, and thus collagen shrinkage, is an integral function of both time and temperature in combination with spatial distribution, and since slight variations in temperature during a specific time period can cause dramatic changes in collagen shrinkage [see FIG. 3 ], temperature measurement alone is not truly sufficient. In response to the need to effectively determine dosage of the thermal treatment, a noninvasive measure of the thermal dose or its affects, including collagen changes, is badly needed.
  • a 20% shrinkage in a collagen structure having an initial collagen content of 30% should cause a 25% increase in the % collagen by weight of the tissue structure. Since many collagen structures are 5% to 30% collagen (by weight), a 25% increase in density due to therapeutic shrinkage would cause a change in collagen content of 1.25% to 7.5%. These changes would correspond to changes in acoustic velocity of 6.25 m/sec to 37.5 m/sec, for initial collagen contents of 5% and 30%, respectively (see FIG. 5A ). These changes would correspond to changes in acoustic attenuation of 0.833 db/m to 5 db/m, respectively [see FIG. 5B ].
  • the structural patterns of the tissue change permanently as a function of thermally-induced necrosis and density changes. These patterns may be characterized in an analysis of the structures in the 2D or 3D acoustic image or in 2D spatial maps of acoustic signals that are reflected from or transmitted through the tissue in those locations. Applying pattern recognition methods and/or expert system techniques to the backscatter image or signal mapping data can yield useful information in addition to attenuation and velocity measurements alone.
  • Ultrasound (acoustic) imaging has been used in medical imaging for years to differentiate tissue structures. These imaging techniques have examined acoustic properties of ultrasound waves as they travel through and reflect off of tissue to distinguish tissue types and the boundaries between those types. These devices map tissue in two dimensions (along a plane in line with an imaging transducer) or three dimensions (by using multi-element imaging transducers or single-line imaging transducers whose focus or location changes during the process. Two of the acoustic properties used in these imaging techniques are acoustic velocity and acoustic attenuation. Further, structural change analyses using pattern recognition methods can provide an ability to track changes in tissue structure/collagen structure from baseline using backscattered image information.
  • the subject invention therefore includes a system and method for mapping acoustic changes from an ultrasound image of tissue induced by a structural change or thermal injury and mapping that information in a pixilated form. That is, each pixel, or square, in a two dimensional image or each voxel, or cube, in a three dimensional image, will be assigned acoustic characteristics based on the backscattered signal from that region and/or based on the transmitted signal through a specific tissue region. In particular, such system and method shall assign acoustic velocity values and acoustic attenuation values to each of the pixels.
  • the system and method will assign change values, and assign changes in collagen content concentration to those change values, to the acoustic properties for each pixel.
  • the changes preferably are color-coded to make interpretation of the changes easier for users who are not familiar with interpreting ultrasound images.
  • An object of the invention is to provide an improved system and method to assign patterns or classes to groups of pixels or voxels for specific collagen changes based upon the backscatter patterns in the ultrasound image. Specific types of changes in patterns are correlated with known classes of patterns for groups of pixels.
  • Another object of this invention is to provide an improved system and method to correlate changes in the acoustic properties directly to changes in the collagen content in tissue and use those changes in acoustic properties to provide feedback for the application of thermal energy during a collagen shrinkage procedure.
  • Another object of this invention is to provide an improved system and method to determine acoustic property changes characteristic of the spatial representations of an acoustic signal and using those assignments to create a two dimensional or three dimensional map of the changes in collagen concentration of a tissue structure.
  • Another object of this invention is to provide an improved system and method to use color-coding when displaying the two dimensional or three dimensional maps of changes in collagen density.
  • Another object of this invention is to provide an improved system and method to use color-coding when displaying the two dimensional or three dimensional maps of changes in structural patterns in groups of pixels/voxels which correspond to changes in collagen structure and/or content.
  • Another object of this invention is to provide an improved system and method to measure the structural patterns of the tissue that are affected as a function of thermally-induced necrosis and density changes. These patterns may be characterized in an analysis of the structures in the acoustic image or in 2D spatial maps of acoustic signals that are reflected from or transmitted through the tissue in those locations. Applying pattern recognition methods to the image or signal mapping data can yield useful information in addition to attenuation and velocity measurements alone.
  • Another object of this invention is to provide an improved system and method to cross-correlate each of the types of measured changes in pairs or in total with each other to provide a sensitive system and method for determining changes in tissue collagen, including shrinkage and tissue necrosis.
  • Another object of this invention is to provide an improved system and method to use the measured tissue changes to correlate to tissue damage with changes in tissue structure and/or acoustic property changes as a result of treatment.
  • this invention is applicable for monitoring collagen shrinkage procedures in all tissue structures of all types, both external and internal.
  • FIG. 1 shows thermal effects on tissue
  • FIG. 2 shows strength of collagen containing tissue post thermal remodeling
  • FIG. 3 shows time/temperature influences on collagen shrinkage
  • FIG. 4A shows relative changes in acoustic velocity and FIG. 4B acoustic attenuation properties with respect to changes in the collagen content of tissue;
  • FIG. 5A shows changes in acoustic velocity as a function of initial collagen content and percentage of collagen shrinkage
  • FIG. 5B shows changes in acoustic attenuation as a function of initial collagen content and percentage of collagen shrinkage
  • FIG. 6 shows a system for monitoring and mapping of tissue collagen shrinkage using ultrasound backscatter information and/or measured acoustic data reflected from or transmitted through tissue structures
  • FIG. 7 shows a block diagram of one form of the tissue mapping workstation
  • FIG. 8A shows a diagram of a neural element with four inputs for determining changes in backscatter associated with a structural change in a small tissue region to determine the magnitude of change for a pixel or group of pixels; and FIG. 8B shows a preferred neural network with three inputs for each group of pixels;
  • FIG. 9A shows a diagram of the surrounding pixels used to define a set of vectors used to determine the transfer function for a single pixel
  • FIG. 9B shows a diagram of groups of pixels used to define the vectors for a transfer function for a region of interest group made up of multiple pixels
  • FIG. 10 shows a flow diagram of a method for monitoring and displaying on a workstation the changes in collagen/tissue structure during a therapy procedure.
  • the subject invention includes a system and method for mapping acoustic changes from an ultrasound image of tissue induced by a structural change or controlled thermal injury and mapping that information in a pixilated form. That is, each pixel, or square, in a two dimensional image or each voxel, or cube, in a three dimensional image, will be assigned acoustic characteristics based on the backscattered signal from that region and/or based on the transmitted signal through a specific tissue region. In particular, such a system and method assigns acoustic velocity values and acoustic attenuation values to each of the pixels.
  • the system and method will assign change values, and assign changes in collagen content concentration to those change values, to the acoustic properties for each pixel.
  • the changes are most preferably color-coded to make interpretation of the changes easier for users who are not familiar with interpreting ultrasound images.
  • the system 10 includes an ultrasound imaging device 12 to which a tissue change mapping workstation or system 14 is connected.
  • the ultrasound imaging system 12 includes a computer/microprocessor 16 and a transmitter/transducer/receiver unit 18 .
  • the transmitter/transducer/receiver unit 18 is coupled to the computer 16 through an electrical cable 20 .
  • the transmitter/transducer/receiver unit 18 emits ultrasound energy and receives the energy backscattered by the tissue. This information is provided to the computer 16 which processes the backscattered data to generate images displayed on a display screen 22 .
  • Such ultrasound imaging systems are well known within the art.
  • the tissue mapping workstation 14 is shown coupled to the computer 16 for communication of image data through an electrical cable 24 or other conventional method (wireless, e.g.).
  • the system 10 of the present invention can be used in conjunction with any conventional thermal treatment device such as a microwave, laser, RF, electrocautery, or ultrasound therapeutic device; but the use of the system 10 is not limited to such applications.
  • the system 10 can be used to monitor changes in collagen or tissue structure which is responding to a chemical, either injected into the tissue or administered topically.
  • the display of image data on the screen 22 of the ultrasound imaging device 12 is well known and can be represented in several forms.
  • ultrasound energy is emitted from the transmitted portion of transmitter/transducer unit 18 towards the tissue to be displayed. This energy is backscattered differently by different types of tissue. This backscattered ultrasound energy is comprised of spectral reflection and interference reflection portions.
  • the backscattered energy is received by the transducers of the transmitter/transducer 18 and converted to grayscale intensity data.
  • the grayscale intensity data is stored in shared memory, a video RAM or similarly disposed within the system 10 .
  • the grayscale intensity data is retrieved from an integral and conventional memory or video RAM and used to generate the driving signal for the display screen 22 .
  • the ultrasound signal is transmitted through the tissue and received by a transducer on the opposing side. This through transmission signal is then converted to grayscale intensity data and displayed on the display screen 22 .
  • Most commercially available ultrasound systems provide a video data port so the signal used to drive the display 22 may be recorded. This port is typically a RS-170 port and the data provided through such ports are well known.
  • ultrasound imaging systems that provide for digitally interfacing with an external form of the computer 16 ; and both the data are received from the ultrasound imaging system in digital form and processed and displayed on the system 10 .
  • Such ultrasound systems 10 often also provide for digital control of a number of the ultrasound system operating parameters, such as field of view, video settings, image probe selection, internal ultrasound imaging system processing settings, and similar functions.
  • a few ultrasound imaging systems 10 also provide access to the raw RF data prior to any internal processing within the ultrasound imaging system.
  • the mapping workstation 14 either receives the video data signal from the video data port and digitizes the signal to generate pixel data or interfaces to an available digital interface port on the ultrasound imaging system 10 . Since most conventional ultrasound systems have a RS-170 port or other video output connector, such an implementation makes the workstation 14 compatible with most available ultrasound systems.
  • the workstation 14 can couple to the system 10 through a parallel, serial, USB, or IEEE1394 data port to communicate pixel or grayscale data with the system 10 , as well as control the ultrasound imaging.
  • An example is the Terason system with an IEEE-1394 interface. This structure has the advantage of providing grayscale data from the system 10 prior to the data undergoing signal processing in the system 10 .
  • the system 10 may perform signal processing which reduces sensitivity to the information available in the interference reflection component of the backscattered energy. This reduction in sensitivity may affect the accuracy of the detection of collagen or other tissue structural changes by the workstation 14 .
  • the information in either of the reflection or transmission components may contribute to more accurate tissue mapping.
  • pixel data generated by the system 10 may contain textual information that may complicate its processing for the tissue change mapping. The transfer of either analog or digital image data or their equivalents for tissue change or collagen mapping is within the scope of the principles of the present invention.
  • Grayscale data is a data word which defines an intensity level for a pixel of a display.
  • the lowest value of the grayscale range corresponds to a low intensity or black shade pixel while the high value of the range corresponds to the brightest intensity or white shade pixel.
  • the intervening values are a shade of gray, hence the term grayscale data.
  • Grayscale data may be displayed on a screen capable of displaying color data.
  • the display 22 of ultrasound system 10 and the display 26 of the workstation 14 are displays capable of displaying both grayscale and color images.
  • the mapping workstation 14 for monitoring tissue structural changes and/or collagen changes includes a personal computer or the like with an Intel Pentium Duo or AMD processor or equivalent or better processor, 533 MHz or faster FSB, 512 MB or more of RAM and a 40 GB or larger hard drive.
  • Digital signal processing can be performed on workstation 14 or by a separate digital signal processing card coupled to the system 10 through its system bus which processes the image data from system 10 to perform the mapping of changes to tissue structure or collagen.
  • the image data to be processed may be displayed on the display 26 of the system 10 .
  • the system 10 can include standard input/output (I/O) devices such as a mouse or trackball for moving a cursor about the image displayed or the display 26 and for identifying coordinates on the image.
  • I/O input/output
  • FIG. 7 is a block diagram of a digital signal processing approach within the workstation 14 used to implement the tissue mapping of the present invention.
  • the system 10 includes a frame grabber 30 , a data buffer 32 , a computer 51 including data signal processor 34 for executing appropriate conventional computer software, a learning set memory 36 , a working memory 38 , and an overlay buffer 40 .
  • the frame grabber 30 retrieves video data from a video data port one frame at a time and converts the analog video data to pixel data which is stored in the data buffer 32 for processing.
  • the frame grabber 30 can receive data from a digital interface 50 which communicates with the system 10 through a digital data port to obtain grayscale data generated by the system 10 .
  • the grayscale data is used by the system 10 to produce the video signal which drives display 22 and may or may not undergo digital signal processing prior to its transfer to the workstation 14 .
  • frame data is provided to the display 22 at a rate of approximately 9 to 30 frames a second.
  • To attenuate noise in the image data several frames of data may be retrieved and averaged for each pixel. This averaged pixel data may then be stored in the data buffer 32 for processing by the processor 34 within the workstation computer 51 .
  • a treating physician Prior to initiation of thermal treatment, a treating physician preferably selects a region of interest within the image displayed on the display 26 , although the entire display 26 may be identified as the region of interest.
  • the region may be identified by using a pointing device 52 to outline the region of interest.
  • the system 10 evaluates the image data within the region of interest during the monitoring period for mapping of tissue changes.
  • the region of interest is defined by clicking on a first and second location on the display 26 to define an upper left and lower right corner of a rectangle or square, although other region shapes may be used.
  • the data signal processor 34 segregates the region of interest into predefined groups of pixels for temperature mapping.
  • a group of pixels may be defined as a single pixel.
  • the groups of pixels contain multiple pixels. Multiple pixel groups are less sensitive to ultrasound probe position changes.
  • the groups of pixels segregate the region of interest into an integer number of pixel groups.
  • the data signal processor 34 and/or the computer 51 preferably generates a neural element for each group of pixels and builds a learning set of initial baseline values and transfer functions for each neural element.
  • the baseline values are stored in the learning set memory 36 .
  • Baseline values for a group of pixels are discussed in more detail below.
  • the baseline values provide information to the data signal processor 34 about the homogeneity of a group of pixels and its surrounding neighborhood. This baseline information may be used to detect changes caused by thermal treatment which are correlated to temperature values and changes.
  • One set of baseline values for a group of pixels may include a differential measurement of the grayscale values within and outside a group of pixels. For example, an average grayscale value for a group of pixels may be computed and differential grayscale value for each pixel bordering the pixel group may be calculated and accumulated. Such a measurement provides an indication of the smoothness of the image in the area of the pixel groups. Likewise, gradients related to grayscale values at the boundaries of a pixel group may be used to define the change in an image area prior to initiation of thermal treatment. Other examples of baseline values include the detection, counting, and measurement of edges in the vicinity of a pixel group. Such edge detection and measurement may be performed using threshold, gradient, and Canny techniques, which are well known in the art.
  • These techniques provide a baseline measurement for detecting and counting of edges in the vicinity of a pixel group, the smoothness of the image in the area of the pixel group, the homogeneity of the area around the pixel group, the similarity of the areas on opposing sides of the pixel group, a measure of the amount of grayscale change in the area of the pixel group, respectively.
  • the data signal processor 34 uses the pixel values stored in the buffer 32 for new image frame data to generate new transfer function elements of the image currently being displayed. These are compared to the learning data in the learning set memory 36 to measure an image change for each group of pixels within the region of interest.
  • the measured image change typically expressed in grayscale units, is correlated to changes in collagen and/or tissue structure for the tissue area corresponding to the pixel group. This change is compared to thresholds for the structural/collagen physiological change ranges to ascertain the particular range in which the tissue resides. Identification of a specific structural/collagen/thermal dose range for a pixel group is then used to generate overlay image data which is stored in the overlay buffer 40 .
  • the overlay image data is converted to analog frame data and provided to the display 26 .
  • the overlay image data is displayed on portions of the image on the display 26 as a colorwash overlay to indicate degree of collagen/structural change for tissue areas corresponding to the groups of pixels.
  • the overlay image data may remain in digital form and an I/O controller implemented for communicating the pixel data to the system 10 .
  • the pixel data may be stored in the video RAM of the system 10 and used to colorwash overlay a portion of the display 22 to indicate degrees of tissue structural changes for each group of pixels within a region of interest.
  • FIG. 8A An example neural element (“NE”) is shown in FIG. 8A .
  • the neural element NE implements a transfer function which correlates the input data I to an output condition O. If an actual reading of the output condition is possible, the difference between the correlated output condition O and the actual condition is used to modify the transfer function so it better correlates the input data I to a corresponding output condition O. In this way, the element NE adjusts or learns during an ongoing process. In the present invention, this adjustment in the neural elements permits the system to compensate for changes in the tissue caused by local physiological changes unassociated with changes in structural elements. In this example, the neural element receives four inputs, I 1 , I 3 , I 5 and I 7 .
  • the transfer function in element NE of FIG. 8A correlates the difference between a weighted average of the current transfer function and a baseline condition for the transfer function contained in the learning set to a collagen change for the pixels corresponding to the element NE.
  • This tissue collagen change results in activation of one of the four outputs.
  • the outputs indicate whether there is no change (O 1 ), the change is reversible (O 2 ), the change is greater than 50% irreversible (O 3 ), or the change is caused by irreversible ablation (O 4 ).
  • these ranges correspond to the ones expressed in terms of percentage change in collagen structure.
  • the use of four descriptors for a neural element is merely exemplary and more or fewer descriptors may be used for each element covering different ranges.
  • FIG. 8B A preferred neural network for each group of pixels is shown in FIG. 8B .
  • That network shows three inputs for a set of three transfer functions, although fewer or more may be used for a group.
  • the vector path transfer functions are measurements between a group of pixels and its neighbor groups. These inputs are each provided to four neural elements which weight them differently to compute a weighted average. These weighted averages are each applied to the output elements, one of each corresponds to a temperature range.
  • the output elements weight the applied averages and one of the output elements is activated.
  • the output elements are also provided with a thermal dose correlation factor to verify whether the correct output element was activated. If there are any errors, an adjustment signal is generated and supplied to the neural elements providing the erroneous weighted average to adjust the weighting factors W at the neural element.
  • the states of the output elements for each group of pixels are used to generate the overlay colorwash image data.
  • the pixels which can be used to generate a transfer function are shown in FIG. 9A .
  • the center pixel P 0 is the pixel for which a change is to be determined.
  • the immediately adjacent pixels P 1 -P 8 are used to define input data I 1 -I 8 .
  • These input values are defined by the difference between P 0 and one of the adjacent pixels.
  • I 1 is defined by the difference in the detected grayscale between P 0 and P 1
  • I 2 the difference between P 0 and P 2 , and so on.
  • adjacent groups may also be utilized in the edge detection and measurements.
  • pixels P 9 -P 24 are adjacent to the region of interest.
  • a weighted average of all or some of these weighted inputs may be used to define an input function for P 0 .
  • this may be defined for a group of pixels.
  • the group of pixels denoted by GR 0 are surrounded by pixel groups GR 1 -GR 8 .
  • the transfer function may be defined by the average of the difference between each of the pixels in the top row of GR 0 and the adjacent pixels in GR 1 .
  • Other transfer functions may be similarly defined for other adjacent regions.
  • the input function for a group of pixels may be derived from four or more vectors about a pixel group.
  • the vectors, V 1 -V 8 for a single pixel, are shown in FIG. 9A .
  • the vectors may correspond to the eight surrounding groups about the center region GR 0 or they may be defined as the single row, column and diagonal vectors extending from the center pixel of GR 0 .
  • the important aspect of selecting a set of vectors is to define neighboring pixels in groups which provide information about the area about a pixel group so changes may be detected as they approach a pixel group or as they emanate from a pixel group.
  • the method of the present invention may be expressed in a flow chart as shown in FIG. 10 .
  • the treating physician preferably begins by selecting the region of interest (Step 1 ).
  • the gain of the ultrasound system is adjusted until the number of pixels at the lower intensity levels dominate the region of interest. In that way, the displayed image is predominated by intensity levels which permit the pixels to change to the higher end of the grayscale which in turn makes a greater dynamic range of tissue structural change mapping possible.
  • the region of interest is then segregated into groups of pixels (Step 2 ). After groups of pixels are defined, a set of vectors is selected for each pixel group and a transfer function which best maps the selected set of vectors to the detected normal homeostasis is selected (Step 3 ).
  • the vectors may be differential edge detection or gradients.
  • Baseline values are determined and placed in the learning memory associated with the neural element (Step 4 ).
  • the mapping workstation 14 is then ready and tissue treatment may begin. This indication may be generated by displaying a message on the display 26 , or by some other indicator.
  • image data from the system 10 is captured and stored in the buffer 32 .
  • the vectors for each neural element are updated and input to the neural elements (Step 5 ).
  • the neural elements compute a current transfer value and measure the differential between the current element value and the baseline or initial value (Step 6 ).
  • the current value is stored in the memory.
  • the thermal dose for the tissue in which a temperature probe is implanted is determined and the differential change in thermal dose is measured. This change is correlated to the change in tissue/collagen structure and to the change in grayscale value between the initial set of transfer function descriptors for the corresponding neural element and the current value (Step 7 ). If this grayscale unit to thermal dose correlation factor is approximately the same as the one being used by the neural elements, the process continues. Otherwise, the correlation or weighting factors for the transfer functions of the neural elements are modified.
  • each neural element computes a structural change corresponding to the grayscale differential between the initial value and the current value (Step 8 ).
  • the differential is added to the measured initial value to determine the degree of change for the tissue. If this change is greater than one of the threshold doses for the thermal treatment ranges (Step 9 ), the information corresponding to the detected structural change is generated, stored in the overlay image buffer 40 , and displayed as an overlay onto the image on the display 26 .
  • the overlay data is colorized or colorwashed to display a change indicative of the amount of thermally-induced structural change for a feature corresponding to a group of pixels.
  • the system 10 described can also use one or more temperature probes to confirm the grayscale-to-collagen change correlation factor.
  • no temperature probes are used.
  • the physician directs the treating device at a tissue region being imaged and delivers a short burst of treating radiation.
  • the power of this radiation and its duration is provided to the mapping system which uses this information with a tissue energy absorption factor to calculate a power deposition for the tissue.
  • the power deposition corresponds to a known thermal dose for the type of tissue identified for the feature.
  • This change from baseline is correlated to the grayscale differential measured for the same region during the test pulse. This is done two or three times to confirm or calculate an average correlation factor for the region.
  • a corresponding correlation factor for other tissue types may be extrapolated from that data using known methods.
  • Yet another alternative embodiment to determine the correlation factor is to match multiple levels of histologically determined tissue changes with the changes in backscatter affecting the transfer function of the neural elements. Once the correlation factor is calibrated in this manner, use of the system proceeds as described above except the steps related to using the temperature measured by the probe to adjust the system are not performed. Yet another alternative embodiment would utilize unprocessed front-end data from the ultrasound system 10 by monitoring either the reflected signal or the transmitted signal through the target tissue. Measured changes in the reflected and/or transmitted signal would then be correlated with multiple levels of histologically determined tissue changes and/or with different levels of delivered thermal dose.
  • the expert part of the system 10 of the present invention can verify that the grayscale differential is therapeutically induced and not caused by signal noise, thus applying pattern recognition.
  • the expert system can use edge detection techniques on the histogram data stored for a neural element to determine whether the gradient established by prior transfer function values confirms the shift in histological/structural/dose range.
  • differences between a current transfer function value for one neural element and the current value for its neighboring neural elements can also be evaluated using edge detection or imaging enhancement techniques to confirm whether the area in the vicinity of the neural element is approaching or at an greater level of structural change.
  • the histogram data can be processed using filtering or other noise alternating techniques to determine whether the current descriptor set accurately defines a tissue or image change.
  • the histogram image data for each neural element and the descriptor set values for neighboring neural elements can be used to confirm the collagen/structural change for a region.
  • the treating clinician prepares a patient for thermal treatment of a targeted tissue area.
  • the mapping system 14 signals the clinician that treatment may begin.
  • the treating clinician then inserts or applies the treating probe/device to a patient by a selected method and directs the emitted energy towards the target area.
  • the mapping system 14 periodically captures pixel data.
  • the captured data is used to update the transfer function values which are compared to the baseline values to generate a grayscale differential.
  • the update rate depends upon the digital signal processor used and the speed and amount of memory.
  • the Integral Technologies video capture card updates video data at a rate of 15-30 frames/second.
  • the grayscale differential is correlated to a tissue collagen/structural change by using a grayscale-to-temperature correlation factor.
  • the corresponding change determined for the tissue corresponding to the neural element is compared to the measurement change threshold for the treatment ranges.
  • unprocessed front-end data from the ultrasound system 10 is monitored from either the reflected signal or the transmitted signal through the target tissue. Measured changes in the reflected and/or transmitted signal would then be correlated with multiple levels of histologically determined tissue changes and/or with different levels of delivered thermal dose and this result mapped onto the display over the number of pixels interrogated.
  • the mapping system 14 can confirm the change degree zone shift by using edge detection techniques on the histogram data for the neural element or by using edge detection techniques on the vector set values for the neural elements adjacent to the neural element under investigation.
  • the mapping system 14 reads one or more temperature probes implanted in the tissue to confirm the grayscale-to-thermal dose correlation factor used by the neural elements.
  • the mapping system 14 stores color overlay data in the overlay buffer 40 which is transmitted to the video RAM 22 to modify the display 26 . As the treating physician observes the addition of color to the displayed image, the probe may be relocated or the energy being supplied to the target area adjusted.
  • the treatment may cease and the treating probe removed. If the treating physician desires to continue monitoring of the area, the transmitter/transducer unit 18 is used to continue the generation of image data which is compared to the baseline by the expert system to indicate degree of collagen/structural change in the tissue. In another embodiment of the invention, the system may establish a predetermined limit for tissue modifications and automatically cease treatment when the stipulated limit is achieved.
  • the change in the backscattered data may be detected in the signals received by the transmitter/transducer unit 18 .
  • the signals generated by the transducers in the unit 18 may be processed by a system built in accordance with the principles of the present invention to determine changes in collagen or tissue structure. Such a system is within the scope of the present invention.
  • the system and method of the present invention may also be used with image data generated from other electromagnetic imaging/detection modalities, such as microwave reflection/transmission detection systems, radiographic imaging methods, and other methods for tomographic transmission measurements of ultrasound, light, microwaves, radiographic, or other imaging energy sources.
  • An example of an electromagnetic system would be an array of small microwave apertures placed adjacent to the target tissues on one or both sides of the body region containing the target tissues.
  • the antenna device apertures can be simultaneously or independently excited in either a continuous mode or a pulsed mode. Either or both the reflected and transmitted signals may be monitored and processed in a manner described in this invention description.
  • tomographic radiographic data taken either continuously or at intervals during a treatment may be processed with this invention to yield tissue structural change information during therapy.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Health & Medical Sciences (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

A method and system for two and three dimensional mapping of tissue density and/or structural changes from image data and/or spatial reflected or transmitted signals and correlating the maps to collagen density.

Description

    CROSS-REFERENCE TO RELATED PATENT APPLICATIONS
  • This application is a continuation of U.S. application Ser. No. 11/656,315, filed Jan. 19, 2007, which claims the benefit of U.S. Provisional Application No. 60/760,220, filed Jan. 19, 2006, both of which are incorporated herein by reference in their entirety.
  • FIELD OF THE INVENTION
  • This invention relates to tissue monitoring and mapping equipment and methods, and more particularly, to methods and systems for monitoring and mapping changes in the collagen content and/or structures in tissue.
  • BACKGROUND OF INVENTION
  • Collagen is the major insoluble fibrous protein in the extracellular matrix and in connective tissue. There are at least nineteen types of collagen, but 80% to 90% of the collagen in the body is composed of the conventional well-known Type I, Type II, and Type III forms. The collagen molecule is a triple helix, each helical coil a biological polypeptide polymer constructed from glycine (C2H5NO2), alanine (C3H7NO2), proline (C5H9NO2), and hydroproline (C5H9NO3). As the three helices wrap around each other, hydrogen bonds form between each helix to maintain the structure. Collagen molecules pack together to form long, thin fibrils. The most prominent collagen types are:
  • Type I. The primary component of tendons, ligaments, and bones.
  • Type II. The primary protein constituent (more than 50%) in cartilage.
  • Type III. Strengthens the walls of hollow structures like arteries, the intestine, and the uterus.
  • Type IV. Forms the basal lamina (also called the basement membrane) of epithelia.
  • As a support structure, collagen fibrils are found in many environments within the body. The strength and relative elasticity of the collagen structure allow tendon and ligament structures to function in their role which requires great strength to manipulate skeletal structures. Up to 90% of the dry weight of tendons is collagen (at least 30% of the total weight), up to 50% of the dry weight of articular cartilage and synovial tissue is collagen (5% to 30% of total weight). Collagen comprises 75% of the dry weight of skin.
  • The helical structure and the nature of the hydrogen bond cross-linking of the collagen molecule cause it to react when heated. The cross-linking bonds break and the helix, which is a type of spring structure, collapses somewhat. The result is a shrinkage, not unlike the process which occurs when a woolen garment is washed in hot water and heated in a dryer. The amount of shrinkage is a function of time and temperature [see FIG. 1] and is well known.
  • Several therapeutic remedies have been developed which utilize the thermal response of collagen to affect shrinkage and/or for treating disease and, thereby, therapy. Among these are capsulorrhaphy, or the shrinkage of the tendon and capsular properties of the joints, shrinkage of the endopelvic fascia to address urinary incontinence, shrinkage of the bladder neck to affect treatment for urinary incontinence, shrinkage of sub-epidermal collagen for producing tightening of tissue to affect cosmetic outcomes, thermal treatment of benign and malignant tumors, among others.
  • A limitation of procedures for collagen shrinkage, however, is that the cell necrosis accompanying those procedures [see FIG. 1] weakens the structural integrity of the fibrils. It is known that shrinking capsular tissue more than 20% will weaken a structure so much that it will distort more under normal forces than it would if there had been no shrinkage at all [see FIG. 2]. Treatment of tumors usually involves delivering a thermal dose that guarantees cell necrosis, thus significantly changing collagen structure.
  • Thermally-induced shrinkage procedures have been received with moderate success because they have often produced variable results. Collagen shrinkage is a function of both time and temperature, but relatively small variations in either time or temperature can have dramatic results on the level of shrinkage [see FIG. 3]. Structures experiencing overshrinkage are likely to have limited, or adverse results. Structures with insufficient shrinkage are likely to have limited results.
  • Some therapy systems measure tissue temperature during treatment using invasive probes and predict associated tissue modification, while others estimate the temperature based on power-time-temperature parametric curves established from imperical measurements. But, because thermal dose, and thus collagen shrinkage, is an integral function of both time and temperature in combination with spatial distribution, and since slight variations in temperature during a specific time period can cause dramatic changes in collagen shrinkage [see FIG. 3], temperature measurement alone is not truly sufficient. In response to the need to effectively determine dosage of the thermal treatment, a noninvasive measure of the thermal dose or its affects, including collagen changes, is badly needed.
  • Since optimizing collagen modification, including shrinkage, is the goal of specific procedures, what is needed is a system and method for directly measuring the shrinkage of the collagen structure, rather than temperature. A system and method which could monitor tissue properties concurrent with treatment, signaling the operator to halt thermal application when collagen shrinkage approaches 20% could produce greatly improved results [see FIG. 3], reducing the inconsistency associated with the thermal shrinkage procedures.
  • The limitations of the previous art for monitoring tissue treatment resulting from thermal injury are inadequate to provide information directly related to collagen changes sufficient to provide optimal control of extent of collagen structural modification, including shrinkage.
  • It has been demonstrated that changes in collagen content of tissue affects certain acoustic properties, namely the speed of sound in that tissue structure and the absorption of the acoustic energy as it passes through that tissue [see FIG. 4A]. It has been shown in the art that there is a linear function of both acoustic velocity and acoustic attenuation (see FIG. 4B) with variations in collagen content. Acoustic velocity changes 5 m/sec/(% change in collagen concentration) and acoustic attenuation changes 0.666 db/m/(% change in collagen concentration).
  • As an example, a 20% shrinkage in a collagen structure having an initial collagen content of 30% should cause a 25% increase in the % collagen by weight of the tissue structure. Since many collagen structures are 5% to 30% collagen (by weight), a 25% increase in density due to therapeutic shrinkage would cause a change in collagen content of 1.25% to 7.5%. These changes would correspond to changes in acoustic velocity of 6.25 m/sec to 37.5 m/sec, for initial collagen contents of 5% and 30%, respectively (see FIG. 5A). These changes would correspond to changes in acoustic attenuation of 0.833 db/m to 5 db/m, respectively [see FIG. 5B].
  • In addition to changes in acoustic velocity and attenuation, the structural patterns of the tissue change permanently as a function of thermally-induced necrosis and density changes. These patterns may be characterized in an analysis of the structures in the 2D or 3D acoustic image or in 2D spatial maps of acoustic signals that are reflected from or transmitted through the tissue in those locations. Applying pattern recognition methods and/or expert system techniques to the backscatter image or signal mapping data can yield useful information in addition to attenuation and velocity measurements alone.
  • Ultrasound (acoustic) imaging has been used in medical imaging for years to differentiate tissue structures. These imaging techniques have examined acoustic properties of ultrasound waves as they travel through and reflect off of tissue to distinguish tissue types and the boundaries between those types. These devices map tissue in two dimensions (along a plane in line with an imaging transducer) or three dimensions (by using multi-element imaging transducers or single-line imaging transducers whose focus or location changes during the process. Two of the acoustic properties used in these imaging techniques are acoustic velocity and acoustic attenuation. Further, structural change analyses using pattern recognition methods can provide an ability to track changes in tissue structure/collagen structure from baseline using backscattered image information.
  • The ability to map those changes in tissue and assign changes in collagen density to the mapped functions would allow users the ability to monitor, in real time, the therapy effect they are seeking to accomplish.
  • SUMMARY OF THE INVENTION
  • The subject invention therefore includes a system and method for mapping acoustic changes from an ultrasound image of tissue induced by a structural change or thermal injury and mapping that information in a pixilated form. That is, each pixel, or square, in a two dimensional image or each voxel, or cube, in a three dimensional image, will be assigned acoustic characteristics based on the backscattered signal from that region and/or based on the transmitted signal through a specific tissue region. In particular, such system and method shall assign acoustic velocity values and acoustic attenuation values to each of the pixels.
  • As the treatment proceeds, the system and method will assign change values, and assign changes in collagen content concentration to those change values, to the acoustic properties for each pixel. The changes preferably are color-coded to make interpretation of the changes easier for users who are not familiar with interpreting ultrasound images.
  • An object of the invention is to provide an improved system and method to assign patterns or classes to groups of pixels or voxels for specific collagen changes based upon the backscatter patterns in the ultrasound image. Specific types of changes in patterns are correlated with known classes of patterns for groups of pixels.
  • Another object of this invention is to provide an improved system and method to correlate changes in the acoustic properties directly to changes in the collagen content in tissue and use those changes in acoustic properties to provide feedback for the application of thermal energy during a collagen shrinkage procedure.
  • Another object of this invention is to provide an improved system and method to determine acoustic property changes characteristic of the spatial representations of an acoustic signal and using those assignments to create a two dimensional or three dimensional map of the changes in collagen concentration of a tissue structure.
  • Another object of this invention is to provide an improved system and method to use color-coding when displaying the two dimensional or three dimensional maps of changes in collagen density.
  • Another object of this invention is to provide an improved system and method to use color-coding when displaying the two dimensional or three dimensional maps of changes in structural patterns in groups of pixels/voxels which correspond to changes in collagen structure and/or content.
  • Another object of this invention is to provide an improved system and method to measure the structural patterns of the tissue that are affected as a function of thermally-induced necrosis and density changes. These patterns may be characterized in an analysis of the structures in the acoustic image or in 2D spatial maps of acoustic signals that are reflected from or transmitted through the tissue in those locations. Applying pattern recognition methods to the image or signal mapping data can yield useful information in addition to attenuation and velocity measurements alone.
  • Another object of this invention is to provide an improved system and method to cross-correlate each of the types of measured changes in pairs or in total with each other to provide a sensitive system and method for determining changes in tissue collagen, including shrinkage and tissue necrosis.
  • Another object of this invention is to provide an improved system and method to use the measured tissue changes to correlate to tissue damage with changes in tissue structure and/or acoustic property changes as a result of treatment.
  • Since essentially all human tissue contains collagen in the interstitial tissue and since this invention describes measurements of changes in collagen concentration and/or collagen structure, and since changes in collagen concentration and structure occur in most thermal treatments, particularly those treatments intended to shrink collagen structures, this invention is applicable for monitoring collagen shrinkage procedures in all tissue structures of all types, both external and internal.
  • Further advantages and features of the invention will be apparent from the following specifications, claims and drawings, illustrating the preferred embodiments of the invention.
  • BRIEF SUMMARY OF THE DRAWINGS
  • FIG. 1 shows thermal effects on tissue;
  • FIG. 2 shows strength of collagen containing tissue post thermal remodeling;
  • FIG. 3 shows time/temperature influences on collagen shrinkage;
  • FIG. 4A shows relative changes in acoustic velocity and FIG. 4B acoustic attenuation properties with respect to changes in the collagen content of tissue;
  • FIG. 5A shows changes in acoustic velocity as a function of initial collagen content and percentage of collagen shrinkage; and FIG. 5B shows changes in acoustic attenuation as a function of initial collagen content and percentage of collagen shrinkage;
  • FIG. 6 shows a system for monitoring and mapping of tissue collagen shrinkage using ultrasound backscatter information and/or measured acoustic data reflected from or transmitted through tissue structures;
  • FIG. 7 shows a block diagram of one form of the tissue mapping workstation;
  • FIG. 8A shows a diagram of a neural element with four inputs for determining changes in backscatter associated with a structural change in a small tissue region to determine the magnitude of change for a pixel or group of pixels; and FIG. 8B shows a preferred neural network with three inputs for each group of pixels;
  • FIG. 9A shows a diagram of the surrounding pixels used to define a set of vectors used to determine the transfer function for a single pixel; FIG. 9B shows a diagram of groups of pixels used to define the vectors for a transfer function for a region of interest group made up of multiple pixels; and
  • FIG. 10 shows a flow diagram of a method for monitoring and displaying on a workstation the changes in collagen/tissue structure during a therapy procedure.
  • DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
  • The subject invention includes a system and method for mapping acoustic changes from an ultrasound image of tissue induced by a structural change or controlled thermal injury and mapping that information in a pixilated form. That is, each pixel, or square, in a two dimensional image or each voxel, or cube, in a three dimensional image, will be assigned acoustic characteristics based on the backscattered signal from that region and/or based on the transmitted signal through a specific tissue region. In particular, such a system and method assigns acoustic velocity values and acoustic attenuation values to each of the pixels.
  • As the treatment proceeds, the system and method will assign change values, and assign changes in collagen content concentration to those change values, to the acoustic properties for each pixel. The changes are most preferably color-coded to make interpretation of the changes easier for users who are not familiar with interpreting ultrasound images.
  • An example of a system 10 which can be used to implement the principles of the present invention is shown in FIG. 6. The system 10 includes an ultrasound imaging device 12 to which a tissue change mapping workstation or system 14 is connected. The ultrasound imaging system 12 includes a computer/microprocessor 16 and a transmitter/transducer/receiver unit 18. The transmitter/transducer/receiver unit 18 is coupled to the computer 16 through an electrical cable 20. The transmitter/transducer/receiver unit 18 emits ultrasound energy and receives the energy backscattered by the tissue. This information is provided to the computer 16 which processes the backscattered data to generate images displayed on a display screen 22. Such ultrasound imaging systems are well known within the art. The tissue mapping workstation 14 is shown coupled to the computer 16 for communication of image data through an electrical cable 24 or other conventional method (wireless, e.g.).
  • The system 10 of the present invention can be used in conjunction with any conventional thermal treatment device such as a microwave, laser, RF, electrocautery, or ultrasound therapeutic device; but the use of the system 10 is not limited to such applications. For example, the system 10 can be used to monitor changes in collagen or tissue structure which is responding to a chemical, either injected into the tissue or administered topically.
  • The display of image data on the screen 22 of the ultrasound imaging device 12 is well known and can be represented in several forms. In operation of the system 10, ultrasound energy is emitted from the transmitted portion of transmitter/transducer unit 18 towards the tissue to be displayed. This energy is backscattered differently by different types of tissue. This backscattered ultrasound energy is comprised of spectral reflection and interference reflection portions. The backscattered energy is received by the transducers of the transmitter/transducer 18 and converted to grayscale intensity data. The grayscale intensity data is stored in shared memory, a video RAM or similarly disposed within the system 10. The grayscale intensity data is retrieved from an integral and conventional memory or video RAM and used to generate the driving signal for the display screen 22. In another embodiment of the invention, the ultrasound signal is transmitted through the tissue and received by a transducer on the opposing side. This through transmission signal is then converted to grayscale intensity data and displayed on the display screen 22. Most commercially available ultrasound systems provide a video data port so the signal used to drive the display 22 may be recorded. This port is typically a RS-170 port and the data provided through such ports are well known. Alternatively, there are ultrasound imaging systems that provide for digitally interfacing with an external form of the computer 16; and both the data are received from the ultrasound imaging system in digital form and processed and displayed on the system 10. Such ultrasound systems 10 often also provide for digital control of a number of the ultrasound system operating parameters, such as field of view, video settings, image probe selection, internal ultrasound imaging system processing settings, and similar functions. A few ultrasound imaging systems 10 also provide access to the raw RF data prior to any internal processing within the ultrasound imaging system.
  • Preferably, the mapping workstation 14 either receives the video data signal from the video data port and digitizes the signal to generate pixel data or interfaces to an available digital interface port on the ultrasound imaging system 10. Since most conventional ultrasound systems have a RS-170 port or other video output connector, such an implementation makes the workstation 14 compatible with most available ultrasound systems. Alternatively, the workstation 14 can couple to the system 10 through a parallel, serial, USB, or IEEE1394 data port to communicate pixel or grayscale data with the system 10, as well as control the ultrasound imaging. An example is the Terason system with an IEEE-1394 interface. This structure has the advantage of providing grayscale data from the system 10 prior to the data undergoing signal processing in the system 10. The system 10 may perform signal processing which reduces sensitivity to the information available in the interference reflection component of the backscattered energy. This reduction in sensitivity may affect the accuracy of the detection of collagen or other tissue structural changes by the workstation 14. By obtaining pixel data prior to the ultrasound imaging system 10 processing it, the information in either of the reflection or transmission components may contribute to more accurate tissue mapping. Additionally, pixel data generated by the system 10 may contain textual information that may complicate its processing for the tissue change mapping. The transfer of either analog or digital image data or their equivalents for tissue change or collagen mapping is within the scope of the principles of the present invention.
  • Grayscale data is a data word which defines an intensity level for a pixel of a display. The lowest value of the grayscale range corresponds to a low intensity or black shade pixel while the high value of the range corresponds to the brightest intensity or white shade pixel. The intervening values are a shade of gray, hence the term grayscale data. Grayscale data may be displayed on a screen capable of displaying color data. The display 22 of ultrasound system 10 and the display 26 of the workstation 14 are displays capable of displaying both grayscale and color images.
  • Preferably, the mapping workstation 14 for monitoring tissue structural changes and/or collagen changes includes a personal computer or the like with an Intel Pentium Duo or AMD processor or equivalent or better processor, 533 MHz or faster FSB, 512 MB or more of RAM and a 40 GB or larger hard drive. Digital signal processing can be performed on workstation 14 or by a separate digital signal processing card coupled to the system 10 through its system bus which processes the image data from system 10 to perform the mapping of changes to tissue structure or collagen. The image data to be processed may be displayed on the display 26 of the system 10. Additionally, the system 10 can include standard input/output (I/O) devices such as a mouse or trackball for moving a cursor about the image displayed or the display 26 and for identifying coordinates on the image.
  • FIG. 7 is a block diagram of a digital signal processing approach within the workstation 14 used to implement the tissue mapping of the present invention. The system 10 includes a frame grabber 30, a data buffer 32, a computer 51 including data signal processor 34 for executing appropriate conventional computer software, a learning set memory 36, a working memory 38, and an overlay buffer 40. The frame grabber 30 retrieves video data from a video data port one frame at a time and converts the analog video data to pixel data which is stored in the data buffer 32 for processing. Alternatively, the frame grabber 30 can receive data from a digital interface 50 which communicates with the system 10 through a digital data port to obtain grayscale data generated by the system 10. The grayscale data is used by the system 10 to produce the video signal which drives display 22 and may or may not undergo digital signal processing prior to its transfer to the workstation 14. Typically, frame data is provided to the display 22 at a rate of approximately 9 to 30 frames a second. To attenuate noise in the image data, several frames of data may be retrieved and averaged for each pixel. This averaged pixel data may then be stored in the data buffer 32 for processing by the processor 34 within the workstation computer 51.
  • Prior to initiation of thermal treatment, a treating physician preferably selects a region of interest within the image displayed on the display 26, although the entire display 26 may be identified as the region of interest. The region may be identified by using a pointing device 52 to outline the region of interest. The system 10 evaluates the image data within the region of interest during the monitoring period for mapping of tissue changes. Preferably, the region of interest is defined by clicking on a first and second location on the display 26 to define an upper left and lower right corner of a rectangle or square, although other region shapes may be used. Once the region of interest is defined, the data signal processor 34 segregates the region of interest into predefined groups of pixels for temperature mapping. A group of pixels may be defined as a single pixel. Preferably, the groups of pixels contain multiple pixels. Multiple pixel groups are less sensitive to ultrasound probe position changes. Preferably, the groups of pixels segregate the region of interest into an integer number of pixel groups.
  • Once the region of interest and groups of pixels are defined, the data signal processor 34 and/or the computer 51 preferably generates a neural element for each group of pixels and builds a learning set of initial baseline values and transfer functions for each neural element. The baseline values are stored in the learning set memory 36. Baseline values for a group of pixels are discussed in more detail below. The baseline values provide information to the data signal processor 34 about the homogeneity of a group of pixels and its surrounding neighborhood. This baseline information may be used to detect changes caused by thermal treatment which are correlated to temperature values and changes.
  • One set of baseline values for a group of pixels may include a differential measurement of the grayscale values within and outside a group of pixels. For example, an average grayscale value for a group of pixels may be computed and differential grayscale value for each pixel bordering the pixel group may be calculated and accumulated. Such a measurement provides an indication of the smoothness of the image in the area of the pixel groups. Likewise, gradients related to grayscale values at the boundaries of a pixel group may be used to define the change in an image area prior to initiation of thermal treatment. Other examples of baseline values include the detection, counting, and measurement of edges in the vicinity of a pixel group. Such edge detection and measurement may be performed using threshold, gradient, and Canny techniques, which are well known in the art. These techniques provide a baseline measurement for detecting and counting of edges in the vicinity of a pixel group, the smoothness of the image in the area of the pixel group, the homogeneity of the area around the pixel group, the similarity of the areas on opposing sides of the pixel group, a measure of the amount of grayscale change in the area of the pixel group, respectively. By using these techniques to establish baseline values and then compute changes in these values as tissue structural change occurs, differential values indicative of the amount of change caused by the treatment-induced structural changes may be discerned and quantized.
  • After thermal treatment begins, the data signal processor 34 uses the pixel values stored in the buffer 32 for new image frame data to generate new transfer function elements of the image currently being displayed. These are compared to the learning data in the learning set memory 36 to measure an image change for each group of pixels within the region of interest. The measured image change, typically expressed in grayscale units, is correlated to changes in collagen and/or tissue structure for the tissue area corresponding to the pixel group. This change is compared to thresholds for the structural/collagen physiological change ranges to ascertain the particular range in which the tissue resides. Identification of a specific structural/collagen/thermal dose range for a pixel group is then used to generate overlay image data which is stored in the overlay buffer 40. The overlay image data is converted to analog frame data and provided to the display 26. The overlay image data is displayed on portions of the image on the display 26 as a colorwash overlay to indicate degree of collagen/structural change for tissue areas corresponding to the groups of pixels. Alternatively, the overlay image data may remain in digital form and an I/O controller implemented for communicating the pixel data to the system 10. The pixel data may be stored in the video RAM of the system 10 and used to colorwash overlay a portion of the display 22 to indicate degrees of tissue structural changes for each group of pixels within a region of interest.
  • An example neural element (“NE”) is shown in FIG. 8A. The neural element NE implements a transfer function which correlates the input data I to an output condition O. If an actual reading of the output condition is possible, the difference between the correlated output condition O and the actual condition is used to modify the transfer function so it better correlates the input data I to a corresponding output condition O. In this way, the element NE adjusts or learns during an ongoing process. In the present invention, this adjustment in the neural elements permits the system to compensate for changes in the tissue caused by local physiological changes unassociated with changes in structural elements. In this example, the neural element receives four inputs, I1, I3, I5 and I7. These inputs describe the neural element and the neural element corresponds to a pixel or group of pixels identified by the process discussed above. The transfer function in element NE of FIG. 8A correlates the difference between a weighted average of the current transfer function and a baseline condition for the transfer function contained in the learning set to a collagen change for the pixels corresponding to the element NE. This tissue collagen change results in activation of one of the four outputs. The outputs indicate whether there is no change (O1), the change is reversible (O2), the change is greater than 50% irreversible (O3), or the change is caused by irreversible ablation (O4). Preferably, these ranges correspond to the ones expressed in terms of percentage change in collagen structure. The use of four descriptors for a neural element is merely exemplary and more or fewer descriptors may be used for each element covering different ranges.
  • A preferred neural network for each group of pixels is shown in FIG. 8B. That network shows three inputs for a set of three transfer functions, although fewer or more may be used for a group. The vector path transfer functions are measurements between a group of pixels and its neighbor groups. These inputs are each provided to four neural elements which weight them differently to compute a weighted average. These weighted averages are each applied to the output elements, one of each corresponds to a temperature range. The output elements weight the applied averages and one of the output elements is activated. Preferably, the output elements are also provided with a thermal dose correlation factor to verify whether the correct output element was activated. If there are any errors, an adjustment signal is generated and supplied to the neural elements providing the erroneous weighted average to adjust the weighting factors W at the neural element. The states of the output elements for each group of pixels are used to generate the overlay colorwash image data.
  • The pixels which can be used to generate a transfer function are shown in FIG. 9A. The center pixel P0 is the pixel for which a change is to be determined. The immediately adjacent pixels P1-P8 are used to define input data I1-I8. These input values are defined by the difference between P0 and one of the adjacent pixels. For example, I1 is defined by the difference in the detected grayscale between P0 and P1, I2 the difference between P0 and P2, and so on. For groups of pixels, adjacent groups may also be utilized in the edge detection and measurements. In FIG. 9B, pixels P9-P24 are adjacent to the region of interest. A weighted average of all or some of these weighted inputs may be used to define an input function for P0. In a similar manner, this may be defined for a group of pixels. As shown in FIG. 9B, the group of pixels denoted by GR0 are surrounded by pixel groups GR1-GR8. In such a case, the transfer function may be defined by the average of the difference between each of the pixels in the top row of GR0 and the adjacent pixels in GR1. Other transfer functions may be similarly defined for other adjacent regions.
  • The input function for a group of pixels may be derived from four or more vectors about a pixel group. The vectors, V1-V8 for a single pixel, are shown in FIG. 9A. In FIG. 9B, the vectors may correspond to the eight surrounding groups about the center region GR0 or they may be defined as the single row, column and diagonal vectors extending from the center pixel of GR0. The important aspect of selecting a set of vectors is to define neighboring pixels in groups which provide information about the area about a pixel group so changes may be detected as they approach a pixel group or as they emanate from a pixel group.
  • The method of the present invention may be expressed in a flow chart as shown in FIG. 10. The treating physician preferably begins by selecting the region of interest (Step 1). The gain of the ultrasound system is adjusted until the number of pixels at the lower intensity levels dominate the region of interest. In that way, the displayed image is predominated by intensity levels which permit the pixels to change to the higher end of the grayscale which in turn makes a greater dynamic range of tissue structural change mapping possible.
  • The region of interest is then segregated into groups of pixels (Step 2). After groups of pixels are defined, a set of vectors is selected for each pixel group and a transfer function which best maps the selected set of vectors to the detected normal homeostasis is selected (Step 3). The vectors may be differential edge detection or gradients. Baseline values are determined and placed in the learning memory associated with the neural element (Step 4). The mapping workstation 14 is then ready and tissue treatment may begin. This indication may be generated by displaying a message on the display 26, or by some other indicator.
  • During treatment, image data from the system 10 is captured and stored in the buffer 32. The vectors for each neural element are updated and input to the neural elements (Step 5). The neural elements compute a current transfer value and measure the differential between the current element value and the baseline or initial value (Step 6). The current value is stored in the memory. Preferably, the thermal dose for the tissue in which a temperature probe is implanted is determined and the differential change in thermal dose is measured. This change is correlated to the change in tissue/collagen structure and to the change in grayscale value between the initial set of transfer function descriptors for the corresponding neural element and the current value (Step 7). If this grayscale unit to thermal dose correlation factor is approximately the same as the one being used by the neural elements, the process continues. Otherwise, the correlation or weighting factors for the transfer functions of the neural elements are modified.
  • Using the thermal dose correlation factor, each neural element computes a structural change corresponding to the grayscale differential between the initial value and the current value (Step 8). The differential is added to the measured initial value to determine the degree of change for the tissue. If this change is greater than one of the threshold doses for the thermal treatment ranges (Step 9), the information corresponding to the detected structural change is generated, stored in the overlay image buffer 40, and displayed as an overlay onto the image on the display 26. The overlay data is colorized or colorwashed to display a change indicative of the amount of thermally-induced structural change for a feature corresponding to a group of pixels.
  • The system 10 described can also use one or more temperature probes to confirm the grayscale-to-collagen change correlation factor. In an alternative embodiment, no temperature probes are used. Instead, the physician directs the treating device at a tissue region being imaged and delivers a short burst of treating radiation. The power of this radiation and its duration is provided to the mapping system which uses this information with a tissue energy absorption factor to calculate a power deposition for the tissue. The power deposition corresponds to a known thermal dose for the type of tissue identified for the feature. This change from baseline is correlated to the grayscale differential measured for the same region during the test pulse. This is done two or three times to confirm or calculate an average correlation factor for the region. A corresponding correlation factor for other tissue types may be extrapolated from that data using known methods. Yet another alternative embodiment to determine the correlation factor is to match multiple levels of histologically determined tissue changes with the changes in backscatter affecting the transfer function of the neural elements. Once the correlation factor is calibrated in this manner, use of the system proceeds as described above except the steps related to using the temperature measured by the probe to adjust the system are not performed. Yet another alternative embodiment would utilize unprocessed front-end data from the ultrasound system 10 by monitoring either the reflected signal or the transmitted signal through the target tissue. Measured changes in the reflected and/or transmitted signal would then be correlated with multiple levels of histologically determined tissue changes and/or with different levels of delivered thermal dose.
  • Additionally, the expert part of the system 10 of the present invention can verify that the grayscale differential is therapeutically induced and not caused by signal noise, thus applying pattern recognition. To verify the differential, the expert system can use edge detection techniques on the histogram data stored for a neural element to determine whether the gradient established by prior transfer function values confirms the shift in histological/structural/dose range. Additionally, differences between a current transfer function value for one neural element and the current value for its neighboring neural elements can also be evaluated using edge detection or imaging enhancement techniques to confirm whether the area in the vicinity of the neural element is approaching or at an greater level of structural change. Additionally, or alternatively, the histogram data can be processed using filtering or other noise alternating techniques to determine whether the current descriptor set accurately defines a tissue or image change. Thus, the histogram image data for each neural element and the descriptor set values for neighboring neural elements can be used to confirm the collagen/structural change for a region.
  • In use, the treating clinician prepares a patient for thermal treatment of a targeted tissue area. After the mapping workstation or the mapping system 14 has established baseline values for the transfer function for each neural element, the mapping system 14 signals the clinician that treatment may begin. The treating clinician then inserts or applies the treating probe/device to a patient by a selected method and directs the emitted energy towards the target area. As the emitted energy is absorbed in the target area, the mapping system 14 periodically captures pixel data. The captured data is used to update the transfer function values which are compared to the baseline values to generate a grayscale differential. The update rate depends upon the digital signal processor used and the speed and amount of memory. In a preferred embodiment, the Integral Technologies video capture card updates video data at a rate of 15-30 frames/second. The grayscale differential is correlated to a tissue collagen/structural change by using a grayscale-to-temperature correlation factor. The corresponding change determined for the tissue corresponding to the neural element is compared to the measurement change threshold for the treatment ranges. In an alternative embodiment of the invention, unprocessed front-end data from the ultrasound system 10 is monitored from either the reflected signal or the transmitted signal through the target tissue. Measured changes in the reflected and/or transmitted signal would then be correlated with multiple levels of histologically determined tissue changes and/or with different levels of delivered thermal dose and this result mapped onto the display over the number of pixels interrogated. If the differential for a pixel value indicates that the structural state of the tissue has shifted to another zone, the mapping system 14 can confirm the change degree zone shift by using edge detection techniques on the histogram data for the neural element or by using edge detection techniques on the vector set values for the neural elements adjacent to the neural element under investigation. Preferably, the mapping system 14 reads one or more temperature probes implanted in the tissue to confirm the grayscale-to-thermal dose correlation factor used by the neural elements. When a range shift is detected for the tissue corresponding to a neural element, the mapping system 14 stores color overlay data in the overlay buffer 40 which is transmitted to the video RAM 22 to modify the display 26. As the treating physician observes the addition of color to the displayed image, the probe may be relocated or the energy being supplied to the target area adjusted. Once the treating physician is satisfied that the target area tissue has been appropriately treated, the treatment may cease and the treating probe removed. If the treating physician desires to continue monitoring of the area, the transmitter/transducer unit 18 is used to continue the generation of image data which is compared to the baseline by the expert system to indicate degree of collagen/structural change in the tissue. In another embodiment of the invention, the system may establish a predetermined limit for tissue modifications and automatically cease treatment when the stipulated limit is achieved.
  • While the present invention has been illustrated by a description of preferred and alternative embodiments and processes, and while the preferred and alternative embodiments and processes have been described in considerable detail, it is not the intention of the applicants to restrict or in any way limit the scope of the appended claims to such detail. Additional advantages and modifications will readily appear to those skilled in the art. For example, the change in the backscattered data may be detected in the signals received by the transmitter/transducer unit 18. As a result, the signals generated by the transducers in the unit 18 may be processed by a system built in accordance with the principles of the present invention to determine changes in collagen or tissue structure. Such a system is within the scope of the present invention. Likewise while the invention has been described with reference to images generated by ultrasound energy, the system and method of the present invention may also be used with image data generated from other electromagnetic imaging/detection modalities, such as microwave reflection/transmission detection systems, radiographic imaging methods, and other methods for tomographic transmission measurements of ultrasound, light, microwaves, radiographic, or other imaging energy sources. An example of an electromagnetic system would be an array of small microwave apertures placed adjacent to the target tissues on one or both sides of the body region containing the target tissues. The antenna device apertures can be simultaneously or independently excited in either a continuous mode or a pulsed mode. Either or both the reflected and transmitted signals may be monitored and processed in a manner described in this invention description. Similarly, tomographic radiographic data taken either continuously or at intervals during a treatment may be processed with this invention to yield tissue structural change information during therapy. These are examples only and other imaging methods may be similarly used and processed.

Claims (8)

1. A system for measuring collagen structural changes responsive to collagen treatment, comprising:
a source for producing ultrasound energy to probe collagen containing tissue which has undergone a change from treatment;
a detector for receiving an acoustic signal from the collagen containing tissue;
a memory unit;
a data analysis system for operating on the acoustic signal for assigning a value to a pixel of a plane in the collagen containing tissue, such value to be based on acoustic characteristics derived from acoustic backscattered signal from that two dimensional region of a plane in the collagen containing tissue;
the data analysis system using the changes in the acoustic value of at least one pixel to provide a qualitative metric of the change in collagen containing tissue properties within the collagen containing tissue as represented by the at least one pixel.
2. The system as defined in claim 1 wherein the data analysis system includes computer software executed by a computer to define a region of interest in said image of the collagen containing tissue and segregates said region of interest into groups of at least one pixel.
3. The system as defined in claim 1 wherein the data analysis system includes computer software executed by a computer to assign color values to at least one pixel corresponding to differing changes in properties in the collagen containing tissue represented by at least one pixel.
4. The system as defined in claim 1 wherein at least one pixel comprises a voxel.
5. The system as defined in claim 1 further including a collagen treatment device.
6. The system as defined in claim 5 wherein the collagen treatment device comprises at least one of a microwave unit, a laser unit, an RF unit, an electrocautery unit, a thermal unit, and an ultrasound unit.
7. The system as defined in claim 1 further including computer software executable by the data analysis system for analyzing the acoustic signal.
8. The system as defined in claim 4 wherein the data analysis system assigns color values to the voxel corresponding to differing changes in the collagen containing tissue properties in the tissue region represented by the voxel; and
generating an overlay data to display said color-coded voxel over a standard acoustic image.
US13/285,635 2006-01-19 2011-10-31 Collagen Density and Structural Change Measurement And Mapping in Tissue Abandoned US20120078098A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13/285,635 US20120078098A1 (en) 2006-01-19 2011-10-31 Collagen Density and Structural Change Measurement And Mapping in Tissue

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US76022006P 2006-01-19 2006-01-19
US11/656,315 US8047990B2 (en) 2006-01-19 2007-01-19 Collagen density and structural change measurement and mapping in tissue
US13/285,635 US20120078098A1 (en) 2006-01-19 2011-10-31 Collagen Density and Structural Change Measurement And Mapping in Tissue

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US11/656,315 Continuation US8047990B2 (en) 2006-01-19 2007-01-19 Collagen density and structural change measurement and mapping in tissue

Publications (1)

Publication Number Publication Date
US20120078098A1 true US20120078098A1 (en) 2012-03-29

Family

ID=38286410

Family Applications (2)

Application Number Title Priority Date Filing Date
US11/656,315 Active 2028-05-19 US8047990B2 (en) 2006-01-19 2007-01-19 Collagen density and structural change measurement and mapping in tissue
US13/285,635 Abandoned US20120078098A1 (en) 2006-01-19 2011-10-31 Collagen Density and Structural Change Measurement And Mapping in Tissue

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US11/656,315 Active 2028-05-19 US8047990B2 (en) 2006-01-19 2007-01-19 Collagen density and structural change measurement and mapping in tissue

Country Status (1)

Country Link
US (2) US8047990B2 (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011112248A2 (en) * 2010-03-08 2011-09-15 Alpha Orthopaedics, Inc. Methods and devices for real time monitoring of collagen and for altering collagen status
US10945656B2 (en) * 2013-04-05 2021-03-16 The University Of Massachusetts Device and method for imaging collagen structure in vivo
KR101563498B1 (en) * 2013-05-02 2015-10-27 삼성메디슨 주식회사 Ultrasound system and method for providing change information of target object
WO2015069824A2 (en) * 2013-11-06 2015-05-14 Lehigh University Diagnostic system and method for biological tissue analysis
CN104720835A (en) * 2013-12-20 2015-06-24 Ge医疗系统环球技术有限公司 Display device, image displaying method and computerized tomography apparatus
US10839509B2 (en) 2015-07-10 2020-11-17 3Scan Inc. Spatial multiplexing of histological stains
CN112932534B (en) * 2021-01-27 2022-02-22 深圳华声医疗技术股份有限公司 Ultrasonic image acquisition processing system and ultrasonic image acquisition processing method

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6067371A (en) * 1995-11-28 2000-05-23 Dornier Medical Systems, Inc. Method and system for non-invasive temperature mapping of tissue
US20030036706A1 (en) * 1997-10-14 2003-02-20 Dr. Michael H. Slayton Imaging, Therapy, and temperature monitoring ultrasonic system
US20050209588A1 (en) * 2003-09-04 2005-09-22 Crum, Kaminski & Larson, Llc HIFU resculpturing and remodeling of heart valves

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6500121B1 (en) * 1997-10-14 2002-12-31 Guided Therapy Systems, Inc. Imaging, therapy, and temperature monitoring ultrasonic system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6067371A (en) * 1995-11-28 2000-05-23 Dornier Medical Systems, Inc. Method and system for non-invasive temperature mapping of tissue
US20030036706A1 (en) * 1997-10-14 2003-02-20 Dr. Michael H. Slayton Imaging, Therapy, and temperature monitoring ultrasonic system
US20050209588A1 (en) * 2003-09-04 2005-09-22 Crum, Kaminski & Larson, Llc HIFU resculpturing and remodeling of heart valves

Also Published As

Publication number Publication date
US20070173720A1 (en) 2007-07-26
US8047990B2 (en) 2011-11-01

Similar Documents

Publication Publication Date Title
US20120078098A1 (en) Collagen Density and Structural Change Measurement And Mapping in Tissue
US6067371A (en) Method and system for non-invasive temperature mapping of tissue
CN107569256B (en) Ultrasonic method for measuring temperature change of biological tissue based on thermal expansion and gating algorithm
CN109199381B (en) Holographic microwave elastography system and imaging method thereof
CN107788950B (en) Blood flow imaging method and system based on self-adaptive threshold segmentation
EP2559375A1 (en) Method and device for monitoring temperature of treatment site by using ultrasound, and system for treatment and diagnosis using ultrasound
JPH10504893A (en) Microwave tomography spectroscopy system and method
KR20000023647A (en) Electromagnetic Imaging and Therapeutic(EMIT) Systems
US20020165455A1 (en) Methods and apparatus for using ultrasound with contrast agent
US20050129297A1 (en) Classification of breast lesion method and system
CN105246415A (en) Ultrasonic observation device, ultrasonic observation device operation method, and ultrasonic observation device operation program
WO2012120498A1 (en) Non-invasive thermal treatment monitoring
CN112204412A (en) K-space adaptive sampling during MR guided non-invasive therapy
Godinho et al. Experimental evaluation of an axillary microwave imaging system to aid breast cancer staging
CN110075430A (en) A kind of ultrasonic cavitation method of real-time and system based on comentropy
US9433397B2 (en) Method and device for determining the elastic modulus of a biological tissue
Li et al. Detection of microwave ablation coagulation areas using ultrasound Nakagami imaging based on Gaussian pyramid decomposition: A feasibility study
Rohrbach et al. The early phases of bone healing can be differentiated in a rat osteotomy model by focused transverse-transmission ultrasound
CN112674791B (en) Optimization method and system for muscle ultrasonic elastography
WO2004000125A1 (en) System and method for analysis of a tissue
Moghimi et al. Automatic evaluation of pressure sore status by combining information obtained from high-frequency ultrasound and digital photography
Hussain et al. Strain-initialized robust bone surface detection in 3-D ultrasound
CN1249164A (en) Ultrasonic attenuation imaging technique for human tissue
Kerut et al. Echocardiographic texture analysis using the wavelet transform: differentiation of early heart muscle disease
US11850022B2 (en) NIR image-guided targeting

Legal Events

Date Code Title Description
STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION