WO2013150419A1 - Contrôle qualité pendant une procédure d'imagerie médicale - Google Patents

Contrôle qualité pendant une procédure d'imagerie médicale Download PDF

Info

Publication number
WO2013150419A1
WO2013150419A1 PCT/IB2013/052455 IB2013052455W WO2013150419A1 WO 2013150419 A1 WO2013150419 A1 WO 2013150419A1 IB 2013052455 W IB2013052455 W IB 2013052455W WO 2013150419 A1 WO2013150419 A1 WO 2013150419A1
Authority
WO
WIPO (PCT)
Prior art keywords
images
medical imaging
tissue
sub
quality
Prior art date
Application number
PCT/IB2013/052455
Other languages
English (en)
Inventor
Marinus Bastiaan Van Leeuwen
Payal Keswarpu
Celine Firtion
Pallavi Vajinepalli
Vipin Gupta
Caifeng Shan
Jelte Peter Vink
Original Assignee
Koninklijke Philips N.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips N.V. filed Critical Koninklijke Philips N.V.
Publication of WO2013150419A1 publication Critical patent/WO2013150419A1/fr

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000094—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope extracting biological structures
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000095—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope for image enhancement
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10068—Endoscopic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20076—Probabilistic image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30028—Colon; Small intestine
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30092—Stomach; Gastric
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection

Definitions

  • the invention relates to a medical imaging system with a camera operative to capture images of an interior of a hollow organ or of a cavity of a body of a human being, a monitoring sub-system configured for use in the medical imaging system, an evaluation sub-system configured for use in the monitoring sub-system, and control software for configuring a computer or a data processor system.
  • Medical imaging procedures have been designed for examining the interior of a hollow organ or of a cavity of the body of a human being by means of specialized medical instruments, typically accommodating a camera. Examples of such medical imaging procedures are endoscopy and colposcopy.
  • endoscopy refers to a class of medical procedures configured for looking inside the body of a patient by means of an endoscope.
  • An endoscope is a medical instrument used to examine the interior of a hollow organ or cavity of the body and requires that the endoscope's camera be inserted directly into the organ or cavity.
  • cystoscopy a medical procedure to inspect the inside of the patient's bladder
  • colonscopy (a medical procedure for inspecting the inside of the patient's large bowel and distal part of the patient's small bowel); laparoscopy (a medical procedure for inspecting the inside of the patient's abdomen and pelvis); and histeroscopy (a medical procedure for inspecting the patient uterine cavity).
  • Colposcopy is a medical procedure for inspecting the inside of the cervix, vulva and vagina of the patient using a specialized medical instrument, referred to as a colposcope, without any part of the colposcope being inserted into the vagina.
  • the colposcope is positioned a short distance from the vagina and a speculum is used to separate the walls of the vagina slightly apart.
  • Colposcopy is considered below in further detail as a typical example of a medical imaging procedure designed for examining the interior of a hollow organ or of a cavity of the body of a human being by means of a specialized medical instrument.
  • cervical cancer is the second most common, and the fifth most deadly, cancer in women.
  • the expression "cervical cancer” is used to refer to a malignant tumor arising from cells originating in the cervix uteri.
  • Colposcopy is a medical imaging procedure that is widely used to create an illuminated, magnified view of the cervix and the tissues of the vagina and the vulva in order to examine for the presence of cervical cancer by identifying visible clues suggestive of abnormal tissue.
  • the medical imaging procedure is carried out with a colposcope, an instrument that functions as a lighted magnifier to magnify the view of the cervix, of the vagina, and of the vulva.
  • a typical colposcopy imaging procedure includes one or more of the steps discussed briefly below.
  • a cleaning step is performed to clean the tissue to be examined, e.g., by removing any (excess) mucus or other secretions by applying a saline cleaning fluid using a cotton swab or a douche.
  • An aceto-whitening step is preformed to identify abnormalities in the vascular pattern (pattern of the blood vessels) at the surface of the tissue under examination.
  • the aceto-whitening step causes a blanching of skin or mucous membranes after application of 3-5% acetic acid solution, thus accentuating the visual appearance of abnormal areas of the tissue. It takes about 2 minutes after the application for the visual effects of the application of the acetic acid to reach a maximum. The visual effects fade away in about 5 minutes after the application.
  • the acetic acid solution is thought to cause swelling of the epithelial tissue, columnar and any abnormal squamous epithelial areas in particular. It causes a reversible coagulation or precipitation of the nuclear proteins and cytokeratins.
  • acetic acid depends upon the amount of nuclear proteins and cytokeratins present in the epithelium.
  • acetic acid When acetic acid is applied to normal squamous epithelium, little coagulation occurs in the superficial cell layer, as this is sparsely nucleated. Although the deeper cells contain more nuclear protein, the acetic acid may not penetrate sufficiently and, hence, the resulting precipitation is not sufficient to obliterate the color of the underlying stroma. Areas of cervical intraepithelial neoplasia undergo maximal coagulation due to their higher content of nuclear protein and prevent light from passing through the epithelium.
  • the sub-epithelial vessel pattern is obliterated and less easy to see and the epithelium appears white.
  • This reaction is termed aceto-whitening, and produces a noticeable effect compared with the normal pinkish color of the surrounding normal squamous epithelium of the cervix, an effect that is commonly visible to the naked eye.
  • the colposcope is controlled to capture the images (still pictures and/or a video) of the tissue.
  • a green filter is used to observe the vascular pattern so as to be able to identify abnormalities, for example, punctuations, mosaic and palisade-like vessels.
  • Punctuation refers to dilated capillaries that terminate on the surface of the tissue, their ends creating the visual impression of a collection of dots.
  • a mosaic pattern refers to a pattern of interconnecting vessels that gives the impression of that of a cobblestone-surface or a honeycomb surface. Palisade-like vessels run almost perpendicularly to the surface.
  • Lugol's iodine is applied to the vagina and to the cervix in order to visualize abnormal areas of tissue.
  • the expression "Lugol's iodine” refers to a solution of elemental iodine and potassium iodide in water. Normal vaginal tissue stains brown due to its high glycogen content, whereas tissue suspicious for cancer does not stain, and thus appears pale or slightly mustard yellow (color of iodine) compared to the surrounding tissue. Biopsy of suspicious tissue can then be performed. This is called a Schiller's Test.
  • Neoplasia A Beginners ' Manual", John W. Sellors M.D., and R. Sankaranarayanan, M.D., IARC Press. 2003.
  • the operator of the colposcope and the medical expert are different persons.
  • the operator is, e.g., a trained nurse or a healthcare worker, and the medical expert is, e.g., a gynecologist.
  • the operator controls the colposcope and records the images so that one or more medical experts will be able to examine the images later on.
  • a clinician or medical expert is present during execution of the medical imaging protocol, a medical diagnosis is carried out, substantially in real-time, based on the clinician or medical expert examining the images as they are being captured.
  • images from a colposcope are digitized and communicated over a data network to a medical expert at a remote location for being examined, either in real-time or later on.
  • This form of telemedicine is referred to as "tele-colposcopy", “remote colposcopy” or "e-colposcopy”.
  • the medical expert In conventional colposcopy, the medical expert will be examining the images as recorded, or as received via a data network from a remote location.
  • digital colposcopy the images captured are stored in a computer-readable memory and are subjected to computerized image analysis techniques at a later time. That is, the capturing of the images through colposcopy, on the one hand, and the analysis of the captured images, on the other hand, are activities carried out in disjoint time slots, hours or even days or weeks apart. It is therefore important that the colposcopy images, subjected to medical diagnosis, be of adequate quality.
  • diagnosis by the medical expert is seriously hampered if the images have an incorrect level of detail, or if the relevant areas of the tissue of the cervix, the vagina and the vulva have not been captured.
  • the patient may have to return in order to be subjected again to a new colposcopy procedure, thus having again to spend time and tying up resources which could have been avoided if the images had been captured with adequate quality the first time.
  • automated or semi-automated image analysis applications are being used as tools, which serve to assist the medical expert in the diagnosis.
  • Low-quality images or images that do not cover the entire area of relevant tissue may then create serious problems as a result of false positives and false negatives in the diagnosis. This could be intercepted if the medical expert routinely double-checks the automated outcome, but this, in turn, runs counter to efficiency as the medical expert could have performed the analysis him/herself the first time.
  • the inventors propose to run a quality check in real-time by means of employing image processing equipment configured for monitoring the quality of the images (still picture(s) and/or video) during the capturing, i.e., during operational use of the camera of the medical imaging system, and for giving feedback to the operator of the medical imaging system, there and then, if it is determined that the quality is insufficient.
  • image processing equipment configured for monitoring the quality of the images (still picture(s) and/or video) during the capturing, i.e., during operational use of the camera of the medical imaging system, and for giving feedback to the operator of the medical imaging system, there and then, if it is determined that the quality is insufficient.
  • a medical imaging system that comprises a camera for capturing images of an interior of a hollow organ or cavity of a body of a human being under control of an operator executing a medical imaging protocol.
  • the medical imaging system also comprises a monitoring sub-system that is operative to monitor an execution of the medical imaging protocol.
  • the monitoring sub-system comprises an evaluation sub-system and a notification sub-system.
  • the evaluation sub-system is arranged for receiving the images and is operative to process the images for determining a quality of the images received.
  • the notification sub-system is operative to provide a human-perceptible notification, e.g., to the operator of the medical imaging system, under control of the quality as determined by the evaluation sub-system.
  • the quality of the images determines whether or not the images are suitable for being subjected to a medical diagnosis carried out by a medical expert or a computerized system.
  • the images as captured are evaluated by the evaluation sub-system.
  • the evaluation sub-system subjects the images to image processing operations for the purpose of determining the quality of the images.
  • the feature "quality of the images” is used throughout this text to refer to one or more attributes in the images that determine whether or not the images are suitable for a medical expert to subject the images to a medical diagnosis. That is, the evaluation sub-system carries out a quality check on the images so as to determine whether or not the images are good enough for a medical expert (or a computerized system) to extract diagnostic information from the images.
  • Image processing operations are widely known in the art and beyond, and are conveniently carried out by image processing equipment that is implemented by means of a general-purpose computer running dedicated software, or dedicated electronic circuitry such as a graphics accelerator, also referred to as a graphics processing unit, or a combination of both dedicated software and dedicated hardware.
  • dedicated software or dedicated electronic circuitry such as a graphics accelerator, also referred to as a graphics processing unit, or a combination of both dedicated software and dedicated hardware.
  • the evaluation subsystem is configured for determining the quality on the basis of determining a presence or absence of at least one of following attributes in one or more images: motion blur; region-of- interest out of focus; too low a magnification factor; an area of a tissue being imaged is obscured from view by a natural fluid; an area of the tissue is obscured from view by a foreign object; specular reflection; inadequate cleaning of the tissue; insufficient coverage of the tissue by a chemical agent applied to the tissue for improving visualization of an abnormal area of the tissue.
  • the evaluation sub-system determines the quality by subjecting the images as captured to specific image processing operations to check for specific attributes that adversely affect the quality of the images and that may render the captured images unsuitable for medical diagnosis, either by a medical expert or by a computerized system.
  • the out-of-focus condition can be detected by, e.g., analysis of the frequency spectrum to determine whether there is a loss of high frequencies, e.g., by using a high-pass filter.
  • the cervix is the lower, narrow portion of the uterus where it joins with the top end of the vagina.
  • the insufficiency of detail may be determined in dependence on the region-of-interest being captured.
  • the region-of-interest is detected through, e.g., contour tracing or template matching.
  • a ratio of an actual characteristic size of a feature within the region-of-interest and the size of the feature within the image of the region-of-interest can then be used to quantify the sufficiency or insufficiency of detail.
  • an area of the tissue to be captured in the images is covered by, e.g., blood or mucus, the area is obscured from view. Bleeding or the secretion of mucus can be detected by, e.g., color analysis and/or movement of the fluid relative to tissue or relative to the camera.
  • the color of blood generally differs from the color of the tissue.
  • the presence of a foreign object in the images can be detected by, e.g., size, shape, and/or color of the foreign object.
  • the size and shape can be determined, e.g., by tracing the coherent movement of the pixels associated with the foreign object when the foreign object enters the area captured by the images as a result of a relative movement of the camera and the foreign object.
  • Specular reflection occurs if the light from the light source, illuminating the inside of the organ or cavity under inspection, is strongly reflected by the surface of the tissue into the camera. Specular reflection causes clipping.
  • clipping is used in digital photography and digital video and refers to a result of capturing or processing an image, wherein the intensity of the captured light in one or more areas of the image falls outside the range between the minimum intensity and the maximum intensity which can still be represented in the image.
  • Inadequate cleaning can be detected as a result of detecting bleeding and/or as a result of glare and/or as a result of an abnormal color or a texture of an abnormal material on the tissue's surface, which can themselves be detected as exemplified above.
  • the attribute "insufficient coverage of the tissue by a chemical agent applied to the tissue for improving visualization of an abnormal area of the tissue" is examined as follows. First, the region-of-interest is determined in the images. Then, the images are analyzed to determine whether or not there is a difference between the appearance of the region-of-interest in the images before applying the chemical agent and the appearance of the region-of-interest in the images just after applying the chemical agent. If the region-of-interest has been covered adequately, the appearance of the region-of-interest (just) after applying the chemical agent should have a uniform character. If the appearance is not uniform, his may be an indication that part of the region-of interest has not been covered adequately or not at all.
  • the human-perceptible notification may be given in a variety of ways through, e.g., a graphical user-interface or an auditory user-interface, or through annotations added to the images as captured, e.g., as an image overlay or added as metadata stored with the images. Accordingly, the monitoring of the quality of the images is delegated to the monitoring sub-system so that the operator can focus on execution of the medical imaging protocol, thus saving the operator from having to monitor quality as well as proper execution of the medical imaging protocol, both in real-time.
  • the evaluation sub-system is configured for processing the images during the execution of the medical imaging protocol; and the notification sub-system is configured for providing the human-perceptible notification via a user-interface to the operator during the execution of the medical imaging protocol.
  • the determining of the quality of the images, as well as the supply of the human-perceptible notification are carried out during operational use of the camera. This enables the operator to adjust his/her operating of the camera in case of a notification representative of insufficient quality and/or to continue operating the camera while being assured of the proper quality of the images that are being captured.
  • the notification sub-system is configured for annotating one or more specific ones of the images with the human- perceptible notification.
  • the assessment of the quality of the images may be done at any time before the images are being subjected to a medical diagnosis by a medical expert (or a computerized system).
  • the images are annotated so as to enable to discriminate between images of a quality adequate for being subjected to a medical diagnosis, and other images of a quality inadequate for being subjected to a medical diagnosis.
  • the quality assessment cannot be carried out at runtime, i.e., during execution of the medical imaging protocol, the assessment of the quality is done as soon as possible after completion of the medical imaging protocol in order to be enable the patient to return to the operator before the patient has left the premises.
  • the medical imaging protocol is carried out in a sequence of multiple steps or phases, and each specific one of the steps or phases is represented by a specific one of multiple series of successively captured ones of the images.
  • the evaluation sub-system is configured for identifying the specific step or phase. The identifying of the specific step comprises determining the specific series.
  • the determining of the specific series includes: identifying a particular one of the images captured next; determining a degree of similarity between a semantic content of the particular image and a further semantic content of one or more preceding ones of the captured images assigned to the specific series; assigning the particular image to the specific series if the degree of similarity complies with a pre-defined criterion; and assigning the particular image to a next one of the multiple series representative of a next one of the multiple steps if the degree of similarity does not comply with the pre-defined criterion.
  • Operation of the fourth embodiment is based on the premise that the content of a pair of images captured one after the other within same phase of the medical imaging protocol have a higher degree of similarity than a pair of images that have been captured not consecutively, e.g., in different steps or one of the pair of images near the beginning of the specific step, and the other one of the pair of images near the completion of the specific step.
  • a series of successively captured images can be formed by determining for a particular captured image whether or not the particular captured image matches one or more images captured immediately preceding the capturing of the particular image.
  • images are captured one after the other, and images captured in succession are grouped if the changes in their semantic content gradually changes from one of the images to the next one captured.
  • different series of captured images thus created may together cover a certain phase of the medical imaging protocol.
  • the phase of the medical imaging protocol may have been interrupted in order to apply an additional cleaning step before proceeding with the same phase.
  • the aceto -whitening step as discussed above may have to be repeated so as to cover anew the entire area of the tissue of interest. This means that the capturing will have to be interrupted for applying the 3-5% acetic acid solution, whose visual effects tend to peak and then wane within a few minutes.
  • a sudden drop in similarity between successively captured images may, but need not, be a candidate for identifying the transition from one step to a next step in the medical imaging protocol.
  • the images clustered in a particular series up to the occurrence of the drop in similarity may then be automatically labeled as belonging to a certain phase of the medical imaging protocol.
  • identifying each individual phase of the medical imaging protocol is facilitated by detecting certain characterizing attributes in the images per individual series. For example, a pinkish-white color dominating the images can be a clue for the aceto -whitening step in a colposcopy protocol, whereas a yellow/orange color can be a clue for the effects of Lugol's iodine in the step of the colposcopy protocol configured to visualize the location of abnormal tissue.
  • Information about the progress of the medical imaging protocol (such as the duration of each phase, the zoom modus of the camera, and the observed motion pattern) can also be exploited to identify a particular phase of the medical imaging protocol.
  • the notification sub-system of the medical imaging system in the invention can be controlled via the evaluation sub-system so as to guide the operator at runtime through the different steps of the medical imaging protocol, e.g., by indicating via a user- interface what step is to be performed next.
  • the notification subsystem is controlled so as to indicate to the operator at runtime what additional action is needed before completing the current step.
  • the evaluation sub-system processes the captured images at runtime in order to check whether the visual effect of the chemical agent applied (e.g., the acetic acid solution or Lugol's iodine) has been waning before the current step of the medical imaging protocol can be completed. If the current step has not been completed, the evaluation sub-system controls the notification sub-system so as to indicate to the operator of the medical imaging system that the chemical agent needs to be applied anew in order to successfully complete the current step of the medical imaging protocol.
  • the chemical agent applied e.g., the acetic acid solution or Lugol's iodine
  • the invention further relates to a monitoring sub-system configured for use in the medical imaging system specified above.
  • a monitoring sub-system can be commercially exploited as an after-market add-on to, or an upgrade of, a medical imaging system already installed at a hospital or at the office of a medical expert.
  • a general-purpose computer or a general-purpose data processing system can be configured to serve as the monitoring sub-system by means of installing suitable software, e.g., image processing software to detect attributes that affect the quality of the images as specified in detail above.
  • the invention also relates to an evaluation sub-system configured for use in the monitoring sub-system of the medical imaging system specified above.
  • an evaluation sub-system can be commercially exploited as an after-market add-on to, or an upgrade of, a medical imaging system already installed at a hospital or at the office of a medical expert.
  • a notification sub-system is then readily created from commercially available components, e.g., a general-purpose computer comprising a display monitor and/or a loudspeaker.
  • the invention further relates to control software for being installed on computer for use with a medical imaging system.
  • the control software may be provided as stored on a computer- readable medium such as a magnetic disc, an optical disc, or a solid-state memory. Alternatively, the control software may be provided as an electronic file downloadable from a server via data network such as the Internet.
  • the medical imaging system comprises a camera operative to capture images of an interior of a hollow organ or cavity of a body of a human being under control of an operator executing a medical imaging protocol.
  • the control software comprises first instructions for configuring the computer for receiving the images and for processing the images for determining a quality of the images received. The quality as determined by the evaluation sub-system determines whether the images are suitable for being subjected to a medical diagnosis.
  • the control software comprises second instructions for generating a signal for control of a user-interface to provide a human-perceptible notification under control of the quality as determined.
  • medical imaging systems that are installed at a hospital or at the office of a medical expert can be upgraded to provide the quality assessment functionality of the invention, as discussed above, by means of installing the control software on a computer, either an external computer or a computer forming a functional part of the medical imaging system.
  • the first instructions are configured for determining the quality on the basis of determining a presence or absence of at least one of following attributes in one or more images: motion blur; region-of-interest out of focus; too low a magnification factor; an area of a tissue being imaged is obscured from view by a natural fluid; an area of the tissue is obscured from view by a foreign object; specular reflection; inadequate cleaning of the tissue; and proper coverage of the tissue by a chemical agent applied to the tissue for improving visualization of an abnormal area of the tissue.
  • control software is configured for being used in a medical imaging protocol that is carried out in a sequence of multiple steps.
  • a specific one of the multiple steps is represented by a specific one of multiple series of successively captured ones of the images.
  • the first instructions comprise third instructions configured for identifying the specific step.
  • the third instructions comprise fourth instructions for determining the specific series.
  • the fourth instructions comprise: fifth instructions for identifying a particular one of the images captured next; sixth instructions for determining a degree of similarity between a semantic content of the particular image and a further semantic content of one or more preceding ones of the captured images assigned to the specific series; seventh instructions for assigning the particular image to the specific series if the degree of similarity complies with a pre-defined criterion; and eighth instructions for assigning the particular image to a next one of the multiple series representative of a next one of the multiple steps if the degree of similarity does not comply with the pre-defined criterion.
  • FIG. 1 is block diagrams of a system in the invention.
  • colposcopy is widely used for the diagnosis of, e.g., cervical cancer, and in some countries even for screening.
  • An area of, e.g., the cervical tissue is checked for visual signs typically observed in pre-cancerous and cancerous cervical lesions.
  • a colposcope 5 to 10 minutes of image/video data is acquired.
  • a video or a temporal sequence of images (referred to in this text as "image” or “images” for convenience) is obtained through the colposcope and is recorded on a computer-readable medium or is communicated at runtime to a remote location via a data network.
  • the images contain information about the cervical tissue during an imaging process, during which the cervical tissue is being subjected to successive steps of the colposcopy protocol: cleaning the cervical tissue, observing the cervical tissue with and then without a green color filter, aceto -whitening the cervical tissue, observing the response over time of the cervical tissue to the aceto -whitening, applying Lugol's iodine to the cervical tissue, and the detailed analysis of regions of interest of the cervical tissue.
  • All steps of the colposcopy protocol, detailed above, are preferably performed and are captured in the images at the appropriate level of detail and with adequate quality.
  • the regions of interest for colposcopy diagnosis vary from patient to patient.
  • interpretation can be hampered.
  • the medical expert discovers that important semantic content is missing, the patient may have left the clinical site. Repeating the procedure for diagnostic purposes therefore results in a lot of overhead and in discomfort for the patient.
  • Typical quality issues that arise in the colposcopy images include: motion blur; region of interest not in focus; insufficient detail (low magnification); bleeding regions; presence of foreign objects such a cotton swab, etc.; too much specular reflection (due to excess lighting or mucous); and inadequate cleaning of the cervix region being imaged.
  • identification of vasculature requires a higher level of detail than, for example, the identification of anatomical objects (e.g., cervix and os).
  • Fig.l is a diagram of a first medical imaging system 100 according to the invention.
  • the first medical imaging system 100 comprises a colposcope 102 with a camera (not shown) that is configured for capturing images under control of an operator and according to a colposcopy protocol .
  • the first medical imaging system 100 also comprises a monitoring sub-system 104 for monitoring execution of the colposcopy protocol.
  • the monitoring sub-system 104 comprises an evaluation sub-system 106 and a notification sub-system 108.
  • the evaluation sub-system 106 is configured for receiving the images from the colposcope 102 and for processing the images as captured to determine a quality of the images received.
  • the notification sub-system 108 is configured for providing a human-perceptible notification, e.g., to the operator of the colposcope 102 and through a user-interface 1 10, under control of the quality as determined by the evaluation sub-system 106.
  • the images from the colposcope 102 are captured for being medically diagnosed later on by a medical expert, e.g., a gynecologist.
  • the captured images are recorded in storage 1 12, for example, a computer-readable medium 112, such as a magnetic disk, an optical disc or a solid-state memory.
  • the evaluation sub-system 106 is configured for processing the images for automatically extracting information from visual attributes in the captured images. The information is then used to evaluate the execution of the colposcopy protocol by means of subjecting the captured images to quality checks. Examples of factors that affect the quality of the images as captured include motion blur; region-of- interest out of focus; insufficient detail; areas of the tissue being covered by a fluid, e.g., blood or mucus; presence of a foreign object, e.g., a cotton swab; specular reflection in the image; inadequate cleaning of the tissue. These factors have been discussed above as well as ways to detect their effects in the captured images.
  • the evaluation sub-system 106 determines that one or more of the images as captured do not comply with pre-determined quality criterions, the evaluation sub-system 106 controls the notification sub-system 108 to provide a notification for use by a human operator.
  • the notification is implemented, for example, by controlling the user-interface 1 10 to provide a visual or auditory signal to the operator during the execution of the colposcopy protocol.
  • the notification is implemented by an annotation added to the images as stored in the storage 112.
  • the notification may be added to the images as, e.g., an icon or mark in an overlay over the semantic content of the images themselves or as metadata that is stored in an electronic file together with the images, and that is rendered, e.g., as text, at the time of retrieval of the images from storage 112 and/or at the time of rendering on a display monitor the images retrieved from storage 112.
  • the notification sub-system 108 also supplies another such notification in case the quality of the images as captured is adequate according to the findings of the evaluation subsystem 106.
  • notifications are given to notify, e.g., the operator of the colposcope 102 or the user of the captured images as stored, of the fact that the images have indeed been subjected to the quality check.
  • the conclusion may then be drawn that at least one of the operator of the colposcope 102, the colposcope 102, the evaluation sub-system 106, the notification sub-system 108, the user-interface 110, the storage 1 12 or their interconnection is not functioning properly.
  • the evaluation sub-system 106 performs an automated analysis of the images as captured or as recorded in storage 1 12, so as to verify whether the captured images are suitable for being subjected to a medical diagnosis.
  • the human-perceptible notification from the notification sub-system 108 serves as feedback to the operator, e.g., via UI 110, so as to guide the operator through the image capturing procedure.
  • the feedback is given to the operator in the form of a "picture-in-picture" via a display monitor sub-system (not shown) of the UI 1 10.
  • the notification subsystem 108 controls the display monitor sub-system to show on the display screen the part at a higher magnification, as well as a smaller, additional picture on the same screen, of the area as it was being captured before the zooming.
  • the location of the part in the magnified image is indicated in the smaller picture of the original area, e.g., by means of a graphics rectangle centered on the location. This picture-in-picture feedback assists the operator in navigating.
  • the feedback is given as visual information, plotted in an overlay on the screen of the display monitor, to indicate the progress of the colposcopy protocol.
  • the notification sub-system 108 controls the display monitor sub-system in such a way, that regions of the tissue being imaged which have been captured in the images at the proper magnification and with adequate quality, are marked in the images with a particular color or with particular shading. Other regions that have not been captured, or that have been captured with insufficient quality are unmarked, or are marked with a different color or with different shading.
  • the part of the tissue being imaged can be covered by a natural fluid, such as blood or mucus, and is thus obscured from view.
  • a natural fluid such as blood or mucus
  • the notification sub-system 108 controls the UI 110 in order to have the UI 1 10 provide feedback to the operator of the colposcope 102.
  • the feedback is given in the form of, e.g., an auditory or visual message, that the tissue is obscured from view, and that cleaning is required.
  • the problematic areas of the tissue that need cleaning can be indicated on the display monitor, e.g., as marked with a specific color in an overlay.
  • a chemical agent for improving visualization of abnormal tissue such as acetic acid or Lugol's iodine as used in a colposcopy protocol. If the chemical agent does not cover the entire area of the tissue of interest, the resulting visual effects in colposcopy images as captured may give rise to a faulty interpretation during the medical diagnosis.
  • the presence or absence of the chemical agent can be inferred from image analysis and the notification sub-system 108 can be used to provide feedback, via the UI 1 10, to the operator of the colposcope 102 on areas where the chemical agent is incompletely applied.
  • the notification sub-system 108 may be used to provide, via the UI 110, feedback to the operator of the colposcope 102 when sufficient images have been captured for a given step of the colposcopy protocol so as to prevent that the next step of the colposcopy protocol is initiated too early.
  • the evaluation sub-system 106 processes the images, for example, to determine if the entire region-of-interest has been covered by the images as captured and with sufficient detail.
  • the evaluation sub-system 106 subjects the images, as soon as they are captured, to image processing techniques in order to detect the presence of abnormal patterns in an area of the tissue being imaged.
  • the evaluation sub-system 106 is then used to control the notification sub-system 108 to provide feedback to the operator on these abnormal patterns.
  • the operator of the colposcope 102 may then control the colposcope 102 so as to capture additional images from the area with the abnormal pattern or to capture additional images at a higher magnification.
  • the medical imaging protocol in colposcopy is carried out in a sequence of multiple steps.
  • a specific step is represented by a specific series of successively captured images.
  • the evaluation sub-system 106 may be configured for identifying one or more specific ones of the steps in the sequence.
  • the evaluation sub-system 106 is configured for identifying the specific step by means of determining the specific series of successively captured images that are representative of the specific step.
  • the specific series may be determined as follows. A particular one of the images captured next is identified. Then, a degree of similarity is determined between a semantic content of the particular image and a further semantic content of one or more preceding ones of the captured images that have been assigned to the specific series.
  • the evaluation sub-system 106 assigns the particular image to the specific series. If the degree of similarity does not comply with the predefined criterion, the evaluation sub-system 106 assigns the particular image to a next series representative of a next step in the colposcopy protocol.
  • Fig.2 is a diagram of a second medical imaging system 200 according to the invention.
  • the second medical imaging system 200 comprises the components of the first medical imaging system 100, to wit: the colposcope 102, the evaluation sub-system 106, the notification subsystem 108, the user- interface 1 10 and the storage 112.
  • these components are now geographically distributed and form a distributed image processing architecture whose components are interconnected via a data network 202, such as the Internet.
  • one of the tasks of the evaluation sub-system 108 is to check the quality of the images as captured so as to determine whether or not the captured images are suitable for being subjected to a medical diagnosis.
  • the evaluation sub-system 108 is then operative to subject the images to image processing operations, either during the execution of the colposcopy protocol or later upon retrieval of the images from storage 1 12 where they have been stored upon having been captured.
  • the evaluation sub-system 108 can be rendered operative for this task by means of configuring a general-purpose computer, e.g., by means of installing application software 202 based on algorithms known from image processing techniques such as, e.g., feature extraction and pattern recognition and by means of installing control software 204 for having the evaluation sub-system 106 interface with the notification sub-system 108.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Physics & Mathematics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Pathology (AREA)
  • Veterinary Medicine (AREA)
  • Optics & Photonics (AREA)
  • Public Health (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Endoscopes (AREA)

Abstract

Un système d'imagerie médicale comprend un colposcope avec une caméra et un sous-système de surveillance. Le sous-système de surveillance est opérationnel pour surveiller la qualité des images capturées par la caméra afin de vérifier si oui ou non la qualité est adéquate pour soumettre les images à un diagnostic médical.
PCT/IB2013/052455 2012-04-02 2013-03-27 Contrôle qualité pendant une procédure d'imagerie médicale WO2013150419A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201261618904P 2012-04-02 2012-04-02
US61/618,904 2012-04-02

Publications (1)

Publication Number Publication Date
WO2013150419A1 true WO2013150419A1 (fr) 2013-10-10

Family

ID=48471046

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2013/052455 WO2013150419A1 (fr) 2012-04-02 2013-03-27 Contrôle qualité pendant une procédure d'imagerie médicale

Country Status (1)

Country Link
WO (1) WO2013150419A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2887266A1 (fr) * 2013-12-23 2015-06-24 Koninklijke Philips N.V. Extraction automatique d'une région d'intérêt sur la base de la détection du spéculum
EP2986199B1 (fr) * 2013-04-18 2018-01-31 Koninklijke Philips N.V. Acquisition d'images des cervicales
CN115484456A (zh) * 2022-09-15 2022-12-16 重庆邮电大学 一种基于语义聚类的视频异常预测方法及装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050192476A1 (en) * 2004-03-01 2005-09-01 Olympus Corporation Endoscope image pick-up apparatus
US20060034503A1 (en) * 2004-08-12 2006-02-16 Fuji Photo Film Co., Ltd. Medical image processing system
EP2149326A1 (fr) * 2007-05-08 2010-02-03 Olympus Corporation Dispositif de traitement d'image et programme de traitement d'image
US7809197B2 (en) * 2004-12-09 2010-10-05 Eastman Kodak Company Method for automatically determining the acceptability of a digital image
US20110251454A1 (en) * 2008-11-21 2011-10-13 Mayo Foundation For Medical Education And Research Colonoscopy Tracking and Evaluation System
US20120008866A1 (en) * 2010-06-28 2012-01-12 Jad Halimeh Method and device for detecting an interfering object in a camera image

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050192476A1 (en) * 2004-03-01 2005-09-01 Olympus Corporation Endoscope image pick-up apparatus
US20060034503A1 (en) * 2004-08-12 2006-02-16 Fuji Photo Film Co., Ltd. Medical image processing system
US7809197B2 (en) * 2004-12-09 2010-10-05 Eastman Kodak Company Method for automatically determining the acceptability of a digital image
EP2149326A1 (fr) * 2007-05-08 2010-02-03 Olympus Corporation Dispositif de traitement d'image et programme de traitement d'image
US20110251454A1 (en) * 2008-11-21 2011-10-13 Mayo Foundation For Medical Education And Research Colonoscopy Tracking and Evaluation System
US20120008866A1 (en) * 2010-06-28 2012-01-12 Jad Halimeh Method and device for detecting an interfering object in a camera image

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
ALFRED E. BRENT, MD; JOHN W. CALKINS, MD; FRANK J. GAUDIANO, JR., MD; KATHLEEN MCINTYRE-SELTMAN, MD; JOSE E. TORRES, MD, CORE CURRICULUM FOR TEACHING COLPOSCOPY IN RESIDENCY PROGRAMS, pages 27
JOHN W. SELLORS M.D.; R. SANKARANARAYANAN, M.D.: "Colposcopy and Treatment of Cervical Intraepithelial Neoplasia: A Beginners' Manual", 2003, IARC PRESS
SUN Y. PARK; DUSTIN SARGENT; RICHARD LIEBERMAN; ULF GUSTAFSSON: "Domain-Specific Image Analysis for Cervical Neoplasia Detection Based on Conditional Random Fields", IEEE TRANS. MED. IMAG., vol. 30, no. 3, March 2011 (2011-03-01), XP011349075, DOI: doi:10.1109/TMI.2011.2106796

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2986199B1 (fr) * 2013-04-18 2018-01-31 Koninklijke Philips N.V. Acquisition d'images des cervicales
US10178971B2 (en) 2013-04-18 2019-01-15 Koninklijke Philips N.V. Acquiring cervical images
EP2887266A1 (fr) * 2013-12-23 2015-06-24 Koninklijke Philips N.V. Extraction automatique d'une région d'intérêt sur la base de la détection du spéculum
CN115484456A (zh) * 2022-09-15 2022-12-16 重庆邮电大学 一种基于语义聚类的视频异常预测方法及装置
CN115484456B (zh) * 2022-09-15 2024-05-07 重庆邮电大学 一种基于语义聚类的视频异常预测方法及装置

Similar Documents

Publication Publication Date Title
CN113573654B (zh) 用于检测并测定病灶尺寸的ai系统、方法和存储介质
JP7335552B2 (ja) 画像診断支援装置、学習済みモデル、画像診断支援装置の作動方法および画像診断支援プログラム
JP6231885B2 (ja) 頸部を検査する医療機器
WO2018165620A1 (fr) Systèmes et procédés de classification d'images cliniques
CN103975364B (zh) 针对宫颈的光学检查的图像选择
WO2021147429A9 (fr) Procédé, appareil, dispositif informatique et support de stockage pour affichage d'image endoscopique
WO2020215810A1 (fr) Procédé de détection d'image à bande étroite basé sur la reconnaissance d'image pour procédure de coloscopie
CN109460717A (zh) 消化道共聚焦激光显微内镜病变图像识别方法及装置
JP2017099509A (ja) 内視鏡業務支援システム
CN112801958A (zh) 超声内镜、人工智能辅助鉴别方法、系统、终端、介质
CN114569043A (zh) 一种基于人工智能的内窥镜辅助检查方法及装置
WO2013150419A1 (fr) Contrôle qualité pendant une procédure d'imagerie médicale
WO2019088008A1 (fr) Appareil de traitement d'image, procédé de traitement d'image, programme et système d'endoscope
KR20190141934A (ko) Oct 의료 영상 기반 인공지능 컴퓨터 보조 진단 시스템 및 그 방법
Li et al. Computer-aided Diagnosis (CAD) for cervical cancer screening and diagnosis: a new system design in medical image processing
JP6710853B2 (ja) プローブ型共焦点レーザー顕微内視鏡画像診断支援装置
US11830185B2 (en) Medical image processing system and learning method
JP2024509105A (ja) Ｔｕｒｐの病理画像から前立腺癌を検出するための人工ニューラルネットワークを学習する方法、及びこれを行うコンピューティングシステム
Bernal et al. Towards intelligent systems for colonoscopy
Andrade A Portable System for Screening of Cervical Cancer
WO2024166731A1 (fr) Dispositif de traitement d'images, endoscope, procédé de traitement d'images, et programme
WO2024171780A1 (fr) Dispositif d'assistance médicale, endoscope, méthode d'assistance médicale, et programme
Andrade et al. Automatic Segmentation of the Cervical Region in Colposcopic Images.
WO2024185468A1 (fr) Dispositif d'assistance médicale, système endoscope, procédé d'assistance médicale et programme
Li et al. Computerized image analysis for acetic acid induced intraepithelial lesions

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13724377

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13724377

Country of ref document: EP

Kind code of ref document: A1