ES2303790B1 - PROCEDURE FOR ANALYSIS, VISUALIZATION AND PROCESSING OF BIOMEDICAL DIGITAL IMAGES. - Google Patents

Abstract

Analysis, visualization and Biomedical digital image processing.

A procedure of analysis, visualization and Biomedical digital image processing, comprising:

- storing at least one biomedical digital image in a first size I g on a remote server (20, 21);

- rescaling said biomedical image to an image in a second size I m (1);

- establishing an initial value to relate said image to the second size I m (1) with an image in a third size I v (4) for an initial execution of the procedure;

- selecting in said image the second size I m or the third size I v a fragment (2) to be displayed, which is determined by initial (xi, yi) and final (xf, yf) coordinates;

- select an increase (3) for said fragment;

- send a request to the remote server containing at least said initial and final coordinates and said
magnification in the image at the second size I m (1) or at the third size I v (4);

- recovering by re-scaling a fragment of the image at the first size I g corresponding to said initial and final coordinates and said magnification and visualizing it in said image at the third size I v.

Description

Analysis, visualization and Biomedical digital image processing.

Object of the invention

The present invention relates to a procedure for remote analysis of digital images Biomedical for viewing, processing and diagnosis. He procedure of the invention performs the analysis, visualization and processing of biomedical images of different formats, sizes and for different pathologies.

Background of the invention

At present there is no tool or procedure that analyzes, visualizes, manages and processes automatically and also remotely and independently of the digitalization format, biomedical digital images, with dimensions of several Gbytes, as the images certainly are digitized in Biomedicine and in particular in the specialty of Pathological anatomy.

Currently the only thing that exists is a remote viewing methodology, to visualize images of up to the order of 2 Gbyte, with split protocols in fragments The methodology is known by Zoomifyer. Zoomifyer was developed in the USA and allows viewing images up to 1.5 Gbytes of formats BMPs, TIFs, JPGs, and GIFs, in web environment. He method consists of copying the image many times to different resolution levels, dividing it into hundreds of files JPGs; This is what is known as pyramidal strip structure. This technology has a number of drawbacks:

-

The division of the image into thousands of pieces deteriorates the image (loss and corruption of files).

-

The division of the image means that when viewed at different magnifications It looks pixelated into pieces.

-

There is limitation up to the order of 2 Gbytes.

-

There is format type limitation.

-

The original image transforms it to JPGs image, which can take associated with a loss due to compression, which makes it difficult for perform processed on the image with results exact.

-

He Image analysis is limited to display only.

Zoomifyer makes use of US5968120 patents, (" Method and system for providing online interactivity over a server-client network ") for remote client-server transmission prior to image fragmentation using the pyramid methodology, previously creating files other than the same image region at different resolutions to be able to view it. This pyramidal methodology is described in US Patent US-A-5760783, (" Method and system for providing texture using a selected portion of a texture map "). The problems and limitations of this methodology are those listed previously.

There are other patented methodologies related to procedures for visualizing images but all with limitations to achieve the necessary objectives for analysis (both visualization and processing) and management of Biomedical images. The following patent documents are mentioned by way of example: Korean patent application KR-A-20040104070, (" Interactive Multimedia Image Viewer Using Flash / Server Side Technology and Database ") allows you to analyze images only for viewing but on video images carrying a limited capacity of 500Mb. European patent application EP-1646201-A1 (" Client dependent image processing for browser-based image document viewer for handheld client devices ") allows you to view static images, stored on a disk drive, without analysis and processing characteristics and with the same limitation of handling images greater than 500 Mb.

Applied to the medical field there is, for example, the Japanese patent application JP-A-2005312683 (" Medical image Distribution System and Its Viewer Terminal Equipment "), which describes a storage and display system for only preprocessed and image contrast , to enhance or improve its resolution by facilitating its display. In addition to being limited by the processing, it is limited in the type of image, associated with the radiological image and therefore its dimensions and its flexibility from the point of view of remote analysis. Also, US Patent Application US-A1-2002 / 0133373, (" Integration of radiology information into an application service provider DICOM image archive and / or web based viewer "), describes a system that manages image information by integrating the DICOM image along with the radiological information. Limited only to the radiological image, not biomedical, whose standard to DICOM is not defined; Moreover, the radiological image does not have the problems of biomedical image dimensions or the problems of color management. Likewise, this tool has unilateral capabilities, that is, it does not allow sharing between several users (medical specialists).

Description of the invention

The invention relates to a method of analysis, visualization and processing of digital images Biomedical according to claim 1. Embodiments Preferred process are defined in the claims Dependents

The present invention provides a analysis, visualization and processing procedure that solves the problem of analyzing (managing), visualizing and processing Automatically biomedical digital images, with no limit of dimensions (Gbytes), of various formats and it is also possible do it remotely, using the inventive material included in the attached independent claims.

The invention relates to a method of analysis, visualization and processing of digital images biomedical that as a methodology has its own architecture that allows the decompression of any image in any format - jpeg, jpeg2000, tiff, bmp, ppm, pgm, gif, svs - and dimension to a only unzipped file free of header information and compression that keeps the "raw" information obtained by the digitalization of the image, for later viewing and indicted.

According to a first aspect of the invention, this refers to an analysis procedure, visualization and processing of biomedical digital images, which understands:

- storing at least one biomedical digital image in a first size I g or original image on a remote server;

- rescaling said original image to an image in a second size I m;

- Selecting in said image in the second size I m a fragment to be displayed - preferably it will be to enlarge it - whose fragment is determined by initial coordinates and final coordinates;

- select an increase for said fragment a visualize, being the increase defined by a user;

- sending to said remote server a request containing at least said initial coordinates and said final coordinates and said increase in the image in the second size I m;

- recovering by re-scaling a fragment of the biomedical digital image in the first corresponding size I g said said initial coordinates and said final coordinates and said increase and displaying it in an image to a third size I v.

Preferably said second size of the image is smaller than the first size of the digital biomedical image original.

The selection of the fragment to be displayed can also be made in said image in the third size I v or image to be displayed. In such a case, an initial value can be set to relate said image in the second size I m with said image in the third size I v, for the first execution of the procedure. In this case, the sending to said remote server of a request containing at least said initial coordinates and said final coordinates and said increase will be referred to on the image in the third size I v.

As just described, this procedure of analysis, visualization and processing of biomedical digital images allows dealing with huge images (biomedical digital image at the first size I g) without dimension limit and without loss or deterioration, avoiding data inconsistency and duplicity of information because it maintains a single image and manages its access for visualization and processing through its relationship with another thumbnail image (biomedical digital image at the second size I m).

Thus, in the process of the invention, a user - through, for example, the corresponding user interface - selects the fragment or the image region that is intended to be displayed, and with what magnification or zoom level that fragment is to be displayed. This is done on the biomedical digital image at a second size I m resulting from rescaling the original biomedical digital image that is at a first size I g. The request that includes the magnification or zoom applied and the initial and final coordinates of said fragment within the image in the second size I m, is sent to a remote server that returns the image fragment to be displayed from the image. original image at the first size I g; This original biomedical digital image is not fragmented or divided, but is a single file without size limitation, stored on a server. The size of the file is not what it really should be displayed, this allows to manage images of values of dimensions of the Gigabyte. Through the procedure outlined, only that file is accessed, the region from which the fragment or region to be managed or visualized (small fragment) must be extracted, so that it can be displayed to the user. In this way the memory to be used and the fragment to be transmitted is of small dimensions (eg, approximately 800x800).

When treating the original biomedical digital image as a single file limitations of size and duplicity are avoided of information, which makes the process of the invention capable to solve the problem of management and analysis (visualization and processed) of images of enormous dimensions greater than 2 Gigabytes In this way there is also no need to maintain a database containing all the information of tile correspondence that requires, for example, Zoomifyer to The image display.

The process of the invention may further comprise detecting in said fragment -fracción R _ {g} or {entirety of I g} in case of increase 1 * -of biomedical digital image to the first size I _ {g} of one or more regions of interest by:

-

statistical analysis of said fragment, determining at least four centroids or values with greater frequency in color coordinates;

-

fuzzy logic clustering by grouping pixels in different classes, one class per each centroid, minimizing the color distance of the value of each pixel to the centroid of the class;

-

determination of values of belonging of each pixel to a class, detecting the one or more regions of interest

You can also analyze one or more regions of interest through architectural criteria and morphological, producing a set of specific descriptors for each type of biomedical digital image, so as to facilitate relevant information to the user (specialist) and thus facilitate final diagnosis of the sample (represented by the digital image biomedical).

To said one or more regions of interest detected a series of descriptors can be applied morphometric and / or densitometric.

The procedure of analysis and processing of the invention also allows you to insert comments as well as your subsequent management (comment modification, acceptance, inclusion of new comments) and their recovery with their  corresponding re-positioning, to the increase in the one that was made, as an integral part of the images.

Preferably the analysis procedure and processing of the invention includes management tools and processed that are associated with said biomedical image.

The increase for the fragment to be displayed can be 2 *, 10 *, 20 *, 40 * or 80 *; can be selected by the user and / or come selected by default.

In case the increase is greater than 20 *, preferably said fragment is divided into six or more classes. In such case, preferably another iteratively is done Clustering by fuzzy logic in the image of two or more regions of interest of the same hue to detect other regions of interest.

The preferred architecture design for the procedure, it is through Web service, which also allows incorporate such functionalities added to the visualization and processed (remote and parallel), such as: inserting comments in a collaborative and image processing environment; environment collaborative in which different users can share comments and opinions on certain regions of the image in real time.

These functionalities are executed preferably remotely; can be analyzed any Biomedical digital image including Biomedical images that are sent remotely.

Preferably the relevant information Diagnostic is included in medical forms, complementing also with the management of information databases of interest clinician-doctor

In addition preferably all communication of files and sending information, both image and management, It is done through secure communication protocols (ssh). The progressive use of this secure transmission protocol allows the  transmission of new cases (of any size) so Remote without posing problems.

There is currently no tool that provides collaborative tools for insertion and Management of diagnostic comments on biomedical images. Being possible to access and manage these comments simultaneously and in real time through the web by several user specialists,  in addition to sharing opinions in real time before or during diagnostic annotations

According to the characteristics that have been defined in the foregoing for the process of the present invention, this It allows:

- Viewing images with different dimensions coming to the order of GBytes.

- The integration of several different formats: .jpeg2000, .jpeg, .tif., .bmp, .ppm, .raw, .pgn, .gif, svs etc. Yes well the use of image formats such as .bmp, .jpeg, .ppm and .gif is extended .jpeg2000 format is a new feature of this procedure, as well as the integration of all of them to the management and processing of biomedical images.

In addition, according to the implementation Preferred of the invention, the process includes:

- Collaborative tools for insertion and comment management in biomedical images: it is possible access and manage these comments simultaneously and in real time through the web by several user specialists, in addition to sharing opinions in real time before or during diagnostic annotations

- Multiple processing of different images Diagnostic biomedical: the procedure integrates a set of methods for processing diagnostic images, adapting each processed to the analysis needs of these images. It allows to analyze histopathology, radiotherapy images Oncology, neurology and breast pathology, through processing of TMA (Tissue MicroArray), H&E (Hematoxylin and Eosin), Cytologies, CT (Computerized Axial Tomography), MR (Resonance Magnetic), Angiographies and Mammograms.

- Visualization at different magnification or zoom for images of different sizes adapting to the specific zooms used, particularly in the pathology specialty to favor the analysis of the images. The response time of this zoom is immediate for any type of images whatever their dimensions.

Brief description of the drawings

Then it goes on to describe very brief a series of drawings that help to better understand the invention and that expressly relate to an embodiment of said invention presented as a non-limiting example of is.

Figure 1 shows a general scheme of a application in which the procedure of the invention.

Figure 2 shows how biomedical images can be viewed at different zooms . And the possibility of measuring areas of interest.

Figure 3 shows a screen of the corresponding application to a Preprocessed image.

Figures 4, 5 and 6 show a screen corresponding to the Insertion, Management and Recovery of comments, respectively.

Figures 7, 8A-8B and 9A-9B, show examples of image processing biomedical, H&E for cell detection, DO7 images and TMA images, respectively.

Figure 10 shows examples of integration of diagnostic information within other information relevant for diagnosis and prognosis.

Figure 11 shows the physical elements that make up a possible realization for the system that implements the method of the invention

Description of a preferred embodiment of the invention

Figure 1 shows a general scheme of a application in which the procedure of the invention. This figure 1 shows indicative elements of the description of the invention, such as:

- Image rescaled to the second size I m (1) of the original biomedical image I g.

- Selection in said image in the second size I m or in the third size I v of a fragment (2) to be displayed, fragment is determined by initial coordinates xi, yi and final coordinates xf, and f .

- The panel for the selection of an increase (3) between 2 *, 10 *, 20 *, 40 * or 80 *.

- Image display area in the third size I v (4), image recovered from the original image I g by rescaling from the coordinates (xi, yi) e (xf, yf ) and the increase (3) selected:

- The processing and measurement panels (5) where the method of the invention includes biomedical image processing tools. The procedure thus allows the user to select and perform the processing of the image or fragment displayed at a third size I v.

- The comment panels (6) where the Procedure of the invention includes collaborative tools for the insertion and management of diagnostic comments in biomedical images. Inside the comments panel (6) a field (6a) with the coordinates of the position of the comment in the image.

Figure 2 shows how biomedical images can be viewed at different zoom or magnification (3). And the possibility of measuring areas of interest.

In the illustrative example, it is shown how the fragment (2) selected in the image at the second size I m (1) for a selected magnification (3) of 40 *, said fragment has been recovered by rescaling Biomedical digital image in the first size I g corresponding to said initial coordinates and said final coordinates and said magnification and has been displayed at a third size I v (4).

In addition, Figure 2 illustrates a work session for measurements of regions of interest, selecting the points (A, B) of interest to be measured on the image at the third size I v (4). The measurements are applied when using panel analysis options (5) and are considered as processing and values of diagnostic interest.

Figure 3 shows an application screen corresponding to an image processing consisting of a variation of the contrast on the image to the third size I v (4), obtained by using panel analysis options
(5).

Figures 4, 5 and 6 illustrate tools collaborative proceedings, through comments associated with Biomedical images These are utilization tools simultaneous and in real time via web by vain specialists users

The application has the possibility to insert comments (see figure 4), selecting the area of interest where you want to comment with the mouse.

Also the tool allows management integral of these comments, and thus it is possible to delete, modify or include new comments (see figure 5). And also recover a comment with its re-positioning to the increase in the one that was done (see figure 6).

The insertion of comments is carried out by means of a mouse click in the area of interest where the fragment of I g has been visualized to a third size I v (4).

As shown in Figure 6, the management of comments are carried out by selecting by clicking with the mouse in areas or regions of interest, after which a form to insert a comment. This can be modified, delete and / or add additional comments. It is also possible retrieve the area referenced by the comment in the original format in which it was inserted, that is, to the increase in corresponding image.

Retrieving comments by locating them in the inserted area and the specified zoom is carried out selecting by mouse click on the panel (6) the coordinates of the comment to retrieve.

Figures 7, 8A-8B and 9A-9B show examples of processing of biomedical digital images, of different types and formats. That is, one or more regions of interest can be detected in I g or in the fragment visualized in I v.

Figure 7 illustrates the detection of regions of interest in images stained with Hematoxylin-Eosin (H&E) by clustering by fuzzy logic to four statistically determined centroids in I g or fragment visualized in I v in a model of HSI color and jointly visualizing two of said centroids corresponding to nuclei and nucleoli.

Figure 8A illustrates the image analysis of DO7 for processing at 40 * (Figure 8B) detecting the regions of interest according to their staining levels. Five have been identified centroids performing a cluster by minimizing the color distance of the value of each pixel to the centroid of the class.

Figure 9A illustrates the analysis of microarray images (TMA) for processing at 10 * (Figure 9B) detecting regions of interest by clustering by fuzzy logic to four statistically determined centroids in I g or fragment visualized in I _ {v} in CIELab color model.

Figure 10 shows examples of integration of diagnostic information within other information relevant for diagnosis and prognosis. It shows through two illustrations:

Figure 10B shows the analysis of said one or more regions of interest by criteria architectural, morphometric and / or densitometric, producing a set of specific descriptors, for each type of image biomedical digital, to help diagnose this image Biomedical digital As an illustrative example, Figure 10B shows shows the number of regions of interest detected in the area analyzed, the dimension of these regions, the region of interest which has acquired staining to the reagent applied, the coordinates of color of said region of interest.

Figure 10A shows the analysis performed on the descriptors, which can be integrated into a relevant clinical history and information corresponding to a patient to whom said biomedical digital image belongs. The diagnostic history produced by the image processing is integrated into a database implemented and integrated into the procedure, which contains other information relevant to the joint diagnosis, from patient characteristics to characteristics of the sample to be analyzed from which it has been acquired the biomedical image I g.

       \ newpage
    

The application is designed following a model three independent layers, but interconnected:

- Domain Layer : represents the logical content of the application, being able to say that it is the computational part of it.

- Presentation Layer : it is oriented to present the information to the user.

- Persistence Layer : stores the context on the operation of the viewer on a server.

Starting from this three-layer model, elaborates in parallel the design of a database, and of the classes necessary to implement the domain layer.

The application viewer is implemented as a Web service to accommodate users remotely using the server as "engine".

It also includes treatment options for image and operations that facilitate actions such as the change between image format or storage of same.

The viewer utilities implemented in a preferred embodiment of the invention, illustrated in figures 1 to 10, are:

- Image management.

- Image preprocessing: zoom , change brightness and contrast.

- Edition: comment management, measurements of points of interest.

Both the load, as the display and Preprocessing of an image is done quickly in a few seconds.

The elements are shown in Figure 11 physicists that make up a possible realization for the system that implements the method of the invention, for use of the Application remotely and parallel processing.

It has three essential components in its architecture:

- Web server applications 10. The main server function is to remotely manage the possible requests that users can make simultaneously (request to store new images in the cluster, visualization, analysis or processing of a certain image). To its instead, it maintains a database management system 11 with the information corresponding to the images themselves and the part Collaborative application environment.

- Cluster 20 with MPP architecture (Massively Parallel Processors). The cluster enables remote execution of various parallel processing tasks (detection of zones of interest, descriptors, general algorithms). Communication between Nodes are designed using MPI (message passing interface). The cluster is also used as a massive device for storage 21 for images managed by the application, as well as for the results obtained after processing. The images can be stored directly in the cluster through of external physical media (through a digitalization equipment, such as a digital microscope 30) or through the request of customers 40.

The connection between the cluster and the server has been designed using a secure connection protocol with various purposes:

-

Update in the database of server possible new cases that enter by direct connection to the cluster (scanning equipment).

-

Task Execution Request for Processing for a certain image.

-

Image transmission complete.

-

Chores display.

- Customers 40. Present the information to user and enable its interaction with the application raised. It communicates with the web server through a protocol Secure remote communication.

The invention has been described according to a preferred embodiment thereof, but for the expert in the matter will be apparent that multiple variations can be introduced in said preferred embodiment without exceeding the object of the claimed invention.

Claims (10)

1. A procedure for analysis, visualization and processing of biomedical digital images, characterized in that it comprises: - storing at least one biomedical digital image in a first size I g on a remote server (20, 21); - rescaling said biomedical image to an image in a second size I m (1); - establishing an initial value to relate said image in the second size I m (1) with an image in a third size I v (4) for an initial execution of the procedure; - select in said image in the second size I m (1) or in said image in the third size I v (4) a fragment (2) to be displayed, whose fragment is determined by initial coordinates ( xi, yi) and final coordinates (xf, yf); - select an increase (3) for said fragment to visualize; - sending to said remote server a request containing at least said initial coordinates and said final coordinates and said increase in the image in the second size I m (1) or in the image in the third size I v (4); - recovering by re-scaling a fragment of the biomedical digital image in the first size I g corresponding to said initial coordinates and said final coordinates and said increase and display it in said image to a third size I v (4 ). 2. Method according to claim 1, characterized in that it further comprises detecting in said fragment of the biomedical digital image at the first size I g one or more regions of interest by:

-

statistical analysis of said fragment, determining at least four centroids or values with greater frequency in color coordinates;

-

fuzzy logic clustering by grouping pixels in different classes, one class per each centroid, minimizing the color distance of the value of each pixel to the centroid of the class;

-

determination of values of belonging of each pixel to a class, detecting the one or more regions of interest

3. Method according to claim 2, characterized in that said one or more regions of interest are also analyzed by means of architectural, morphometric and / or densitometric criteria, producing a set of specific descriptors, for each type of biomedical digital image, to aid in a diagnosis of said biomedical digital image. 4. Method according to claim 3, characterized in that an analysis of the descriptors can also be integrated into a relevant clinical history and information corresponding to a patient to whom said biomedical digital image belongs. 5. Procedure according to any of the preceding claims, characterized in that it also includes inserting comments as well as their subsequent management -modification of comments, acceptance, inclusion of new comments-, and recovery thereof with their re-positioning to the increase in which He did, as an integral part of biomedical digital images. 6. Method of analysis and processing of biomedical digital images according to any of the preceding claims, characterized in that said increase is 2 *, 10 *, 20 *, 40 * or 80 *. 7. Method according to any of the preceding claims, characterized in that in case of an increase greater than 20 * said fragment is divided into six or more classes. 8. Method according to any of the preceding claims, characterized in that in case of an increase greater than 20 * another clustering is done iteratively by diffuse logic in the image of two or more regions of interest of the same hue to detect other regions of interest. 9. A program product that comprises means of program instructions to carry out the procedure defined in any of the claims 1-8. 10. A product according to claim 9, stored in a program support medium.