AU765024B2 - Data distribution system - Google Patents

Description

'4

AUSTRALIA

Patents Act 1990 COMPLETE SPECIFICATION STANDARD PATENT Applicant(s): ALGOTEC SYSTEMS LTD.

Invention Title: DATA DISTRIBUTION SYSTEM The following statement is a full description of this invention, including the best method of performing it known to me/us: 2 DATA DISTRIBUTION SYSTEM FIELD OF THE INVENTION This invention is concerned with interactive communication systems linking central locations having access to stores of data and images used for medical purposes and a plurality of outlying users of the images and data for medical review, processing, assessment and diagnostics.

BACKGROUND OF THE INVENTION Modern hospitals and health centres today usually have several computerized systems for medical information gathering, exchange, storage and processing. Herein such a system is referred to as a "data source". Medical information may come in textual, voice, sound, graphical, and image modalities. Such medical information may be required by authorized personnel, including those located outside the hospital premises, and equipped with computers of some sort. Herein the requiring side is referred to as the "user". In present systems the users are equipped with their own software to access the data source. Difficulties in the use of such computerized systems are caused by such things as the varied networking procedures required to fetch the data, the lack of an industry standard, the lack of an easy to use user interface and, in the case of image data transfer, the channel bandwidth requirements along with the typically large volumes of the image information, which in turn translates into very long transmission periods. In addition to that, a typical user might be required to master the skills of operating a large number of software systems like those used with various data 2 processors, the varied communication software, software installations procedures, etc.

The system administrator needs to install the different types of application software in large numbers of computers, and update this softwdre, in each computer, every time a ooooo new version is used. This proliferation of software and hardware in the medical data "0 processing systems make such systems difficult to maintain and a burden to update.

Presently, more and more hospitals and clinics are uniting for economic reasons to form healthcare enterprises with consolidated resources, having a single headquarters for managing the organization. The consolidation of resources also takes place inside :....individual hospitals, with the primary goal of facilitating data exchanges inside the 000hospital, with hospital personnel outside the hospital premises, as well as with other related facilities and with the enterprise headquarters. Generally, most individual facilities that make up the enterprise operate special systems to store and manage various parts of their clinical data. One can generally view these systems as being composed of data acquisition devices, data storage devices (data banks), and data H:\AJM\Keep\peci fict ion \47215.7 d iv. oc 3/08/0 1 3 management and coimrununication modules. The users are connected to the data banks via various networking procedures and communications protocols. These users may operate a variety of computer hardware systems. Access of each user to stored patient data is presently done through the use of special application software on the user's computer. Since presently, most health and medical organizations have constructed their information systems and communications network over a period of time, access to these systems is often complicated, and sometimes requires the user to master several application, software and communication protocols. Typically, no corummon access method or user interface is available to the user, and users are often confined to the use of particular hardware at a specific location to access the data. The need to access image data further complicates the situation. The large bandwidth required from the communications link, the large data volumes, and the special processing that is usually needed, often requires the use of special software and hardware on the user's side.

Thus, one problem encountered with the present server-user communication systers for transferring medical data is the many different interfaces, software applications, and corununication protocols required and the many different types of work stations that make up the "installed base". Due to this proliferation of different work stations requiring different software applications, interfaces and corununications protocols, then whenever a new improved system or a new data type become available, the many different work stations have to be equipped with the software for utilizing the new systems or data. This is not only expensive, but time consuming in that the installation of the software in each of the many different work stations and the central server requires time and usually requires expertise beyond that of the doctor or medical professional using the workstation.

25 A second and equally troublesome problem is encountered when the data requested by the user includes images that must be transmitted over a given enterprise network. This is due to the long time required for transmitting image data as compared to other forms of data. Image compression is used to reduce transmission times. For clinical image data, special precautions must be taken if lossy compression is implemented, due to the potential loss of possibly vital findings. Lossless compression schemes are therefore employed, which provide a relatively small reduction of image transmission time (a factor of 2-3 for radiology images). Interactive compression schemes, that optimize the transmission time for any given user and user type are currently not available in existing healthcare information systems. Such an interactive compression scheme is presented as part of this invention.

The above can be summarized in a conceptual diagram (Fig. An enterprise wide healthcare information system 11 may be conceptually conceived as comprising H:\AJM\KeeD\Spec if ict ions\472 15 'i v.toc J/O/O1 4 several local facilities, 12-14, connected to a central facility 16. Each local facility comprises data sources 17 connected through appropriate interfaces such as interface 18X, to the local network, to which the various local users 19 are connected too. The local network of each local facility is in turn connected to the central facility through another interface (possibly including firewalls and security features). The central facility comprises a similar structure, with the addition of central repositories 23, data bases, and data management tools. This structure of the presently available systems suffers from the problems described above. Thus those skilled in the art are still searching for effective solution to the existing problems.

SUMMARY OF THE INVENTION The present invention provides, therefore, an interactive method for allowing a user to obtain image data for diagnostic purposes from a server having access to stored data, involving: connecting a user's computer to the server over a co-ununication network; requesting specific image data for transmission from the server to the user's computer; progressively transmitting the requested specific data over the network to the user's computer; reconstructing images of progressively improving quality from the progressively received data; S"using the produced improved images of progressively produced quality to decide on processing of the images; and i 'interactively selecting regions of interests in the images based on said progressively improved images, prior to the progressively received data being completely received, •wherein said data transmission continues while said regions are being selected on an image reconstructed from previously transmitted data of a lower quality.

Preferably the image data represents a set of images and wherein using the 30 produced images to decide comprises using the produced images to decide on processing images which have not yet been received.

Preferably the method includes deciding on a termination of transmission of said progressive image data, responsive to a decision based on said reconstructed images.

Preferably the processing comprises converting said image data to image data representing an image with fewer bits per pixel, which fewer bits represent a grey scale component of said image, wherein said converting does not affect the number of pixels in said image.

H:\NJM\Keep\speci f ic., ion s\472 15 .147 rli v.doc )/08/01 5 Preferably the processing comprises selecting a region of interest of said images.

Preferably the processing said images is performed on said image data, at said server, prior to their being transmitted to said user's computer.

In one embodiment the invention provides systems and methods including data distribution servers, such as local server 24 and central server 26 as indicated in Fig. 2.

The concept detailed in Fig. 2 is logically summarized in Fig. 3. The various clinical data acquisition devices and data banks are conceptually grouped into a "data source" block 28. A server 29 is introduced as an intermediate level between the data sources and the users. The introduction of the server, with the appropriate functionality and data handling algorithms, alleviates many of the problems presented above.

While the concept and method introduced here is applicable for the distribution of any type of clinical and non-clinical information, this invention will focus on solving the problem of distributing clinical images over the network, which poses one of the major obstacles in implementing a complete and comprehensive healthcare clinical information system.

BRIEF DESCRIPTION OF THE DRAWINGS In order that the present invention may be more clearly ascertained, a preferred embodiment will now be described, by way of example, with reference to the accompanying drawings, in which: Figure 1 is a conceptual representation of a present prior art enterprise wide •healthcare information system; S.""•Figure 2 adds data distribution servers to the enterprise healthcare information i system to facilitate enterprise wide data transfer; 25 Figure 3 presents a logical representation of the data distribution server concept of figure 2; Figure 4 is a general block diagram of a preferred compression-decompression scheme; Figure 5 is a showing of the reduce and enlarge operations in the pyramidal decomposition; V "Figure 6 is a showing of the pyramidal structure concept; Figures 7a and 7b illustrate the background transmission approach; Figure 8 illustrates the order of transmission; "Figure 9a illustrates a predictor for a single image; 35 Figure 9b illustrates a predictor for a group of images; and •Figure 10 illustrates an example of vector partitioning.

H:\AJM\Keep\pecificaiong\misc\47215.97 div.doc 14/07/03 -6- GENERAL DESCRIPTION OF THE PREFERRED EMBODIMENT 1. System Overview The system consists of a server that has access to data banks and distributes the data on demand. Several users can connect, simultaneously, to the server, over communication lines. In this system the server is also responsible for image preprocessing and for distributing user software. The user's function is to manage the medical image acquisition and processing through the use of an intuitive Man-Machine Interface, a special protocol and the available hardware and communication resources.

A typical medical image acquisition session will start by a simple data request, made by the user, to the system's server communication site. This generic request can be accomplished using any of standard communication protocols and, for example, through an HTTP (Hyper Text Transmission Protocol) connection to the server (which can be designed for access purposes as a Web (WWW) site. There are no requirements on the user's hardware and browser software other than the basic capability to communicate over the chosen communication line and for the browser to support a network computing language such as Java or ActiveX. Using the Web, these requirements will include a link to the Intemet and a standard Web browser as described above. Upon such a request, the server will download, to the user's machine, a network application applet. This network application will serve as the user's application in all future interactions with the server. The network application is a generic, platform independent application written in a suitable network application language such as, but "not limited to, Java or ActiveX. The network language may also be any other software that utilizes the communication capabilities of the user's machine. After a short •authorization and authentication procedure, the user will be presented with an i 25 opportunity to request medical data. The communication can also be accomplished using "dial-up" or other "networking" schemes.

Medical data includes Medical Image Data, throughout this description. It may comprise a number of medical images, of various modalities, which are available for transmission through the server. The user may define the specific medical case of interest through the use of network application queries into the server's database.

i: Selecting the case is done using case identifiers which are usually, but not limited to, textual, image icons, etc. A typical CT (Computerized Tomography) case may contain 50-100 medical images. The actual transmission of the medical image information is accomplished through the use of a compression/decompression algorithm and a 35 powerful client/server protocol. The transmission is relatively fast owing to a smart utilization of the available hardware and network resources and focusing on the needed medical information by providing the user with interim information, thus letting the H:\AJM\Keep\specifications\Misc\47215.97 div.doc 14/07/03 7 user refine the information query parameters during the acquisition process itself The compression-decompression algorithm is basic to the explanation of the user/server acquisition protocol. Therefore, this general description will start with an explanation of the compression-decompression algorithm followed by a discussion of the acquisition protocol and conclude with a more detailed review of the Man-Machine Interface.

2. The Compression-Decompression Algorithm The goal of the compression-decompression algorithm is to achieve maximal compression ratios but at the same time supply the user with visually adequate interim images. The algorithm should also support loss-less as well as lossy interim and final results, be suited to the medical image processing common to these images and as much as possible be asymmetric and easy to implement using the network computing language.

Figure 4 presents an overview of the compression-decompression algorithm for use with the system described. Compression starts by (optional) segmentation (block Al in the figure where the background of the image (if it exists) is separated from the actual image.

Figures 7a and 7b show graphical presentations of such possible background segmentations. The regions denoted A, B, C, and D in figure 7a are background regions. The proposed segmentation bounds the region of actual tissue by a rectangle.

i' •Only the inner part is progressively transmitted. Other methods of segmentation are possible as shown in figure 7b where the actual tissue is shown peripherally bounded by -the dashed lines.

25 The second step (optional) in image coding for compression is a windowing operation (block B 1 in figure where the dynamic range of the input image is reduced to a lower number of bits per pixel. The new number of bits can represent the client's display capabilities or be derived from the communication bandwidth restrictions. The windowing operation could be done, for example, by estimating the average M and the standard deviation S of the image values, and rescaling these values in the range S 2) (M S using the required new number of bits. As an alternative, an improved locally adaptive windowing method can be applied, which estimates the mean and standard deviation locally. Other well known windowing procedures can be used.

Since one of the goals to be accomplished is to supply the user with meaningful 35 interim results, the medical images are sent progressively. This requirement in turn implies that a pyramidal re-structuring of the image is required (block C 1 in figure 4).

The concept of pyramidal decomposition of an image is shown in figure 6, where the H,\AJM\Keep\specificatiofs\Mi8C\47215.97 div.doc 14/07/03 8 two basic operations Reduce and Enlarge are further described in figure 5. The reduce operation revises or decomposes the image by, for example, simply discarding all even rows and columns, creating an array that is a quarter of the original size. The enlarge operation, for example, bilinearly interpolates the image, resulting in an array four times larger. The interpolation process is not limited to a bilinear interpolation. The exact type of interpolation is selected based on the user's computational and display capabilities. The pyramidal structure contains at the zero level one small image with reduced resolution. All the remaining levels contain residual values with increased resolution. The pyramidal decomposition of the image could also be achieved through the use of other pyramidal decomposition algorithms. The pyramidal data structures consist of several versions of the original image. Each version is of different size and nature. The pyramidal information is ordered such that the top of the pyramid is the version of the original image which least resembles the original image. If the pyramid is loss-less the final level of the pyramid is an exact replication of the original image. It is clear that after decompressing a specific level we can reconstruct the image up to that level and get an interim result. This interim result resembles the original image according to the level of the pyramid.

In order to facilitate an efficient coding scheme, further decorrelation of the data is required. This is achieved by spatial and temporal decorrelation operation (block D 1 in figure At this stage, each pixel in the current resolution level is predicted by its spatial casual (already transmitted) neighbours. If groups of images are being coded •••together, temporal neighbours from previous images are used to compute a second predictor, and the best predictor is chosen for each block of pixels. At the end of the i o prediction stage, the residuals are rescanned into a vector. If the user selected only part 25 of the image to be transmitted (ROI Region of Interest) only that part of the residual image is scanned and the ROI parameters are added to the header of the image.

Following is an example of a predictor for a single image (Fig. 9a): max(a, b) c <min(a,b) x f(a, b, c) min(a,b) c max(a,b) La b c otherwise J Using similar reasoning a predictor for a group of images can be effective in case there is correlation between successive images (Fig. 9b): x f b, c, al, bl, cl, xl) The residual vector is partitioned into variable length sub-vectors with a relatively homogeneous probability distribution function (block El in figure The adaptive H:\AJN\Keep\specifications\misc\47215.97 div.doc 14/07/03 9 partitioning is accomplished by estimating the local mean and variance on the vector, and sectioning the vector on high transients. Each sub-vector is then coded using an entropy coder. One example of such coding is a Golomb-Rice code (block Fl in Fig.

An example of a possible partitioning is shown in Fig. The decompression algorithm is basically the compression operations in inverse order. First, a header is obtained, stating whether segmentation and/or windowing operations were applied, the size of the images and their number, the pyramid depth, etc. (block A2 in figure A zeros pyramid is then constructed in order to be filled during the decoding process (block B2 in figure Each sub-sector is decoded using inverse entropy coding, Golomb-Rice code (block C2 in figure and all these sub-blocks are rearranged into matrix form. The spatial/temporal prediction is then computed and added to the residuals (block D2 in figure and the obtained values are loaded into the pyramid (block E2 in figure The pyramid can be restructured to an image at any time during this operation, yielding the obtained image so far.

If segmentation is applied, the background will be transmitted at the end of the transmission of the inner image part. This is for loss-less transmission. For lossy transmission the user can stop the transmission, thus disregarding the background.

Transmitting the background is supported by dividing the background into four parts as indicated in figure 7a or by mapping the image as indicated in figure 7b. Each such part is raster scanned into a vector and the same coding operations presented above apply again, namely, decorrelation, adaptive sectioning, and entropy coding Golomb- Rice coding). Other compression-decompression methods of course can be applied within the scope of this invention.

~If windowing is applied, the received image at the user's location is a lossy 25 representation of the original image. Upon the user's request, the error image (the difference between the original and the windowed image) should be coded and transmitted. This error image is coded using the same methodology as presented for the background transmission decorrelation, adaptive sectioning, and entropy coding Golomb-Rice coding), For image group/series, the order of transmission is as shown in figure 8. First, all the low-resolution levels are sent. At the end of this stage, the client may view all S the required images in an overview form using the basic version of the entire image set.

At the second stage, each of the images is updated by sending the next resolution level.

As soon as a resolution level for a specific image is received, the image can be updated 35 to the next interim version which is better than the current version. After several such steps, the images are obtained in error-less form on the user's display Another option, instead of the loss-less entropy coding described above, may be H.\AJM\Keep\specif ications\Mic\4 725 .97 div.doc 14/07/03 10 implemented by applying the already existing JPEG routines within the browser software. This approach consists of optional (as before) windowing and/or segmentation steps, followed by a pyramidal decomposition of the obtained image.

Each resolution level is then compressed using the lossy JPEG algorithm. At the user, each such level is decompressed accordingly. Since lossy compression- decompression results in deeper compression ratios, and since the decompression routines are written in the computer native language, much shorter waiting periods are obtained at the user's end. As described in detail for the windowing and segmentation operations, one final step of residuals transmission is required in order to support final loss-less representation of the original images at the user's workstation.

In addition to the above described techniques, other methods can be applied within the scope of the present invention. All the above are examples of the invention, which is not limited to those methods.

3. Image Acquisition Process "Stamps" Using the insight gained in the explanations rendered until now, the image acquisition process itself can now be described in greater detail. As presented herein above, the user defines the specific medical case (patient, study, series, images) of interest through the use of network application queries into the server's database.

Selecting the case is done using the case identifiers which are usually, but not limited to, text or image icons. A typical CT case may contain 50-100 medical images. Out of i all these images the goal is to supply the user with the images really needed for the purpose of drawing conclusions (diagnostic, second opinion, etc.) as fast as possible.

i' "Usually, out of the entire image case the user will require only a limited number of 25 images and only a specific region of interest (ROI) in the limited number of images.

Typically these requirements are case dependent and the user cannot decide which images and what part of these are really needed until the images are viewed. The protocol thus should let the user specify these requirements as soon as possible by supplying the user with interim information which will arrive fast and be sufficiently adequate to make these decisions.

Upon selection of the medical image case the server starts to prepare (as an option) a very basic version of the entire medical case. This basic version of the images "will be referred to as "stamps" or icons and will consist of a reduced version, which is visually similar, for every image in the case. The size, in bits, of these "stamps" is 35 small compared with the size of the original images. The entire "stamp" collection is thus a reduced representation of the entire image case. Its size is selected to enable the user to visually select which images are of interest while retaining the small total size.

H.\AJM\Keep\specifications\MiBc\47235.97 div.doc 14/07/03 11 This will assure that the entire reduced representation of the image case will arrive at the user in a relatively short time. Having presented the entire "stamp" collection to the user, the server awaits the user selection of a sub-set of the entire image case. The subset can include the entire case but will typically include only several images. This subset of the image case will be referred to hereinafter as the "image group". In place of the icons, text can be used to describe the images.

After the user selects the image group, the server prepares a pyramidal decomposition for each and every image in the group or volume process for the whole series. If segmentation and/or windowing were selected, the server performs these operations at this stage. It then goes on and performs the rest of the compression chain (D1 through F1 in Fig.4) for the top level of the pyramid. This level is the most reduced version of the image and thus is also the smallest. As an option, the server can utilize the "icons" which have been prepared in the previous stage for this purpose.

Optionally, as a first stage, only the smallest representation of the image is sent from the server to the user. The user receives and displays the images. After, or even during, this recuperation stage the user can select either a smaller sub set of the group images and/or smaller region of interest out of the image space. This serves as a finger query into the entire image data base and is sent from the user to the server over the communication line. If no finer selection is required the user is enabled to specify whether the visual level obtained so far is sufficient, thus ending the image acquisition process. However, if a better visual level is needed the acquisition process is combined to obtain the next level in the pyramid. Alternatively if the user does nothing the next level is sent. As soon as the server gets the request it performs blocks D1 through Fl on °the next level in the pyramid. This is done only to those images which are required and within these images only to that part of the image which is of interest (the ROI). Within .the image group the order of compression and communication is presented in Fig. 8.

The protocol preferably works on a resolution first basis. All the images in the image group may be brought to the same resolution level and only then the server advances to the next resolution level. Other orders of operation can be used without losing the generality of the invention.

The above process is iterated for all resolution levels. The process is stopped S"either when the user indicates that the visual level is adequate or the entire image has •been sent resulting in a perfect, loss-less, replication of the original image on the user's screen.

35 The type of temporal prediction, (block D 1 in Fig. 4) is selected by the server according to the user's computational capabilities.

If segmentation and/or windowing and/or lossy compression was performed on H:\AJM\Keep\specifications\Misc\47215.97 div.doc 14/07/03 12 the images, the user can request the server to complete the images to their loss-less representation. In such a case, the server will compress and transmit the needed information for the user to complete the images to their loss-less version.

At each and every stage the user can choose to broaden the information query requirements, for example, by enlarging the number of images rather than reducing it.

In that case, the server will "backtrack" and send the required information to the user.

When the needed information has arrived and been presented to the user, the user is presented with the option to acquire another medical image case from the server.

4. The Man-Machine-Interface The man-machine interface (MMI) of the user serves as the means by which the user interacts with the system and as a display surface for the medical images. Being a medical images communication network based system, the MMI combines the known and familiar user interface environment of communication software with the tools needed for medical image processing. The goal is to give the user the tools to be part of the described image acquisition process as well as to enable the user to perform tasks regarding the medical image information. The MMI should achieve these goals with minimal to zero intervention or requirements of the user. For that end the entire user software is completely downloaded from the server to the user's machine and for the most part uses part of the communication software already part of the user's machine.

All this is done without any user intervention. This also makes user software updates i' and improvements irrelevant to the end-user. The user software relies heavily on the communication software browser) already installed on the user's machine. This i' enables the user to operate on different machines with different computational and 25 display capabilities. The first task of the user, upon loading the user software into the user's machine is to automatically profile the machine and the network capabilities.

This information is then relayed to the server and is used to select various parameters for the rest of the session. This is done without any user intervention.

The MMI, the user is presented with, contains controls which are part of the image acquisition process as well as typical medical image processing tools. The image acquisition tools include case specification tools, image selection tools, resolution level advancement tools, tools for windowing, zooming, panning, graphics and annotations, CINE, and so on.

At all times, the user has full information as to what part of the entire medical 5 image case is currently being viewed on the user's display screen. This information °"includes, but is not limited to, image number, resolution level, loss-less indicator, region of interest indication, segmentation and window parameters, and so on. By these, the H:\AJM\Keep\pecificatiofl\MiSc\47215.9 7 div.doc 14/07/03 13 system makes sure the user is fully aware of what exactly is being presented at all times.

While the invention has been particularly shown and described with reference to preferred embodiments thereof, it is to be understood by those skilled in the art that various changes may be made in form and details without departing from the spirit and scope of the invention as defined in the appended claims.

Further, any reference herein to prior art is not intended to imply that such prior art forms or formed a part of the common general knowledge.

H,\AJM\Keep\specifications\Misc\47215.97 div.doc 14/07/03

Claims (5)

1. An interactive method for allowing a user to obtain image data for diagnostic purposes from a server having access to stored data, involving: connecting a user's computer to the server over a communication network; requesting specific image data for transmission from the server to the user's computer; progressively transmitting the requested specific data over the network to the user's computer; reconstructing images of progressively improving quality from the progressively received data; using the produced improved images of progressively produced quality to decide on processing of the images; and interactively selecting regions of interests in the images based on said progressively improved images, prior to the progressively received data being completely received, wherein said data transmission continues while said regions are being selected on an image reconstructed from previously transmitted data of a lower quality.

2. A method as claimed in claim 1, wherein said image data represents a set of images and wherein using the produced images to decide comprises using the produced images to decide on processing images which have not yet been received. 0 A method as claimed in either claim 1 or 2, comprising deciding on a termination of transmission of said progressive image data, responsive to a decision based on said reconstructed images.

4. A method as claimed in any one of the preceding claims, wherein said processing comprises converting said image data to image data representing an image with fewer bits per pixel, which fewer bits represent a grey scale component of said image, wherein said converting does not affect the number of pixels in said image. A method as claimed in any one of the preceding claims, wherein said processing comprises selecting a region of interest of said images.

6. A method as claimed in any one of the preceding claims, wherein said processing said images is performed on said image data, at said server, prior to their being H:\AJM\Keep\specificationg\MiS\47215.97 div.doc 14/07/03 15 transmitted to said user's computer.

7. An interactive method for allowing a user to obtain image data for diagnostic purposes from a server having access to stored data substantially as hereinbefore described with reference to figures 2 to 10 of the accompanying drawings. Dated this 15th day of July 2003 ALGOTEC SYSTEMS LTD. By their Patent Attorneys GRIFFITH HACK Fellows Institute of Patent and Trade Mark Attorneys of Australia **o \\melb_files\homeS\AJM\Keep\specifications\Misc\47215.97 div.doc 15/07/03