JP2009053701A - Method and apparatus for creating virtual microscope slide - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a novel method and apparatus which construct digitally scanned images, and store the digitally scanned images in a tiled format convenient for viewing the images without a microscope, and are improved for transferring the tiled images of a plurality of magnifications in such a manner that the images can be viewed by other users in remote locations. <P>SOLUTION: The creation of the virtual microscope slide is performed by capturing a plurality of the low-magnification images of a specimen using the computer-controlled microscope (10), tiling the images, and creating the reconstructed macro images (24). A plurality of the images of the higher magnification are also captured and tiled, and thereby the micro images (26) are created. Then, the macro images (26) and the micro images (24) are thereafter stored together with mapping coordinates of the images in order to interactively view the images. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  This application is a continuation-in-part of US patent application Ser. No. 08 / 805,856 filed Mar. 3, 1997, which is hereby numbered and made a part of this specification.

  The present invention acquires and builds a tiled digital image from a sample on a support, such as a microscope slide, and transfers the image for viewing by other users at a local or remote location And device.

  Image analysis and quantification of DNA from tissue sections disclosed in US Pat. No. 4,741,031, in particular a cell analysis system of the type disclosed in US Pat. In the immunohistochemical analysis, the cancer area to be analyzed was first located at a low magnification, and then the area had to be remembered when the analysis was performed at a high magnification. There is a need and requirement to image an object in a relatively flat plane with high resolution / high magnification and record it digitally.

  The invention described in the above application meets the requirements and requirements of imaging and digitally recording objects in a relatively flat plane at high resolution / high magnification. Today, for example, it is impractical to create an optical image sensor large enough to cover the entire image area of a sample on a microscope slide with the required resolution. This is because the size of the field of view of the enlarged object and the resulting image are limited by the issues of lens size and resolution / magnification. Watching through a microscope is like watching through a telescope, like a human seeing a very narrow field of view at a low magnification of 1.25x. While using a microscope, pathologists often scan the slides to imagine the overall view or get a hint about the structure of the sample, which is important for the examination of the sample. Remember the overall position of the strips. These are usually diseased parts, for example, malignant parts or potentially malignant parts of a sample. In order to obtain higher resolution and higher magnification for these suspicious parts, the pathologist switches to a higher magnification objective lens, which further narrows the field of view. A pathologist often does this by switching back and forth between a wide objective lens with a low-power field of view and a narrow field of view with a high-power lens, making himself familiar with the sample, and details of the suspicious area of the sample. Has won a high-resolution view. Thus, the user cannot obtain a magnified and compressed overall view of the sample or part of the sample, but must remember a series of views acquired at low magnification. Similarly, at high resolution and magnification, the user cannot receive or view a collection of adjacent images and must have these consecutive images related in his head.

  A similar problem exists on the Internet and Intranet because the pathologist will receive an enlarged image of a single field of view taken from the sample through the Internet or Intranet on his browser. There is a need to provide pathologists with instructions for coordinating high and low resolution views.

  The number of views available to the pathologist is very limited and you cannot select another view or scroll to a view that is close to the area of interest to you.

  According to the method and apparatus disclosed by the above prior application, an image view of the entire sample, digitized entirely at low magnification, or a selected portion of the sample on the slide, e.g. a base layer of a tissue. Humans can build. From the overall low-magnification digitized image, the user can understand where he is currently looking and where he needs to look. That is, the low-resolution overall view is color-coded overall, giving experienced users a visual overall or thumbnail view of the slide, and malignant tumors appearing at a location on the sample image being viewed and The area of interest for other diseases is indicated. This low resolution overall view allows the user to select the point on the view that needs to be viewed at high magnification.

  This overall view is obtained by a large number of low magnification images of the sample with a microscope scanning system, and then each of these smaller views or images (hereinafter referred to as image tiles) is one consistent low magnification macro image of the sample. It was built by harmonizing as a combination. The digitized macro image is reduced to a smaller size by the software system for display on a local screen or transfer to a remote viewing screen via a low bandwidth channel or a high bandwidth channel.

  The above prior application should combine multiple image tiles of a macro image, for example 35 image tiles, and then obtain a series of other tiles of higher magnification that will also be viewed by the user. Teaching Kika. To achieve this goal, the cursor is selected in order to display the selected higher magnification digitized image on the screen for viewing by the user by selecting the relevant region defined and a simple command. Such a marker is prepared for the user. The higher magnification image may be one of several magnifications or resolutions such as 10x, 20x, 40x.

  As disclosed in the above application, it is preferred that a user, such as a pathologist, can quickly switch between a high resolution micro image and a low resolution macro image, or can provide a live separate screen. By doing so, the pathologist is presented with an overall macro view and the pathologist is presented with a marker indicating where the current higher magnification view is located. Since there are a plurality of magnifications, the user can switch to the intermediate magnification as in the case of switching between the intermediate objective lenses. This gives the pathologist a view corresponding to switching between objectives in the microscope, that is, a procedure in which most pathologists are familiar and trained.

  Furthermore, the above application is not limited to a pathologist only viewing a complete tile view, but allows the user to view the adjacent image material from adjacent neighboring tile images, and the user can zoom in on the screen. The user is also provided with a scrolling function that allows moving to a display screen adjacent to the image.

  The above patent application discloses transmitting low-magnification images through various servers and computers via a local area network (LAN) or the Internet. The transmitted tiled image is obtained by using a fully computer controlled microscope, such that the user does not have to digitize and store the entire sample so that the user can place the sample region in the base region, such as the base region. It is possible to move along or to other suspicious points that spread throughout the sample to obtain a tiled image of the selected area. As disclosed in the preferred embodiment of the above application, the Internet browser remote control automated microscope can be used from a remote location by a pathologist to view the reconstructed macro image tile, and the pathologist By operating the microscope using an intranet browser or the Internet browser, a single image can be obtained at a higher magnification if necessary. Some users can see a particular digitized image obtained by a particular pathologist and transmitted over the Internet, and some users can still view the stored image, There have been problems with control over the operation of the microscope by each user viewing the digitized image and with obtaining and transmitting a larger image area with higher magnification using the tiling method.

  As already described in detail, the current archive state of digital images obtained with a microscope is often retained by photographs or videotapes. Pictures, like videotapes, are difficult to use, especially when the user wants to quickly move back and forth between different images or scroll through various adjacent parts of a sample image. In addition, the current archiving method does not provide an overall macro image of the sample, which allows the user to know exactly where the particular high-resolution view was obtained when analyzing the high-resolution image. There is nothing.

  While digitized images can be stored magnetically or digitized and recorded on a variety of recording media, current archiving systems allow users to switch between high and low magnification images. Or, it is not possible to switch between different images at different magnifications so that a pathologist can switch the microscope objective lens in real time to obtain a macro image and a micro image from the same position on the sample. Until now, the use of microscopes has been relatively limited in the field of pathology and has been limited to pathologists who need to use a microscope to examine a particular sample.

  There is a need for a dynamic system in which one or more pathologists, including consultant pathologists, can view the same area at the same time and interact with each other during diagnosis or analysis. You can also store images from samples, and pathologists can easily inspect the images at a later date by simply using the intranet or Internet browser to access the specific website where the images are located. If so, it would be most desirable.

  Of course, to allow Intranet users or the Internet users to view useful low-resolution macro images and high-resolution micro images of several adjacent first microscope images on their respective monitors. Many problems need to be solved. One of the first questions is how to combine adjacent tile images to form a seamless overall view of these tiles. To date, attempts to combine tiles have combined pixels at tile boundaries using software, and such attempts have largely failed. Another problem is not the mapping of the coordinates of the microscope stage holding the slide, usually starting from the X and Y coordinates, followed by a mapping for only one magnification, but typically 1.25 The problem of the mapping of coordinates on the scanning screen which is also the mapping to coordinate the mapping for each multiple resolution image obtained at x10, x40, x40 or more. These coordinates must be maintained for multiple tiled images of one macro image, for example 40 tiled images for one macro image. In order for a remote user to view these tile images and switch between tile images of different resolutions, the user's computer and monitor must not only receive the address and stored parameters for each pixel, but also You must also run on the generic viewing program.

  Another problem with obtaining image tiles and transmitting them over low bandwidth Internet channels is that the storage requirements on the server, eg, 120MB to 1GB, and the amount of data obtained per slide are high. It is. 120 MB is an area selected by a pathologist without tracing image tiles of the entire sample when tracing at high resolution along the base layer, or only high-resolution traces of suspicious breast cancer that are dispersed in breast cancer. It can be obtained by extracting only the image tiles. Such selective interaction by pathologists, even when building macro- and micro-digitized images using a significantly smaller amount of image tiles compared to capturing the entire image at each of multiple magnifications. The amount of data obtained is a major problem for transmitting to a normal web browser having a limited storage capacity over a narrow bandwidth channel in a reasonable time. Coarse compression techniques can be used, but such techniques cannot be used at the expense of providing a high resolution image that the pathologist must have to diagnose the sample.

  In accordance with the present invention, a digitally scanned image can be constructed from a microscope sample, stored in a tiled format convenient for viewing without a microscope, and viewed by other users at remote locations Thus, an improved novel method and apparatus for transferring tiled images at multiple magnifications is provided. This combines several adjacent initial microscope views at a first magnification to obtain an overall macro image of the sample, and combines several adjacent initial microscope views at a higher magnification to produce a combined data structure. Is done by creating This data structure can then be transferred to a remote viewer to provide the viewer with multiple resolution macro and micro images of the area on the slide sample. The data structure is constructed by digitally scanning and storing a low magnification image containing mapping coordinates and similarly digitally scanning and storing a higher magnification image containing mapping coordinates. In addition, pathologists can digitally scan and interactively select only diagnostically important areas of the stored sample, significantly reducing the number of image pixels stored at high resolution.

  The data structure can be transmitted over the Internet or an intranet, allowing multiple users to each diagnose on a particular microscope using a user's own virtual image of the sample. Each of these users can switch between images of different resolutions in the same way as shifting the objective lens relative to each other to obtain different resolution views. However, the preferred embodiment of the present invention provides an overall indication to the remote user where the higher resolution image is located on the sample so that the user does not have to store the position of the higher resolution image. A marker on a simple macro view. Unlike the single small optical view currently available, the remote user is given a series of abutted tiled images, each image approximately equal to one small optical view.

  Thus, remote users are provided with macro and micro tiled images that are better and larger than a single small optical view obtained at the same magnification as a single tiled image.

  The preferred data structure also includes a general purpose display program that allows the remote user to process and interpret the tiled image on the user's browser. This generic display program incorporates its own interactive program that can be used with various computers, browsers, and monitors. The data structure uses selectively compressed data to reduce the resulting large amount of data, eg 120 MB, to a small amount of data, eg 1.4 MB.

  Such smaller and more manageable amounts of data can be transmitted over low bandwidth channels such as the Internet without compromising the resolution that hinders remote pathologist analysis. In addition, this interactive program allows the pathologist to scroll and view nearby image areas of nearby image tiles that the pathologist could not obtain until the invention described in the above application and this application. Enable.

  As can be seen in greater detail with respect to aspects of the present invention, multiple images are possible that are tileable of samples on a microscope slide (ie, continuous images that can be seamlessly abutted with each other to reproduce the original image at various magnifications). The problem with obtaining is solved by the system of the present invention. The system includes a microscope and a stage, and the digital location on the stage is predetermined according to an electromechanically addressable coordinate system (for convenience, XY). Each point on the stage is assigned X and Y coordinates that uniquely define its location. Each increment in the X and Y directions is established for a predefined amount, for example, a 0.1 μm increment. The key factor in achieving a superior resolution of the sample image at a higher magnification is to establish more physical increments on the stage for each pixel of the image sensor and intended display. For example, using the bitmap addressed scrollable image method described herein, at a magnification of 1.25, 64 points on the stage correspond to one pixel on the CCD optical sensor. This pixel corresponds to one pixel on a 640 × 480 monitor (for VGA display).

  Once a coordinate system is defined for the microscope stage and the sample on the microscope slide is placed on the stage, each feature of interest on the slide can be uniquely positioned with respect to the stage. The image is then digitally scanned using a microscope system. The first scan is performed at a relatively low magnification. This is because this image is used to obtain a “macro” image of the entire sample. In the preferred embodiment, a magnification of 1.25 times is used. The microscope system then scans the slide using a 1.25X objective. Since the image is detected by a rectangular optical sensor, such as an optical sensor in the CCD grid, the stage is moved in relatively large increments, and the adjacent physical part of the slide is precisely imaged on the CCD sensor. Must be placed exactly in the area.

  Although the moving area is relatively large, the accuracy must be high to allow alignment of each image portion within the pixel resolution of the CCD sensor. For example, at a magnification of 1.25, 48143 X steps and 35800 Y steps are required to move a sample object on the stage to a new continuous region for optical imaging on a CCD sensor. is necessary. In this case, the signal generated by the optical sensor in the CCD grid is transmitted to the computer, which stores the image signal as a series of tiled images. Since each image frame is defined by a predetermined XY coordinate, these images can easily be converted into a series of consecutive tiled images.

  When a computer displays a scanned digital image on a monitor, it uses a method of reserving an image bitmap that corresponds to the overall size of the tiled image, for example 10x8 in this example. Get a 25x tiled image. In this case, when a 752 × 480 pixel CCD sensor is used, an image bitmap having a size of 7520 × 3840 is required. For each image tile, the XY coordinates are known for each pixel in each tile, so a bitmap is used to coordinate and display the stored image tiles, screen pixels and image pixels one by one Corresponding macro views of images can be displayed. Normally, the XY size of the screen pixels is smaller than the macro tiled image (ie, the entire image cannot be viewed on a monitor without some sort of image compression), in which case the macro tiled image Scrolls over the displayable window segments and maintains a one-to-one correspondence. The advantage of a one-to-one correspondence is that the user can use significant image details. In addition, since the physical X, Y position on the sample is known via the stage coordinate relationship with the image pixel, the tiled macro image is used to locate the region and this from the higher magnification collection of tiled images. The stage can be moved to the area.

  Because of the nature of the optical system or lens, the image obtained by the lens is a high resolution that is centered in the optical image because the central area of the image is clear and the periphery of the image is a fuzzy, almost circular image. It is configured to move stepwise to various positions on the slide so as to scan only the image portion. Fuzzy marginal areas are discarded. This also has the benefit of ensuring a high resolution image once the tiled image is reconstructed so that the user can view it on the monitor.

  After the macro image is completed, a trained expert, such as an advisor pathologist, views the sample image by looking at the macro image and looking for a region of interest. In general, most sample slides contain only a few small areas that are diagnostically important. The slide balance has little meaning. When an advisory pathologist views the slide, it is possible that areas of interest in some areas that are viewed and analyzed at a higher magnification have already been marked. Once such an area is marked, the microscope is set to the desired high magnification and then the marked area is scanned and stored. Alternatively, the pathologist can define a new area directly on the macro image. In either case, these areas are outlined directly on the display window displaying the macro image using a pointing device such as a mouse. As described above for the 1.25x image, the stage has a predefined coordinate system so that the scanned high magnification image portion can be easily located against the macro image to create a series of micro images. Can do.

  By virtue of the fact that a typical microscope sample slide contains only limited information of interest and the ability of the system implementing the present invention to accurately locate such areas, the system is capable of virtual microscope slides, ie actual samples. You can create a data structure that can be used in place of a slide. Thus, conveniently, multiple users can diagnose a particular sample. Further, since the size of the data structure is reduced, the data structure can be viewed locally on the personal computer and transmitted via the intranet or globally via the Internet. The created data structure can be stored on various storage devices or recording media, for example, a server hard disk, a Jazz drive, a CD-ROM, or the like. Further, storing the data structure on a portable storage medium allows the microscope slide data structure to be transferred and archived by multiple users.

  Another feature of the present invention is a self-executing data structure. This is obtained by packaging the tiled image using an active dynamic control program. When an active dynamic control program is used by a display program such as a general web browser, the browser can interpret the dynamic control program. In this case, the user can interact and control the image from the recording medium that is viewed on the screen by the viewer. Specifically, in the preferred embodiment of the present invention, a number of low-magnification digitized tiled images are formed and incorporated into the data structure, and link information is appropriately tiled during display of these tiled images. A series of higher-magnification tiled images can be constructed as micro-images as well, and a control program such as a JAVA applet can be generated and used by a remote user so that a macro image can be formed. And transferred with the microtiled image. Thus, for example, macro-tiled and micro-tiled images containing active control programs can be transmitted over the Internet or an intranet to a browser or other application program that displays the image, and then the user accesses the browser However, the image can be analyzed using a macro view in front of the user at multiple resolutions. This allows the image to be viewed as if using an optical microscope, but in this case, visually, the view is a view of a multiple resolution virtual microscope slide.

  Further, according to the present invention, the constructed tiled macro image and tiled micro image can be arranged on a web server together with a control program, and these images can be locally distributed via a wide area network (WAN). Access, and possibly globally by multiple users at various times. For example, each micro-tiled and macro-tiled image of a number of already scanned and stored sample slides, such as 300 sample slides, can be placed on the server. The medical or pathology students can then access the slides or all 300 slides, respectively, and examine them on their web browsers when convenient. Similarly, pathologists may dial up or otherwise connect to the Internet or other long-level network through an Internet service provider to access a web server and obtain sample results for a particular patient. it can. Such results are stored as a data structure (including macro-tiled and micro-tiled images with a control interpretation program). The pathologist can then perform the analysis at home or in the office without having to have a microscope, specific slides, or control such slides. The pathologist switches between micro- and macro-images and then directs his analysis, results or diagnosis from these stored images or otherwise prepares them. This conveniently allows pathologists to do some of their work at home or in the office as convenient, and laboratories can use slides for microscopy at remote locations. The actual sample slide can be maintained in a safe location that does not need to be shipped and is not subject to damage.

The control program, in the preferred embodiment of the present invention, is a dynamic self-executing program, such as a JAVA applet, that allows the user to process and interpret the image while on the browser. This dynamic self-executing program is a fully self-contained program with its own display and interpreter program for operation by the browser user. The tiled digitized image and the active control program are stored on a CD-ROM or other portable storage medium and transferred to the user via email or otherwise, and a dedicated viewer at the convenience of the user The present invention is not limited to use on a browser.
Thus, from the foregoing, an improved novel method and apparatus for archiving microscope slide information on a storage medium is provided, along with an active control program that allows for the display and interpretation of various macro and micro images. I understand that
According to another important aspect of the present invention, a self-executable data structure (stored macro and micro images and self-executed programs that display, reconstruct and process stored images), And the ability to scroll the displayed image. This allows the user not only to view an image tile at a specific magnification, but also to move adjacent points using a pointer or other method, and nearby neighboring images that previously could not be viewed The display image of the tile can also be included in the view that the user is viewing. That is, the user can shift the viewing location across a tile boundary from one tile to another, up, down, left, or right, or to another point of interest in a normal two-dimensional scrolling manner. Therefore, the user can display archived and stored slides at a plurality of magnifications, and can easily scroll the slides in an arbitrarily selected direction. Similar to the above-mentioned application, the user moves to various selected areas by interaction, operates the pointer or marker to select a high magnification for displaying a specific attention area, and scrolls the attention vicinity area. You can also.

  Data images can be displayed, reconstructed, and processed using not only Internet browsers but also dynamic self-executable programs such as JAVA applets and ACTIVE-X applets. The advantage of using a dynamic self-executing program linked to the data image on the data structure is that the data image is displayed, reconstructed and processed independently of the operating system (OS) of the user's computer. It is possible to do. In addition, since the latest version of the dynamic self-executable program is already linked with the data image on the data structure or the storage medium and supplied together with this data image, the user does not need to obtain the latest version. Thus, the user can always view the data image regardless of the various program versions.

  This dynamic self-executing program allows the exchange of entire images that simulate the visual effect of changing the objective lens in a normal mechanical optical microscope view. Thus, the user can easily simulate image tracing by switching from one magnification to another, scrolling through portions of the image, and moving the slide under the microscope lens.

  This dynamic self-executing program allows scrolling of the image in the window and display of a reconstructed large view image. The user can select a portion of the image on the large view image using a mouse or other pointing device, and the program displays the selected portion in the other window at the desired magnification.

  A method for constructing a digital image record of a sample on a microscope slide using image tiles includes scanning the image at a first low magnification so that almost all samples are obtained. The sample is then scanned at a second high magnification so that an image of a selected (or all) portion of the sample is obtained. The spatial relationship between the first low magnification image and the second high magnification image is used to reconstruct the image during display. The individual parts or tiles of the scanned image are combined by a dynamic self-executing program to create a digital image area that is significantly larger than the image view obtained individually without tiling.

  The data structure according to the present invention is created by first digitally scanning a desired sample at multiple image magnifications. The scanned image is then stored in a series of consecutive image tiles. The stored image is then linked with the dynamic self-executing program. A software program can be used to create the data structure. The image is preferably first stored as a bitmap file (.bmp). (Storing the resulting image file in bitmap format is different from the bitmap method of creating an image file described herein.) Using an image compression program, the bitmap file is JPEG-encoded. Convert to (.jpg) format. In this case, the required storage space is reduced, and therefore the time that needs to be displayed on the computer is reduced. The person creating the data structure can choose how many details to include in the transformation. A JPEG image can be created, for example, using a compression ratio of 20% to 80% of the original image. The advantage of the JPEG format is that almost empty tiles (tiles with almost white or black space) are compressed into very small files. However, detailed files are not very compressed. The dynamic self-executable program can include a compression algorithm for displaying the entire image or parts thereof on a display window.

  After downloading or installing the data structure onto the storage medium, the user “clicks” on the icon for the self-executing data structure using the mouse when the data image needs to be viewed. This dynamic self-executing program displays an image in a window. Typically, this dynamic self-executing program displays a macro or thumbnail view of the entire sample image at a lower magnification and a smaller window containing a particular image tile or group of tiles at a higher magnification. indicate. This dynamic self-executing program allows the user to select a point using a mouse or other pointing device, or to show an area on the thumbnail view. The selected view is then displayed at a second magnification in a smaller window. The user can move the mouse or pointing device, and the image in the smaller window scrolls according to the selection on the thumbnail view. In this way, this dynamic self-executing program simulates the movement of the microscope slide under the view of the mechanical microscope. However, it should be noted that not all macro images can be displayed on a monitor because there is a one-to-one correspondence between CCD pixels and screen pixels. The user can scroll through the macro image or select a compression function to display the entire macro image in the window.

  Another feature of the self-executing data structure is that when the image is displayed on the display screen, the user can select an image tile or part of the image, or view this part of the image at each selected magnification. Is what you can do. For example, if the data is scanned at magnifications of 1.25, 20, and 40, the user can “click” to view the same tile in place of each of these magnifications.

  FIG. 1 is a block diagram showing a system according to the present invention. This is an example of a virtual microscope slide, ie, a system that creates a correlated data structure and transmits this data structure over an intranet or the Internet.

  FIG. 1 shows a display procedure for displaying an image of a sample on a microscope slide at a plurality of resolutions. The system includes a microscope, which has a digital platform that supports a microscope slide. The digital platform or stage 11 is specially calibrated to include multiple increments in order to align each part of the sample image with high accuracy. In a microscope setup, after calibrating and initial registration of the stage 11, a microscope slide or other substrate with a scan sample is placed on the stage 11.

  An example of creating a breast cancer sample will be described as an example of a virtual microscope slide sample according to the present invention. In the first step of creating a data structure according to the present invention, a macro image of the entire sample (or the portion of the sample that needs to be stored as a macro image) is established. The purpose of creating this macro image or thumbnail image of a large area is to allow the viewer to view the entire sample at once, and to view at a high magnification, the entire image is used on the image. It is to be able to select a significant part. In this example, the magnification selected by the user to display the entire breast cancer slide is 1.25 times. Once the sample 13a is placed on the stage 11, the rotating optical assembly 15 is rotated and the lens 17 corresponding to a magnification of 1.25 times is selected.

  According to the teachings of prior patent applications, the computer controlled microscope is moved and the entire image of the sample 13a is scanned. This focusing system is programmed to step increments that detect / select only the high-resolution region in the center of the view to avoid storing fuzzy regions around the view. In this example, the macro image will be stored as a 10 × 8 array, ie a total of 80 consecutive image tiles, as shown in FIG. 1a.

  A typical microscope slide is about 77 mm x 25 mm, and the usable area without labels is about 57 mm x 25 mm. Each of these 80 image segments is approximately 4.8 mm × 3.5 mm in size. This means that 80 image segments are scanned separately and stored as separate image tiles.

  The accuracy of this microscope system is set up so that the accuracy of each step of the motor is 0.1 μm. In this example, the microscope is set up to move 48,143 steps in the X direction and 35,800 steps in the Y direction at a magnification of 1.25 for each of the 80 image areas. .

At high magnification, the scan image area is quite small, and therefore the number of steps is reduced depending on the size of the scan image area. Of these 80 image areas, only the high-resolution area at the center of the view is detected by the microscope lens.
The optical image of the desired image area is then detected by an optical array sensor 19 (preferably a CCD sensor array). In this example, these 80 scanned areas are each detected across the entire optical array, which includes 752 pixels by 480 pixels. The optical array sensor transmits an electrical signal representing the detected image to the microscope control computer 32. Computer 32 stores the scanned image. This scanned image contains the upper left XY stage coordinates for 80 individual areas of the microscope slide. Each of the 80 scanned image area pixel locations is stored in a bitmap file (ie, a file containing a map of the location of each bit in the area). This bitmap file corresponds to the layout of the individual images on the image area. Thus, a unique location in the computer memory bitmap file (FIG. 6) is individually assigned to all pixels of the image tile taken from region A of FIG. 1A, ie, the seventh region from the left of the top row. Assigned. All pixels of this retrieved image tile are also stored in the data structure image tile file, as shown in FIG. 1B.

  Each of these stored data image tiles is a standard image file having an extension of .bmp and is on the order of 1 MB. That is, 752 × 480 pixels are stored as 3-byte red, green, and blue image data (752 × 480 × 32 = 11,082,880 bytes). Since the position of each image tile is known by the bitmap, the complete microscopic image can be reproduced by painting each image tile according to the grid position of each image tile. Please note this.

  To display the resulting image, the computer 32 calculates the appropriate portion to be displayed from each image tile, depending on the relative size of the display screen. Since the stored image data is usually larger than the typical monitor size, the viewer must scroll the image on the window when viewing the entire image data. However, an optional compression algorithm can be used to compress the entire image into a viewing window. The XY coordinate information is used by the viewing processing program to reconstruct the image tiles into a complete image of the sample. The resulting image is larger than the image obtained by a single lens and has a good resolution if a single lens capable of viewing the entire sample in one view is constructed by optical technology. In this example, each of the 80 image tiles has a digital resolution of 752 × 480 pixels, and the corresponding optical resolution ranges from about 0.2 μ at 40 times to about 6.4 μ at 1.25 times.

  After the macro image or thumbnail is digitally scanned and stored with its XY coordinate information, the user checks if there is significant detail in the macro image or the first sample. Usually, an area viewed at a high magnification is highlighted by a user with a marking pen. Next, the user changes the magnification of the optical system 15 to a desired high magnification, moves the scanning system, and views the selected area. The computer 32 then repeats the process of scanning and creating image tiles for the selected area, which is performed at a high magnification and using a new grid system to locate the scanned and selected area. Is called.

  In this example, the user has selected region B of FIG. 1a to perform the second view at a high magnification. For example, the user selects a magnification of 40 times. The computer 32 calculates the number of tiles to cover the selected area at a magnification of 40 and sets up a second grid.

Note that region B of FIG. 1a intersects with some of the larger tiles in FIG. 1a. The accuracy of this instrument is extremely good, i.e. 0.
Since it is 1μ, it is easy to locate such a selected region with high resolution. As already indicated, the computer 32 calculates the size of the image portion, in this example, for example, X = 1500 step increments and Y = 1200 step increments. Each image portion at 40 × resolution is detected by an optical sensor array of 752 × 480 pixels. Each data file obtained is a high magnification of the memory so that the computer 32 can easily recall the location of region B or any of the 200 individual image tiles as requested by the user. Stored separately in the map area.

  Once the user has selected a digital image in the image tile and completed scanning and storing this image in a computer controlled microscope system, the computer 32 stores the mapped .bmp file with its coordinate information, The slide image data structure 31 of FIG. 1 is created. The slide image data structure includes all bitmap image tile files at both magnifications (also note that additional images can be stored at other magnifications as needed), as well as various Contains information about the location of the image tile.

  FIG. 7a is a file listing as observed under the Windows 95 file manager showing the data files included in the data structure for breast cancer. This file listing includes FinalScan.ini and SlideScan.ini, and similarly includes 60 bitmap data files. SlideScan.ini is a listing of all original bitmap (.bmp) files. These bitmap files represent individual image tiles that are scanned, for example, at a magnification of 1.25. SlideScan.ini is clarified in the following Table 1, and describes the XY coordinates for each image tile file. When the data structure for breast cancer is viewed by the control program, the control program uses XY coordinates to display all image tiles sequentially.

  The computer 32 can also use the scanned image file to create a self-executing data structure. By compressing the .bmp image into a .jpg and adding a dynamic self-executing program that allows the user to view, reconstruct and process the image tiles, the user can make this data structure a virtual sample of the original sample. Can be used as a microscope slide. This dynamic self-executing program is preferably a Java applet as shown in FIG. 7b.

  The computer 32 can supply the slide image data structure 31 to the local viewer 34 directly or via the intranet browser 33 or via the Internet server 38. As shown in FIG. 1, the slide image data structure 37 can be directly accessed from the Internet server 38. Alternatively, the user can download the slide image data structure to his computer 39 and use the Internet browser 43 to view the reconstructed image. Alternatively, computer 32 can store the slide image data structure on a CD-ROM, jazz drive, or other storage medium.

  In order to view the slide image data structure 31 or 37, for example, a user who has acquired the data structure by using a CD-ROM first installs the CD-ROM in the CD-ROM drive of his computer. The user then opens a browser or other application program that can read the Java applet installed with the image tile on this CD-ROM. Note that there is no need for a separate browser program. In some cases, the CD-ROM may include a complete application program that displays, reconstructs and processes image tiles. In this example, the user selects icon listing or file listing for the slide image data structure, and the control program displays the data file.

  FIG. 2 shows a screen of a system embodying the present invention. This screen shows a low magnification image 24 of the sample on the microscope slide in one window, and shows a high magnification image 26 of the portion of the low magnification image selected by the region marker 30 for control. A window 28 is shown. FIG. 3 is a view of a display screen of a device embodying the present invention. The display screen shows a control window 28 and shows a low magnification window 24 having a plurality of high magnification micro image areas 310 depicted in the low magnification window 24 and one or more micro image areas 310. A high magnification window 26 including 314, 316 is shown. FIG. 4 is a macro image of an actual breast cancer sample, displayed at 1.25 times as observed on a computer monitor. FIG. 5 is a view of the grid portion of FIG. 4 and shows a region selected by a pathologist displayed at a magnification of 40 times.

  Recall that region A in FIG. 1 a was about 4.8 mm × 3.5 mm. This area generates detection data of 752 × 480 pixels, that is, information of 360,930 pixels. Each pixel sends information about each location and the image detected by this region to the computer. The computer stores this information in a series of data files (usually in .bmp format, but can also use .tif or .gif). Thus, it can be seen that on a computer monitor operating at 640 × 480, the detection data of many times more pixels can be used. When viewing the entire image, the user must scroll the image tiles. However, scrolling does not have to be done on each tile. Scrolling is performed by the user pointing to a pixel on the monitor.

  FIG. 6 is a block diagram showing how the control program locates and scrolls stored image tiles. Using the example of FIG. 1a, a complete data structure was created. When the user loads the data structure (of the microscope slide) into his personal computer or views it from an Internet browser, the control program recreates a bitmap of the stored data. FIG. 6 shows a bitmap of the entire slide. Image tile A is also highlighted. According to this bitmap, the user can point to a position on the slide or refer to it by other methods.

  According to the XY coordinate information specified in the data structure, XY conversion between a specific image tile and a specific pixel in the image tile can be performed.

  When the control program first loads an image, the loaded image file is so large that the active window on the user's monitor displays only a few available tiles. The user uses his mouse or pointing device to scroll through the active window and view the entire macro image. The XY coordinate information selected by the mouse is converted into a specific image tile or a part thereof. The computer retrieves the mouse pointer information, retrieves image data from a series of stored tile images, and displays it on the monitor for viewing by the user.

  Since a large amount of CCD pixel information is stored, the actual pixel information can be recreated in the display window. The entire system operates in a loop where the user enters a mouse position and the computer converts the mouse position from screen coordinates (screen pixels) to XY coordinates on the bitmap.

  Similarly, the user can select a high magnification data image. This data image is shown as a black grid showing the stored area. The user operates the mouse in the same manner as described above. The control program locates the stored XY coordinates and retrieves selected portions of the image for each CCD storage pixel.

  As already described, to save storage space, the computer 32 can perform data compression on each image tile file. The preferred data compression is JPEG, which is easily transferred and recognized by most Internet browser programs. According to JPEG, the amount of data to be compressed can be increased from 20% to 80%. FIG. 8 is a file listing that would be observed under the Windows 95 file manager, showing the data files included in the alternative data structure. In this data structure, the data file for the breast cancer sample is compressed or converted to JPEG (.jpg) format. The file index.html (shown in Table 3) is a listing including XY coordinate information regarding these data files. This is information that is read by a dynamic self-executing program to view, reconstruct and process image tiles as macro and micro images.

  Drawings, particularly FIGS. 9A, 9B, and 10, will be described. In these figures, an apparatus for synthesizing a low-magnification microscope image and a high-magnification microscope image is shown, and this apparatus is denoted by reference numeral 10. The system includes a computer 12 which is a dual Pentium Pro personal computer, which is combined with a Hitachi HV-C20 video camera 14 associated with a Zeiss Axioplan 2 microscope 16. The camera 14 captures light from a microscope 16 having a microscope slide 16 positioned on a LUDL encoded motorized stage 20 so that the computer system 12 can receive signals from the camera 14. The coded motorized stage 20 includes a MAC2000 stage controller that controls the stage in response to the computer 12. The microscope slide 18 includes a biological sample 21 viewed through a microscope, and the image of the biological sample is digitized at both a low magnification and a high magnification selected by the user. This low-magnification digitized image is displayed on a 21-inch IIyama video display monitor 22 with a resolution of 1600 × 1200. For example, a low-magnification image 24 of 1.25 times and a high-magnification image 26 of 40 times, for example. A display screen of the type shown in FIGS. 1-3 and a control window or image 28 are included. According to this low magnification image, the region 30 that is reproduced at high magnification in the high magnification screen or window 26 may have been identified in this low magnification image. Once region identification is performed within the low magnification image, the pathologist or other operator of the system considers the structural region with the low magnification image 24 and at the same time places the structural region within the high magnification screen or window 26. It can be determined whether it is necessary to further inspect whether the cells that display the magnification and form part of the architectural feature are cancerous.

  The computer 10 is built around a PCI system bus 40 and has a first Pentium Pro microprocessor 42 and a second Pentium Pro microprocessor 44 connected to the PCI system bus. A PCI bus 50 and an ISA bus 52 are connected to the system bus 40. A SCSI controller 60 is connected to the PCI bus 50 for transmitting and receiving information to and from the hard disk 62. The hard disk 62 is coupled to a high capacity removable disk and CD-ROM drive 66 by a daisy chain SCSI. A program stored on the hard disk 62 controls the microscope 16 to process the image and operate the system to perform a quantitative analysis of selected portions of the tissue sample viewed on the slide 18. belongs to. A RAM 70 and a ROM 72 are connected to the system bus 40. The RAM 70 stores a part of the program being executed. The ROM 72 is for holding a bootstrap loader, and similarly for holding a part of a basic input / output operation system (BIOS). A floppy disk controller 74 is coupled to the system bus 40 and is connected to a floppy disk drive 76. The floppy disk controller 74 reads information from the floppy disk as necessary and writes information to the floppy disk. A mouse controller 80 is coupled to the system bus 40 and has a mouse. The mouse operates as a pointing device that controls processing on the screen 22 and in the windows 24, 26, and 28. A keyboard controller 90 is connected to the system bus 40, and a keyboard 92 is connected to the keyboard controller 90. The keyboard 92 can be used to send and receive alphanumeric signals to and from other parts of the computer.

  A plurality of speakers 102 and one microphone 104 are connected to the audio controller 100 for audio input / output, and the audio controller 100 is connected to the system bus 40. A network interface, such as a network interface card 105, is connected to the system bus 40, which can supply signals via a channel 106 to a network to which the system can be connected or to other parts of the Internet. Similarly, a signal can be transmitted from the system via a modem 110 connected to the ISA bus 52, and a signal from the system can be transmitted to, for example, the Internet via a channel 112. . A printer 116 is connected to the system bus via a parallel I / O controller 118. This is to print out this information as needed as the screen and other information is generated. A serial I / O controller 122 is connected to the system bus 40, and a camera controller 124 coupled to a CCD sensor 126 in the camera is connected to the serial I / O controller 122. The CCD sensor 126 provides a pixel or image signal representing what was found on the slide 18 to the Epix pixci image acquisition controller 130. The Epix pixci image acquisition controller 130 is coupled to the PCI bus 50.

  The microscope 16 includes a base 140, and the stage 20 is supported on the base 140. Similarly, the microscope 16 includes an objective lens turret 142 having a plurality of objective lenses 144, 146, and 148. include. For example, the objective lens 144 may be a 1.25 × objective lens. The objective lens 146 may be a 20 × objective lens. The objective lens 148 may be a 40 × objective lens. Signals from the camera sensor and controller are supplied to the image acquisition system via bus 128, digitized by the image acquisition system, supplied to the PCI bus, stored in RAM, or backed up to hard disk 62.

  When there is a sample on the slide 18, the stage 20 can be processed by a stage controller 160 coupled to the serial I / O controller 122 under computer control. Similarly, the microscope controller 162 controls the aspect of the microscope, such as the illuminance, color temperature, and spectral output of the lamp 168. For example, in normal operation, once a sample is placed on the slide, the sample slide 18 is placed on the stage 20 at step 200 as shown in FIG. 11, and at step 202 the processor 42 or 44 is connected to the system bus. The command is issued via the serial I / O controller 122 to cause the microscope controller to signal and change the magnification to 1.25 times. This magnification change is performed by rotating the objective lens turret of the Axioplan 2 microscope and selecting the objective lens 144. Similarly, the microscope controller sets the color temperature of the lamp 168, sets a pair of neutral gray filter wheels 170 and 172, and sets the field diaphragm 174 to obtain the correct illuminance. The capacitor diaphragm 176 is also controlled, and the color filter wheel 180 can be controlled to provide the appropriate filter color to the CCD sensor 126 in the camera. Next, at step 204, the entire slide is scanned. The images are tiled and each combined into an overall image 24 that is fed onto the screen 22 and at step 206 a macro image that can be visually inspected in the relevant area of the slide is displayed to the operator. Is done.

  To provide a magnified image, the mouse can be moved to identify marker segments or marker regions. This marker segment or region can be, for example, a rectangular region. By moving the mouse, as in step 208, the turret is rotated and the appropriate objective lens system is brought to the viewing position, so that the magnification of the microscope is changed to 4, 20, 40, or the like.

  Next, at step 209a, the user uses the mouse to select a region on the macro image and select a micro image to be viewed on the screen 22.

  A test is performed at step 209b to determine if the user has commanded to continue the test, and if the user has commanded to continue the test, the test is performed and selected at step 209c. It is determined whether to change the magnification by changing the objective lens. When changing the magnification, control proceeds to step 208. If the magnification is not changed, control proceeds to step 209a. If the inspection is not continued, in step 209d, the selected area is outlined so that it can be scanned at a higher magnification. At step 209e, a command can be received to scan or acquire a higher magnification image for display on the screen 26. The image can then be archived, displayed for later analysis, or analyzed immediately.

  In order to perform the magnification required in step 208, the overall illumination and control of the microscope will be controlled, so that in step 210, a higher magnification objective is positioned on the slide 18. In addition, the objective lens turret 142 is rotated. In step 212, the applied voltage of the lamp is changed to adjust the lamp 168 to provide a predetermined appropriate illuminance and color temperature for the selected objective. In order to provide the proper illumination for the selected objective lens, at step 214 the capacitor diaphragm 176 will have a properly selected aperture as required. In step 216, the filter turret 180 selects an appropriate optical wavelength filter to be supplied to the camera sensor. In particular, when the sample is dirty, for example, a red filter, a blue filter, or a green filter is selected as necessary. At step 218, field diaphragm 174 will change the aperture. At step 220, a neutral gray filter is selected by neutral gray filter wheel 170, and at step 222, a neutral gray filter is also selected by neutral gray filter wheel 172. In step 224, the recorded image is reconstructed at this magnification using the X, Y, and Z offsets, and in step 226, the current position is read from the encoder in the stage with a precision of 0.10μ. Is done.

  In order to identify the selected area, the pointing operation in step 240 shown in FIG. 14 moves the mouse to this area. The mouse can be moved to draw a box around the selected area. In step 242, for the selected region edges, X and Y screen points are calculated and the calculated image or pixel points are converted to stage coordinate points to control the stage of the microscope. . In step 244, a list of all X fields is stored in the RAM and can be backed up on the hard disk to position the stage relative to the objective lens.

  Information from the X offset to the objective lens and the stage offset is used, and similarly the field size is used to properly position the slide below the objective lens to capture the microimage.

  When the slide is properly positioned, as shown in FIG. 15, in step 250, the stage is positioned for each of the X coordinate value and the Y coordinate value in the stage coordinate value, and a digitized image is obtained by the camera. Captured, stored in RAM, and backed up on hard disk. The images can then be analyzed quantitatively in various ways as described in the above-mentioned US patent application. Optionally, at step 254, the image can be stored for archiving.

  In order to override a specific control function as shown in FIG. 12, a screen as shown in FIG. 13 is prepared. In this screen, you can edit the XY step size, you can edit the X, Y, and Z offsets, you can select the ramp voltage, you can select the neutral gray filter and the field diaphragm Apertures and some other microscope functions can be selected. FIG. 13 is a diagram showing the setting of the microscope objective lens of Axioplan 2, that is, a computer-controlled microscope.

  Specifically, the positioning of X and Y is performed as shown in FIG. In FIG. 16, slide 18 is shown with slide boundaries 270, 272, 274, and 276. The stage can move from the upper left corner 278 to the lower right corner 280 due to the stage boundary to limit the stage travel. In the upper left corner 278, a signal indicating that the movement end position has been reached is output, and then the stage is converted into a short distance 282 in the X direction and a short distance 284 in the Y direction, and the reference point 290 in the upper left corner is used as a reference. As a first tile 288 is defined. Since the size of the macro image tile 288 is known, by moving the stage appropriately and measuring the position of the stage from the stage position entered in the counter without performing image processing, the next macro is obtained. The image tile 292 can be arranged to be continuous with the first tile 288. Image tile 288 and image tile 292 can be abutted with one another without substantially overlapping, or can be slightly overlapped so that they overlap by one pixel. One pixel overlap can be ignored as blurring of adjacent edges of image tiles abutting each other. The upper left corner 300 of the tile 292 defines the rest of the tile 292, and other tiles can be defined as well. Microimage tiles can be defined so as to be continuous but hardly overlap so as not to interfere with the composite image. Therefore, the following problems can be avoided. In other words, even if the images are matched or adjacent to each other, the edges of successive image tiles are not blurred so that it can be extended to one frame storage or multiple frame storage digital images. You can avoid the problems you encounter when you have to perform the calculated calculations. Of course, the low magnification image 24 defines a plurality of microimages within the low magnification image 24 and these microimages are tiled and shown at a higher magnification as individual tiles 312, 314, 316, etc. Has been. Moreover, as shown in window 26, the enlarged area 310 may exceed the border of window 26, and therefore window 26 is larger than window 26 by a scroll bar or other means. Any image 310 that can be examined from within window 26 can be included.

  Stage 200 is best shown in FIG. 16A, but stage 200 includes an X stepper motor 279 and a Y stepper motor 282. Each of the X stepper motor 279 and the Y stepper motor 282 has an encoder, and a closed loop system is provided. Thus, in the absence of a closed loop system, the accuracy of most microscope stages is typically 5μ or 6μ, whereas the stage 200 with this closed loop system provides an accuracy of 0.1μ. This closed-loop system and this very high accuracy allow tile images to abut both high- and low-magnification images with little overlapping of the images and the current time-consuming and expensive software. And overlapping and blurring at overlapping edges of adjacent image tiles can be eliminated. Use a precisely positioned stage and use the tiling system described with respect to FIG. 16 where the slide is precisely positioned with respect to the center point CP of the slide and the known position of the point 278 is always derived from the same point. Allows the tiles to be precisely positioned in the horizontal direction and strictly positioned in the vertical direction to reconstruct macro and micro images. Unlike the prior art, this reconstruction is performed without using extensive software processing in order to eliminate horizontal or vertical overlapping image tiles and random orientation of image tiles.

  The present invention also includes a network communication function, such as a function capable of remote observation by coupling to an intranet via a network interface or to the Internet via a modem or other suitable connection. ing. Therefore, once the image is scanned and stored on the hard disk or other storage, the remote user can access the low magnification image, as well as the high magnification image, in both images. To make a determination regarding the tissue properties of the sample.

  The system includes a plurality of networked workstations as other functions, the networked workstations being coupled to the first computer console 12, and the first computer console is a display. It has a screen 22 and is connected to the microscope 14. Satellite workstations 350 and 352 are substantially identical to workstation 12 and individually include computers 354 and 356, which are individually coupled to displays 358 and 360. These devices can be processed by input devices 360 and 362, which can include a keyboard, a mouse, and the like. A system including a workstation 370 having a display 372, a computer 374, and an input device 376 may be connected to the system as a third device. Each of these devices is connected to the computer 12 via a separate network line 380, 382, 384, and their transmission can take place via either a network or something like a network. Each different operator at a physically separate viewing station can locate each area from a view of the entire tissue cross-section with a macro view, and then perform these scans and / or quantitative analyses. Can be labeled. A single operator at the instrument station 12 can view the entire tissue cross section to locate each region. These areas can be labeled. This labeling is for subsequent scanning and / or quantitative analysis at a physically distant viewing station, such as an operating room or individual pathologist sign-out area, for subsequent review. Doing so considers the results of the analysis while maintaining and reviewing the overall macro view of the organization and / or individual images that are stored (which can yield quantitative results). Because. Viewing stations 350, 352, and 370 can comprise desktop computers, laptops, and the like. The network stations 350, 352, and 370 do not require a microscope.

  In other alternative embodiments, remote workstations 400, 402, 404, 406, and 408 can be connected via a server 410 that can be provisioned via a packet switched network. The server 410 may be an HTTP (hypertext transport protocol) based server of the type used for the World Wide Web, or a telnet type server previously used in Internet remote operation applications. Server 410 is in communication with local computer 416 via communication channel 414. A local computer 416 has a display 418 associated with it. Local computer 416 is connected to microscope 420. Remote workstations 400, 402, 404, 406, and 408 can perform the same operations as stations 350, 352, and 370, respectively. However, remote workstations perform this operation from nearby buildings or even from around the world. Therefore, the flexibility is further increased in that other people use the sample that is obtained and observed under the microscope 420. In addition, the stored images can be distributed via the server 410 to the remote servers 400-408 for further analysis and review of the stored images.

  While specific embodiments of the present invention have been illustrated and described, those skilled in the art can naturally make numerous changes and modifications. However, these changes and modifications do not depart from the true spirit and scope of the present invention, even in the claims.

1 is a block diagram of a system according to the present invention that creates a data structure of an image of a sample on a microscope slide and transmits it locally via an intranet or the Internet. FIG. 6 represents a microscope slide arbitrarily assigned to be scanned as 80 tiled images. FIG. 4 is a diagram representing the detection signals of individual pixel sensors in a CCD optical array and a reference data file containing information describing the detection signals after detecting a selected image area to be tiled. FIG. 2 is a screen view of a system implementing the present invention showing a low magnification image of a sample on a microscope slide in a window, a high magnification image of a portion of the low magnification image selected by a region marker, and a control window. is there. A display screen of an apparatus embodying the present invention showing a control window, a low magnification window having a plurality of high magnification micro image areas shown therein, and a high magnification window including one or more micro image areas. FIG. FIG. 3 is a macro image of an actual breast cancer sample displayed at 1.25 times viewed on a computer monitor. FIG. 5 is a diagram of the grid portion of FIG. 4 showing the region selected by the pathologist displayed at a magnification of 40 times. FIG. 5 is a block diagram of steps for mapping a scanned image from an optical sensor array to a computer bitmap in memory and to a display on a user's monitor. FIG. 4 is a diagram showing a file list as observed under the Windows 95 file manager showing data files included in the data structure of a breast cancer sample. Is a file list of a Java applet that controls the data structure. FIG. 4 is a diagram showing a file list as observed under the Windows 95 file manager showing data files included in an alternative data structure for a breast cancer sample. It is a block diagram of the apparatus which implement | achieves this invention. It is a block diagram of the apparatus which implement | achieves this invention. FIG. 10 is a block diagram of a part of the apparatus shown in FIG. 9 showing details of the mechanical configuration of the microscope. It is a flowchart regarding operation | movement of an apparatus. 12 is a detailed flowchart of one step in FIG. FIG. 6 is a diagram showing a display screen showing control parameters to be processed on the display screen. It is a flowchart of a region outline routine. It is a flowchart of a scan analysis routine. It is the schematic of the restriction | limiting of the movement of the microscope stage with respect to an image tile. FIG. 3 is a perspective view of a microscope stage, a step motor, and an encoder that performs closed-loop driving of the motor. 1 is a block diagram of a networked system that allows multiple workstations to access a microscope and operate the microscope locally at each workstation. FIG. FIG. 11 is a diagram of the system described with respect to FIG. 1 is a block diagram of a remote networking system that delivers and accesses diagnostic microscope images and data, ie, virtual microscope slides directly through an HTTP-based server or via a packet network. FIG.

Claims (10)

An apparatus for constructing a data structure representing a virtual microscope slide,
A computer-controlled microscope imaging system that digitally scans an image from a sample on a microscope slide;
A microscope stage for supporting the microscope slide, the microscope stage being movable according to a coordinate system addressable to the objective lens of the microscope;
A program for digitally scanning and storing a first series of digitized images obtained from a portion of a sample on the microscope slide, the program comprising:
Incrementally moving the microscope stage within the coordinate system to store a digital image at each stage of the increment along with the addressable coordinates of the system;
The size of the increase is selected to produce a first series of consecutive image tiles;
The microscope is a first optical magnification that allows to form an overall view;
The program further provides for digitally scanning and storing a second series of digitized images obtained from a portion of the sample on the microscope slide, the program comprising:
Incrementally moving the microscope stage within the same coordinate system to store a digital image at each stage of the increase along with the addressable coordinates of the system at each stage;
The size of the increase is selected to produce a second series of consecutive image tiles;
Comprising the microscope at the second higher optical magnification;
The program further provides the first and second series of digitized and stored images in the data structure and the stored addressable coordinate system to allow a user of the data structure from the sample. Provides a variety of resolution images, so that the images are stitched together, allowing points on the higher magnification image to be easily found relative to the same point on the lower magnification image An apparatus comprising: a program configured as described above.   The apparatus of claim 1, further comprising a data compressor that significantly compresses the digitally scanned image so that it can be transmitted over the Internet.   The apparatus of claim 2, further comprising an internet communication channel connected to the apparatus for transmitting the compressed data to a remote location via an internet channel.   The apparatus of claim 2, wherein the data structure further comprises an active control program usable on a dedicated viewer to manipulate and interpret the stored image.   The apparatus of claim 2, comprising a viewer that allows the user to scroll a portion of an adjacent image into a view.   The apparatus of claim 2, further comprising an address display that displays the coordinates to assist various viewers to identify the same region for analysis and annotation.   A browser for storing a dynamic self-executable program; and a monitor for viewing the image from the data structure and switching the image between a low resolution macro image and a high resolution micro image. The apparatus according to claim 2.   The viewer includes a marker for marking an addressable area on a low resolution macro image so that the addressed area appears in a high resolution micro image on the monitor. 5. The apparatus according to 5.   The apparatus of claim 1, wherein the microscope stage has an increase in stage movement that is greater than a single pixel range of the microscope imaging system.   The apparatus of claim 1, wherein the optical magnification of the image tile is about 0.2 microns at least 40 × magnification.

Images