JP3321198B2 - Microscope still image observation system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system for handling still images, and more particularly to a microscope still image observation system for managing images and displaying a state for assisting an observer in an observation operation.

[0002]

2. Description of the Related Art Conventionally, a microscope still image observation system has
For example, it is used for pathological diagnosis using a microscope TV image of a pathologist.

[0003] When a tissue sample is observed in a pathological diagnosis or biology by a microscope still image observation system, for example, the whole image of a sample such as a slide,
What size, shape, and where on the slide
It is said that it is important to grasp whether a color observation target is present or not for efficient observation without oversight.

[0004] As described above, in order to grasp the entire image of the preparation, it is common to observe the preparation with a naked eye or a magnifying glass before entering the speculum, and copy the preparation with a copy machine or take an instant photograph. Things have also been done.
Further, images of important parts for diagnosis and observation, and parts for which a record is desired to be recorded later are individually recorded in the image recording device.

[0005]

However, the above-mentioned conventional method for grasping the whole image of the preparation is performed by visual observation,
Alternatively, a method according to the method is employed, and when relying on visual observation, it is necessary to store the entire image of the preparation during the microscopy.

[0006] In addition, a copy or an instant photograph can be viewed as needed at hand, but the image quality is poor and the correspondence with the observation image of the microscope cannot be clearly shown. Not very useful. Then, even if the user wants to observe the whole image and decide the position to observe with a microscope, moving the stage to the predetermined position requires skill.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and has as its object to formulate observation means for a pathological observer and incorporate it into a microscope image TV observation system so that an observation operation for the observer can be performed. To help.

[0008]

In order to achieve the above object, a microscope still image observation system according to the present invention includes a microscope means for observing a specimen under a microscope, and a photographing apparatus for photographing the specimen and including an entire image of the specimen. Photographing means for obtaining an observation image; display means for selectively displaying the observation image obtained by the photographing means; frame memory means for capturing image information of the observation image obtained by the photographing means; and display means for displaying on the display means. Pointing means for designating coordinates relating to a desired region and designating a desired magnification for an observation image of the specimen to be displayed on the display means based on information designated by the pointing means. In addition to displaying an index indicating a predetermined rectangular area on the observation image in a superimposed manner, the rectangular area is further divided into a plurality of pieces based on the information on the designation of the coordinates and the designation of the magnification. Control means for automatically displaying an index which is divided into a shape area and further controlling the photographing means and the microscope means to be automatically adjusted based on the information designated by the pointing means; And characterized in that: Further, when the pointing means designates a part of the divided plurality of further rectangular areas, the control means performs control for excluding a part of the designated rectangular area from the observation target. Doing is a further feature. Further, the image capturing information further includes management means for managing coordinate information relating to the predetermined rectangular area, coordinate information relating to a plurality of further rectangular areas obtained by dividing the rectangular area, magnification information, and image capturing information. It is a further feature that the information is managed by a predetermined state flag.

[0009]

That is, in the microscope still image observation system of the present invention, the specimen is observed by the microscope means under the microscope, the specimen is photographed by the photographing means, and an observation image including the whole image of the specimen is obtained. The observation image obtained by the photographing means is selectively displayed, the image information of the observation image obtained by the photographing means is captured by the frame memory means, and the observation image of the specimen displayed on the display means by the pointing means. In response to this, designation of coordinates relating to a desired area and designation of a desired magnification are performed. Based on the information designated by the pointing means, the control means controls the observation image displayed on the display means. A control for displaying an index indicating a predetermined rectangular area in a superimposed manner is performed, and based on the information on the designation of the coordinates and the designation of the magnification, Control is performed to automatically display the split indicators showing the the rectangular area into a plurality of further rectangular areas,
Further, control for automatically adjusting the photographing means and the microscope means is performed based on the information designated by the pointing means. Further, when a part of the divided plurality of further rectangular areas is designated by the pointing means, the control means
A part of the specified rectangular area is excluded from the observation target. Further, the management unit manages coordinate information relating to the predetermined rectangular area, coordinate information relating to a plurality of further rectangular areas obtained by dividing the rectangular area, magnification information, and image capturing information. It is managed by a status flag.

[0010]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram illustrating a configuration of a microscope still image observation system according to one embodiment of the present invention.

As shown in FIG. 1, a controller 1 for performing overall control includes a console display 2, a frame memory 4, an image compression / expansion unit 5, an image storage unit 6, a stage control unit 7, and a pointing unit 8. Each is connected.

The frame memory 4 has a video display 3 and a television (TV) signal switch 11.
Are connected, and the TV signal switch 11 is connected to the TV camera control units 9 and 10, respectively.

Further, the TV camera control units 9 and 10 are provided with TV sets.
The TV cameras 12 and 15 are connected to a microscope 13 and a macro photography stand 16 respectively.
Connected to each other. And the microscope 13
An electric stage 14 for controlling the observation position on the specimen is provided.

In such a configuration, the controller 1 controls the entire system. The console display 2 displays a system state, an operation menu, and the like for an observer who operates the system, and the video display 3 displays an observation image.

Further, the frame memory 4 has an NTS
The C-system TV signal is digitally converted to generate digital image information. The image compression / decompression unit 5 compresses the captured image information, restores the compressed image information, and the image storage unit 6 Save the image information.

The stage controller 7 controls the motorized stage 14 of the microscope 13, and the pointing unit 8 allows the observer to select a menu or specify an image area. A mouse is an example. Further, the TV cameras 12 and 15 are controlled by TV camera controllers 9 and 10, respectively, and obtain observation images on the microscope 13 and the macro photography stand 16.

Note that the console display 2 is used for displaying the system status and the operation menu for the operator of the present system, and the pointing unit 8 is used for selecting menus and giving instructions on the screen. The screen of the console display 2 has its area divided according to the purpose of use, as shown in FIG. Hereinafter, the outline of the operation of the microscope still image observation system of the present embodiment will be described with reference to the flowchart of FIG.

The operation is started (step S100). First, a macro image and a weakly enlarged image of a specimen are taken in, compressed,
Save is performed (step S101). Then, the captured image is observed (step S102), and the image of interest is displayed (step S103). Next, an area to be observed at a higher magnification, a magnification, and an image compression ratio are specified on the display screen (step S104), and the specification information is stored (step S105).

In this way, it is determined whether or not the designation has been completed. If the designation has been completed, the next step S107 is performed.
If the designation has not been completed, go to step S
Returning to 104, the above designation is performed (step S106).

Next, in step S107, the stage is moved to the designated position, the lens is changed to the designated magnification (step S108), and the focus and aperture are adjusted (step S109). ), Image capture, compression, and storage are performed (step S110).

In this way, it is determined whether or not the capturing of all the regions of the sample has been completed. If the capturing has not been completed, the process returns to step S107, and the above operation is repeated. (Step S111).

Then, the captured image is observed (step S112), a decision is made. If the observation is to be continued, the operation returns to step S103, and the above operation is repeated. To terminate the observation, the operation proceeds to the next step S114 (step S114). Step S113). Thus, all the operations are completed (step S114).

The operation of the microscope still image observation system of the present invention has been described above. Next, the above operation is performed in the first to eighth steps while mixing the movements of the screens of the console display 2 and the video display 3. This will be described in more detail separately. First, the input of the whole image of the specimen will be described as a first process.

In the guidance message area of the console display 2, the controller 1 sets the specimen on the macro photography stand, and adjusts the focus and aperture.
Is displayed, and "Ready" is displayed in the selection menu display area.

The observer places the specimen on the macro photography stand 16.
And set the TV signal switch 11 to the TV camera 15
Set to the side. At this time, the frame memory 4 is set to the transmission mode.

Subsequently, the observer observes the entire image of the sample set on the macro photography stand 16 through the video display 3, adjusts the focus and brightness, and then uses the pointing unit 8 to display a “ready” menu. Then, the controller 1 freezes the frame memory 4 and digitizes the image of the entire specimen.

The digitized image is compressed by the image compression / decompression unit 5 and stored in the image storage unit 6. Note that the image number of the whole specimen image is “0”. In this system, the camera lens and the camera height are adjusted so that the field of view of the entire sample image is 24 mm long and 32 mm wide. Since this area is divided into 384 pixels vertically and 512 pixels horizontally and working on the frame memory 4, the pixel interval is 62.5 μm.
m. Next, as a second process, the designation of the microscope initial observation area will be described.

In the guidance message area of the console display 2 of the present system, a message "Please specify the initial observation area of the specimen" is displayed, and on the video display 3 screen, an arrow for pointing is displayed. .

The observer moves the arrow appropriately using the pointing unit 8 on the whole image of the sample frozen in the frame memory 4, and specifies an area to be initially observed with the microscope 13. Specifically, on the display screen as shown in FIG. 4, two vertices A (Xa, Ya) and B (X
(b, Yb) is designated on the screen of the video display 3 using the pointing unit 8. FIG. 5 is a diagram illustrating a state of the entire sample image in which a rectangular area is specified. Next, as a third process, division of the microscope initial observation region and designation of observation will be described.

As shown in FIG. 6, the controller 1 of the present system automatically converts the rectangular area designated by the following formula into an initial observation block determined by the objective lens and the eyepiece used for the initial observation of the microscope in a mesh form. Divided.

A 2 × objective lens and 2.5 × for initial magnification observation
The field of view when using a double eyepiece and a 1/2 inch CCD camera is 2.4 mm long and 3.2 mm wide, corresponding to 38 pixels vertically and 51 pixels horizontally on the whole image of the specimen. The division numbers Px and Py of the mesh division are expressed by the following equation (1). Px = int ((Xb−Xa) / 51) +1 Py = int ((Yb−Ya) / 51) +1 (1) where int (X / Y) is an integer part of a quotient of Y divided by X. Function. As a result, the size Sx,
Sy is expressed by the following equation (2). Sx = int ((Xb−Xa) / Px) +1 Sy = int ((Yb−Ya) / Py) +1 (2) Therefore, the coordinates P (X, Y) of each division point of the mesh are expressed by the following equation ( Indicated by 3). X = Xa + n * Sx (n = 0 to Px) Y = Ya + n * Sy (n = 0 to Py) (3)

As described above, when the division into meshes is completed, the message “Please designate a block that does not require the initial observation of the sample” is displayed in the guidance message area of the console display 2, and is displayed on the screen of the video display 3. Displays an arrow to indicate a block.

As shown in FIG. 7, the observer uses the pointing unit 8 to designate a block that does not require initial observation among the divided blocks. This specification is performed by moving the arrow to the inside of the block and clicking it, and once the specified area is clicked again, the specification can be canceled. Normally, it is useless to take in an image of a block that does not touch the sample at all, and an instruction is given to remove the block from the target of initial observation.

When the block to be initially observed is determined in this way, a schematic diagram of the block is displayed in a map in the image block state display area of the console display 2 in FIG. In the figure, each rectangular block is one-to-one at each position with the rectangular block shown on the video display 3.
It corresponds to. In this state, as shown in FIG. 8, all the rectangular areas on the console display 2 are separated by white lines, and the internal areas thereof are shown in black.

At this time, the controller 1 calculates the coordinates of the center position of the visual field of each pixel block, and creates an image management table. FIG. 9 is a diagram showing a format of the image management table. Note that the image numbers given at the time of creating the image management table are, as shown in FIG.
2, numbered. Then, the image number “0” is assigned to the previously captured whole image of the specimen as a parent of the initial observation image. Next, as a fourth process, the capturing of the whole sample image and the microscope initial observation image will be described.

The controller 1 displays a message "Set the specimen on the microscope and switch the TV signal to the microscope" in the guidance message area of the console display 2 and displays "Ready" in the selection menu display area.
Is displayed. The observer sets the sample on the motorized stage 14 on the microscope side, sets the TV signal switch 11 on the TV camera 12, and clicks "Ready".

The controller 2 moves the motorized stage 14 of the microscope to the center of the visual field of the first microscope initial observation image block, and sets “the objective lens of the microscope to 2 × in the guidance message area of the console display 2.
Please adjust the focus and aperture "and display" Ready "in the selection menu display area. At the same time, the frame memory 4 is set to the transparent mode.

The observer observes the sample image set on the motorized stage 14 of the microscope 13 through the video display 3, adjusts the focus and brightness, and then clicks "Ready".

The controller 1 controls the X-axis and the Y-axis of the motorized stage 14 of the microscope through the stage control unit 7 using the block coordinates specified in the image management table as indices, so that the position corresponding to the visual field of each block is obtained. The specimen is moved, the frame memory 4 is frozen, and the image of the block image of the specimen is digitized.

The digitized image is compressed by the image compression / decompression unit 5 and stored in the image storage unit 6. Then
The controller 1 sets the frame memory 4 to the transmission mode again, and digitizes, compresses, and stores the image by repeating the above operation for all the previously specified image blocks. Hereinafter, this processing is simply referred to as image capture.

At this time, the image block status display on the console display 2 is displayed in different colors according to the status of each image block as shown in FIG. The corresponding status flags and display colors of the image management table are as follows. State flag = 0, color = black: no image is captured. Status flag = 1, color = red: image is being captured. State flag = 2, color = yellow: Image capture completed. Next, as a fifth process, continuous observation of the whole sample image and the microscope initial observation image will be described.

When the capture of the whole sample image and the microscope initial observation image is completed, the controller 1 displays the frame of the image block on the console display 2 as shown in FIG. 12, and then proceeds to the continuous observation step.

Then, the message “Continuous image observation is performed” is displayed in the guidance message display area of the console display 2, and “Next image display”, “Mark”, “Previous image” is displayed in the menu display area. The menu of "image display" is displayed.

On the other hand, on the video display 3, the whole observation image with the mesh is displayed as the first observation image. The image is displayed by decompressing and decompressing the compressed image information read from the image storage device by the image compression / decompression unit 5 and transferring the decompressed image information to the frame memory 4. Hereinafter, this process is simply called image calling. When the observer clicks "display next image" in the pointing unit 8, the next image can be called up and displayed on the video display 3.

The image block status display of the console display 1 corresponding to the image being displayed on the video display 3 is changed to red, and the image block status display of the console display 2 corresponding to the observed image is shown in FIG.
Is changed to white, and the status flag of the corresponding image management table is changed to "4".

When the "mark" is clicked,
The image can be marked for later review. This mark is recorded by setting a flag in the mark field of the image in the image management table, changes the image block status display on the console display 2 to green, and changes the status flag of the corresponding image management table to “5”. Further, when "previous image display" is clicked, the image observed and displayed immediately before can be called on the video display 3 again.
By repeating the “next image display” one after another, the observer can observe all captured images without omission.

Further, in the image block status display on the console display 2, each image block is color-coded according to the status as described below to assist the operator on the observation side. Status flag = 1, color = red: image block under observation status flag = 2, color = blue: received, unobserved image block status flag = 4, color = white: observed image block status flag = 5, color = green : Marked image block When the observation of all the taken images is completed, the video display 3 displays the whole sample with the mesh, which is the parent of all the microscope initial observation images. Next, as a sixth process, an instruction to capture a child image will be described.

When the continuous observation of the previously photographed image is completed, the observer observes the mark image in more detail. The detailed observation using a microscope refers to exchanging the magnification of the objective lens for a higher one and performing an enlarged observation. Hereinafter, the procedure will be described.

In the present system, all images are managed in a parent-child relationship, and the parent of all images is the whole sample image.
Then, in the state where the continuous observation is completed and the whole sample image is displayed, the menu displays “display parent image”, “display child image”, and “end”.

Here, when "display child image" is selected,
When the arrow cursor of the pointing unit 8 is displayed on the video display 3 and the image block previously marked in the seventh step is clicked, the image is called.

A similar effect can be obtained by clicking a green block in the image block display of the console display 2 instead of clicking "display child image". In this state, if "instruction to capture child image" is selected, the child image can be captured and observed by the following operations.

At this time, a guidance message "Please select an objective lens" is displayed on the console display 2, and "10x" is displayed in the menu selection area.
An objective lens type selection menu such as "20x" or "40x" is displayed.

When a desired magnification of the objective lens is selected, the field frame of the child image when the objective lens is used is displayed on the screen of the video display 3, and the position is determined by using the pointing unit 8 on the screen. Can be moved.
When the child image capturing position can be moved to a desired position, the position can be fixed by clicking.

Next, as a seventh process, the capture of a child image and the magnification observation will be described. When the observer determines the capturing position of the child image, the controller 1 operates the stage control unit 7 to move the sample to the center of the objective lens visual field. At the same time, a message "Change the objective lens to XX times and adjust the focus and illumination" is displayed on the console display 2.

After replacing the objective lens, adjusting the focus, and adjusting the illumination, the observer clicks “ready” in the menu display area. By this click, the enlarged image is taken into the frame memory 4, compressed, and stored.

In this way, a child image of the initial observation image is captured, and more detailed enlarged observation can be performed. By repeatedly instructing and capturing the child image, it is possible to obtain a diagnostic finding regarding the specimen.

An image block status display on the console display 2 of an image block having such a child image is shown in FIG.
As shown in FIG. 4, it is represented by brown (the status flag of the image management table = 6), and assists the operation of the observer. next,
The end of the operation will be described as an eighth step.

While observing the child image, the menu displays "display parent image", "display child image", "instruction to capture child image", and "end". At this time, if "end" is selected, the system is ended.

As described above in detail, in the present embodiment, the whole image of the sample can be divided into blocks and the microscope observation can be performed. A block corresponding area display (map) is provided in a similar manner.

Further, when performing a low-magnification microscope observation using the entire sample image, it is possible to instruct whether or not each sample block is to be observed and to designate a child image from the entire sample image. At this time, it was possible to specify both the block map on the console display and the block on the whole image.

Here, the capturing of the whole image of the specimen and the initial observation image of the microscope in the fourth step and the continuous observation of the whole image of the specimen and the initial observation image of the microscope in the fifth step are performed as described below. And image observation can proceed simultaneously.

That is, the controller 1 displays “Please set the specimen on the microscope and switch the TV signal to the microscope” in the guidance message area of the console display 2.
Is displayed, and "Ready" is displayed in the selection menu display area. The observer sets the specimen on the microscope stage, sets the TV signal switch on the TV camera 12, and clicks "Ready".

The controller 1 is a motorized stage 1 of the microscope.
4 is moved to the center of the field of view of the first microscope initial observation image block, and in the guidance message area of the console display 2, "the objective lens of the microscope is set to double,
"Adjust the focus and aperture" is displayed, and "Ready", "Mark", and "Next image" are displayed in the selection menu display area. At the same time, the frame memory 4 is set to the transparent mode.

Then, the observer operates the video display 3
Observes the sample image set on the microscope through, adjusts the focus and brightness, and then clicks “Ready”, the controller 1 freezes the frame memory 4 and the image of the block image of the sample Is digitized. The digitized image is compressed by the image compression / decompression unit 5 and stored in the image storage unit 6.

When the "mark" is clicked,
The image can be marked for later review. This mark is The image is recorded by setting a flag in the mark field of the image in the image management table, the image block status display on the console display 2 is painted green, and the status flag in the corresponding image management table is changed to “5”. When "next image" is clicked, the status flag of the image table on the console display 2 which has been being observed is changed to "4".

Then, even when one of “mark” and “next image” is clicked, the controller 1 uses the block coordinates designated in the image management table as an index.
The X-axis and Y-axis of the motorized stage 14 of the microscope are controlled through the stage control unit 7, the specimen is moved to a position corresponding to the visual field of each block, and the frame memory 4 is changed to the transmission mode. At this time, the image block status display on the console display 2 is displayed in different colors according to the status of each image block, as shown in FIG. The corresponding status flags and display colors of the image management table are as follows. Status flag = 0, color = black: no image captured Status flag = 1, color = red: image capturing status flag = 4, color = white: observed image block status flag = 5, color = green: mark image block

When all of the captured images and the observation of the initial observation image have been completed, the video display 3 displays the entire image of the specimen with the mesh, which is the parent of all the microscope initial observation images.

[0068]

According to the present invention, it is possible to provide a microscope still image observation system that supports the observation work for the observer by formulating the observation means of the pathological observer.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a microscope still image observation system according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating an observation operation of the microscope still image observation system according to the embodiment.

FIG. 3 is a diagram showing screen division of a console display 2 and how to use each area.

FIG. 4 is a diagram showing a captured image of a whole image of a specimen.

FIG. 5 is a diagram showing a designated initial observation area frame.

FIG. 6 is a diagram showing a divided initial observation region.

FIG. 7 is a diagram showing designation of an initial observation block.

FIG. 8 is a diagram showing an image block state map display on the console display 2 when the observation block designation is completed and an explanation display thereof.

FIG. 9 is a diagram showing a format of an image management table.

FIG. 10 is a diagram showing the correspondence between screen display colors and status flags.

FIG. 11 is a diagram for explaining a method of numbering a microscope initial observation image block;

FIG. 12 is a diagram showing an image block state map display on the console display 2 at the time of capturing an initial observation image and a display example thereof.

FIG. 13 is a diagram showing an image block state map display on the console display 2 during continuous observation of the microscope initial observation image and an explanation display thereof.

FIG. 14 is a console display 2 at the time of detailed enlarged observation.
FIG. 5 is a diagram showing an image block state map display and an explanation display thereof.

FIG. 15 is a diagram showing an image block state map display on the console display 2 at the time of continuous capture and initial observation, and an explanatory display thereof.

[Explanation of symbols]

1. Controller, 2. Console display, 3.
Video display, 4 ... Frame memory, 5 ... Image compression / expansion unit, 6 ... Image storage unit, 7 ... Stage control unit, 8 ...
Pointing unit, 9, 10, TV camera control unit, 11
... TV signal switcher, 12 ... TV camera, 13 ... microscope, 14 ... motorized stage, 15 ... TV camera, 16 ... macro photography stand.

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-61-156214 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G02B 19/00-21/00 G02B 21 / 06-21/36

Claims (3)

(57) [Claims] 1. Microscope means for observing a specimen under a microscope, photographing means for photographing the specimen to obtain an observation image including an entire image of the specimen, and selectively displaying the observation image obtained by the photographing means. Display means, frame memory means for taking in image information of the observation image obtained by the photographing means, and designation of coordinates relating to a desired region with respect to the observation image of the specimen displayed on the display means, as desired Pointing means for designating a magnification, and based on the information specified by the pointing means, an index indicating a predetermined rectangular area is superimposed on the observation image displayed on the display means, and the coordinates of the coordinates are displayed. Based on the information related to the designation and the magnification, the rectangular area is automatically divided into a plurality of further rectangular areas, and an index indicating the divided area is displayed. Based on the constant information, it said photographing means and microscopic still image observation system characterized by comprising a control means for controlling to automatically adjust the microscope means. 2. When the pointing means designates a part of the plurality of divided rectangular areas, the control means removes a part of the designated rectangular area from an observation target. The microscope still image observation system according to claim 1, further comprising: 3. The image processing apparatus according to claim 1, further comprising: management means for managing coordinate information relating to the predetermined rectangular area, coordinate information relating to a plurality of further rectangular areas obtained by dividing the rectangular area, magnification information, and image capturing information. The microscope still image observation system according to claim 1, wherein the captured information is managed by a predetermined state flag.