JP3366352B2 - Microscope system - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system mainly used for microscopic observation of medical and biological systems, and particularly to a microscope system for observing in a pathological field, telepathology, teleconsultation, etc. more easily and efficiently. Regarding

[0002]

2. Description of the Related Art Conventionally, for example, Japanese Patent Application Laid-Open No. 64-20449 discloses a technique relating to a "remote pathological diagnosis network", and further a microscope for instructing enlargement of a part of a parent image and capturing a child image. The system technology has already been proposed by the applicant. Then, in these microscope systems, it has been desired to manage and observe still images with parent-child relationships.

Here, the parent-child relationship of images means, as shown in FIG. 21, a wide range of images (a) captured at a low magnification.
And a part of it enlarged and captured at high magnification (b)
Say the relationship. The image captured at the low magnification at this time is the parent image, and the image captured at the high magnification is the child image.

As described above, the observation according to the procedure from the parent image to the child image is very effective in telepathology using a tissue specimen because the entire specimen can be observed on the observation side without oversight.

[0005]

However, if the observation from the parent image to the child image does not make sense, the efficiency of the observation may be reduced. That is, in the case of remote cytodiagnosis, a high-magnification image of a desired site is required and a low-magnification whole image is not required, so that the observation efficiency is reduced.

Also, in the case of a stand-alone system that does not use a communication line, there is no restriction due to the communication speed, so it is not necessary to capture the whole image as a still image and gradually enlarge it at a high magnification, only the image to be saved. Should be taken in. Then, in the case of consultation, it is general to transmit only necessary images.

The present invention has been made in view of the above problems, and an object of the present invention is to flexibly generate a parent-child relationship of a microscope image, and to efficiently perform observation and reproduction after observation.

[0008]

In order to achieve the above object, a microscope system according to a first aspect of the present invention comprises an image capturing means for capturing an observation image of a desired specimen by a microscope objective lens as a still image. , The objective lens
Is a low magnification, the first still image captured by the image capturing means and a part thereof cuts the objective lens.
A second still image captured by increasing the magnification by 換Ru, part of the second still image switching the objective lens
Third stationary captured by increasing the magnification by switching
For images , the image that includes yourself is the parent image,
The first to the third stillness when the image is a child image
And a storage unit for storing the parent-child relationship information generated by the parent-child relationship information generation unit and the parent-child relationship information generation unit for generating the parent-child relationship information between the images. And Further, in the microscope system according to the second aspect of the present invention, the parent-child relationship information generating means automatically generates the parent-child relationship information by comparing the position relationship between the position of the parent image and the child image. It is characterized by doing. Further, in the microscope system according to the third aspect of the present invention, the parent-child relationship information generating means is
The user can input the parent-child relationship information. Further, the microscope system according to the fourth aspect of the present invention is characterized by displaying the parent-child relationship between the captured still images in a tree structure diagram based on the parent-child relationship information stored in the storage means. . Further, the microscope system according to the fifth aspect of the present invention is characterized by displaying the position and area of the child image in the parent image based on the parent-child relationship information stored in the storage means. Sixth of the present invention
The microscope system according to the embodiment of the
Capture the observation image from the mirror objective lens as a still image
If the image capturing means and the objective lens have low magnification,
First still image captured by the image capturing means
And a part of it can switch the objective lens to high magnification.
For the second still image captured at a higher magnification
Te, the first still image parent image, the second still picture image
Is a child image, the parent-child relationship information between these still images
Parent-child relationship information that is generated after each image is captured
Generated by the generation means and the parent-child relationship information generation means
Storage means for storing parent-child relationship information
Is characterized by.

[0009]

In other words, in the microscope system according to the first aspect of the present invention, the desired sample is microscopically examined by the image capturing means.
Observation image by the mirror objective lens is taken as a still image, the parent-child relationship information generating means, a first still image captured by the image capturing means, a part of the previous
The second still image captured by increasing the magnification by switching the objective lens and a part of the second still image
Increase the magnification by switching the objective lens
About the 3rd still image that was embedded , parent the image containing yourself
Image, the above when the image contained in oneself is a child image
First to third parent-child relationship information of the still image each other, is generated after each image is captured and by Riki憶in the storage means. Further, in the microscope system according to the second aspect of the present invention, the parent-child relationship information generating means automatically generates the parent-child relationship information by comparing the position relationship between the parent image and the child image. To be done. The third aspect of the present invention
In the microscope system according to this aspect, the parent-child relationship information generating means is configured such that the user can input the parent-child relationship information. Furthermore, in the microscope system according to the fourth aspect of the present invention, the parent-child relationship between the captured still images is displayed in a tree structure diagram based on the parent-child relationship information stored in the storage means. Further, in the microscope system according to the fifth aspect of the present invention, the position and area of the child image are displayed in the parent image based on the parent-child relationship information stored in the storage means. Microscope system according to sixth embodiment of the present invention
In this case, the desired sample microscopic
The image observed by the mirror objective lens is captured as a still image
The objective lens is lowered by the parent-child relationship information generating means.
In the case of double,
1 still image and a part of it cuts the objective lens at high magnification.
Second stationary captured by increasing the magnification by switching
Regarding the image, the first still image is the parent image and the second still image is the second image.
When the still images of the
Child relationship information is generated and recorded after each image is captured.
It is stored by storage means.

[0010]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a diagram showing the configuration of a microscope system according to a first embodiment of the present invention. As shown in the figure,
In the microscope system 100, a personal computer monitor 1, a keyboard 3 for inputting information, a mouse 4, a TV monitor 5, and a camera control unit 6 are connected to the personal computer 2, respectively. There is a microscope 8
The TV camera 7 attached to is connected. And
FIG. 2 is a diagram showing the configuration of part of the first embodiment in more detail, and shows the flow of image signals to the TV monitor 5.

As shown in FIG. 1, the personal computer 2 is connected to the TV monitor 5 via a frame memory 9, and the frame memory 9 is connected to the personal computer 2 via a compression / expansion board 10. Further, the frame memory 9 is connected to the TV camera 7. Then, the mouse display signal 21 is output from the personal computer 2 to the frame memory 9. And
The image file signal 22 is exchanged between the personal computer 2 and the compression / expansion board 10, and the compression / expansion signal 2 is transmitted between the compression / expansion board 10 and the frame memory 9.
3 exchanges are carried out. Then, the video signal 24 is output from the TV camera 7 to the frame memory 9, and the video signal 25 is output from the frame memory 9 to the TV monitor 5. Further, FIG. 3 is a diagram showing a display of a typical menu screen of the stand-alone microscope system of the embodiment, and FIG. 3A shows a menu screen displayed on the personal computer monitor 1, and FIG. b) is TV
The observation image displayed on the monitor 5 is shown.

As described above, in the stand-alone microscope system of the first embodiment, there are less restrictions on observation than in the case of remote observation, and not only the TV monitor 5 but also an eyepiece can be used for observation. Since the image inside is not a still image but a moving image, it has a feature that the stage can be freely moved and observed. Therefore, it is not necessary to perform a troublesome operation such as specifying the position of the child image on the low-magnification still image, and it can be handled in the same way as a normal microscope without the TV monitor 5. The observation procedure of the microscope system according to the first embodiment will be described below.

First, the motorized stage of the microscope is moved vertically and horizontally to set it at a desired position. The movement of the electric stage is performed manually or by pressing the cursor key of the keyboard 3, and by moving the mouse cursor to any place on the personal computer screen with the mouse 4 and clicking.

After moving the motorized stage to a desired position, observation at low magnification is performed. Since the video signal is output to the TV monitor 5 via the frame memory 9 as described above, the mouse cursor can be displayed on the TV monitor 5 or loaded into a file at any time. .

Furthermore, the observation image is captured by moving the mouse cursor to "image capture" displayed on the personal computer screen and clicking with the mouse 4 or by corresponding key operation.

On the personal computer screen, each time an image is captured, the number of captured images is displayed, and the captured image is recorded together with stage coordinate data and lens magnification data. Instead of loading the captured image, you can move the stage to the captured position. Then, in order to perform high-magnification observation, the objective lens of the microscope 8 is manually or automatically switched to a high-magnification lens, and the magnification of the lens is displayed on the personal computer screen. Then, similarly to the observation at the low magnification described above, the motorized stage of the microscope is set at a desired position, and the observation of the specimen at the high magnification is performed.

At the time of this high-magnification observation, the pitch moved by the mouse cursor or the arrow buttons is made smaller than that at the low-magnification, so that the observer does not miss the part he or she wants to see from the TV screen. ing. Then, the moving pitch of the mouse cursor changes according to the magnification of the objective lens.

Subsequently, the mouse cursor is moved to the "image capture" position, and the mouse 4 is clicked to capture the image. If the personal computer 2 has an image that can be a parent image, a parent-child relationship is automatically generated. To do. The parent-child relationship of this image is displayed as a tree diagram on the personal computer screen as shown in FIG. 4, and when the parent image is displayed again on the screen, a rectangle is displayed at the position of the child image as shown in FIG. . Here, the condition of whether the parent-child relationship is established is based on the following calculation formula.

That is, assuming that the image a is an image having a lower magnification than the image b, the number of pixels in the X direction in the image a is 1x.
The number of pixels in the a and Y directions is 1ya, the actual distance per dot is da, and the stage coordinate of the center position is (Xa
0, Ya0), the stage coordinates (Xa
1, Ya1) and the stage coordinates (Xa2, Ya
2) is represented by the following equations (1a) to (1d). Xa1 = Xa0-1xa / 2 * da (1a) Ya1 = Ya0 + 1ya / 2 * da (1b) Xa2 = Xa0 + 1xa / 2 * da (1c) Ya2 = Ya0-1ya / 2 * da (1d) On the other hand, In image b, the center coordinates are (Xb0, Yb0)
Then, if the following condition (2) is satisfied, a can be regarded as the parent image of b. ( Xa1 ≦ Xb0 ≦ Xa2 ) and (Ya1 ≦ Yb0 ≦ Ya2) (2) Further, the position (x, y) of b on the a image is expressed by the following formula (3a),
It is shown in (3b). x = (Xb0-Xa0) / da (3a) y = (Yb0-Ya0) / da (3b)

By the way, the order of observation is usually to increase the magnification from a low magnification to a high magnification, but the order of capturing images is, for example, to capture a macro image for mapping later. In addition, it is possible that a low-magnification image is captured later. Then, also at this time, the parent-child relationship link is generated, and all the high-magnification images included in the low-magnification image area are obtained by the calculation by the above formula.

At this time, the parent-child relationship is indicated by a tree diagram on the personal computer screen and a rectangle indicating the position of the child image on the parent image of the TV. When the portions showing the respective images are clicked with the mouse 4, the image is displayed. It is displayed on the TV monitor 5. The series of observation procedures described above are shown in FIGS. 17 (a) to 17 (c).

As described above, in the microscope system of the first embodiment, it is possible to perform observation with a parent-child relationship in which a part of the low-magnification image is enlarged and observed at high magnification in the still image file captured by the microscope 8. In addition, a parent-child relationship can be generated later between a low-magnification image and a high-magnification image that are captured without having a parent-child relationship. Next, remote cytodiagnosis of the microscope system according to the second embodiment will be described. FIG. 6 is a diagram showing the configuration of the microscope system according to the second embodiment. As shown in the figure, the microscope system of the present embodiment includes a request-side terminal 100 and an observation-side terminal 10.
1 and 1 are connected by a communication line 11.

The request-side terminal has the same configuration as that of the first embodiment, and the observing-side terminal 101 includes a personal computer 2, a monitor 1, a keyboard 3, a mouse 4, and a TV monitor connected to the personal computer 2. It is composed of 5 and 5.

In the cytodiagnosis by the microscope system of this embodiment, the observation such as the histological examination which does not increase the magnification from low magnification to high magnification is not performed. That is, a general method of scanning a sample by using an objective lens having a high magnification of 40 times, 60 times, or 100 times is used. Also, even if it is called remote operation, the requesting terminal 100
Until the observation side terminal 101 is connected by the communication line 11, the same operation as the stand-alone according to the first embodiment is performed. The observation procedure of the microscope system according to the second embodiment will be described below. In the present embodiment, the stage is set at a desired position and the attention spot is captured. There are two types of moving methods for this stage: manual operation and automatic scanning.

First, in the manual operation, the stage is moved by pressing the cursor key of the keyboard 3 or by clicking the arrow button on the personal computer monitor 1 with the mouse 4.

On the other hand, in the automatic scanning, as shown in FIG. 7, first, the scanning range to be scanned by the electric stage is determined. At this time, a low magnification lens is used to project a cross in the center of the screen, and the upper left of the scanning range is aligned with the center of the cross. Then, when the position is determined, the fact that the keyboard 3 or the mouse 4 is scanned is notified to the computer.

Then, the personal computer 2 reads the coordinates of the electric stage at that time and stores them in a memory or a file. Next, determine the lower right point. Also at this time, the lower right point is aligned with the cross in the center of the screen, and the mouse 4 or the keyboard 3 is operated to determine. Then, the personal computer 2 reads the coordinates of the electric stage at that time,
Store in memory or file. Between the coordinates of these two points is the scanning range of the auto scan. It is also possible to scan the entire preparation without specifying the scanning range.

Subsequently, with respect to the designation of the attention position and the image capture, the movement of the stage is stopped when the stage is moved to the attention position by the auto scan or the manual scan. Then, the focus and aperture are readjusted and the image capture button is pressed to capture the image in the file, and the stage coordinates are automatically recorded. Then, the captured image is transmitted to a hospital in which the observation side terminal 101 is located, and the observation side terminal 101 designates capture at a low magnification.

If the user wants to enlarge the image further while observing the image, or if he / she also wants to see the surrounding image at a low magnification, for example, if an instruction is made to capture an image at a low magnification, that is, to capture the parent image, the operation side is given to the requesting side. Then, the observer waits for the image data to be received.

Then, in the request-side terminal 100, the electric stage moves to the coordinates of the original image, that is, the child image, the objective lens is switched to the designated low-magnification one, and the microscope image is displayed on the display screen of the TV monitor 5. The image is displayed. At this time, a rectangle indicating a child image is displayed in the center of the TV monitor 5, as shown in FIG. Since it is not always necessary to capture the child image so that it is in the center, the stage can be moved. Then, when the electric stage moves, the rectangle indicating the child image moves in accordance with the movement. This rectangle always surrounds the area of the child image.

Thus, when the fine adjustment of the position is completed, the image is captured, and the captured image is transmitted to the observing side, and such an image capturing designation method makes the observation easier. 18A to 18C show a series of observation procedures of the request-side terminal 100, and FIGS. 19A and 19B show a series of observation procedures of the observation-side terminal 101.

As described above, in the microscope system according to the second embodiment, it is possible to perform observation with a parent-child relationship in which a part of the low-magnification image is enlarged and observed at high magnification in a still image file captured from the microscope. Furthermore, in addition to the above functions, in a microscope remote control system that performs remote observation through a communication line, it is possible to generate a parent-child relationship by instructing the capture of a low-magnification image including the image area from a high-magnification image. it can.

Next, the consultation by the microscope system according to the third embodiment will be explained. Here, the consultation refers to a case in which there are pathologists A and B at two terminals and a doctor A at one terminal side consults a doctor B at the other terminal side with a case that the doctor A cannot understand.

First, in the stand-alone observation, the doctor A at the observing side terminal observes the sample in a stand-alone manner. If the specimen cannot be judged by oneself, another pathologist will be requested to consult.

Then, the image file which is not understood is sent to the doctor B of the requesting terminal. That is, when requesting a consultation, it is generally considered that in many cases, only the part to be consulted is left as a file. This image file is transferred to the other terminal. And doctors A and B
Consultation with. That is, doctors A and B
At a convenient time, reconnect the line and conduct a consultation. At this time, the same image is displayed on the monitors of the doctors A and B, the mouse cursors of the doctors are displayed on the screen of the TV monitor 5, and communication can be performed by the telephone voice and the movement of the mouse cursor.

Further, the low-magnification image is taken in and the file is transferred. That is, during the consultation, the doctor A may request a new image capture. In some cases, a low-magnification image including the periphery of the image already sent may be required. In this case, the import method is
This is exactly the same as the parent image capturing method of the second embodiment.
This feature allows for more flexible consultation. 20A and 20B show a series of observation procedures by consultation on the request side and the observation side of the microscope system according to the third embodiment described above.

As described above, the microscope system according to the third embodiment is most suitable in the first and second embodiments when there are a plurality of images that can be parent images for one child image. It is possible to create a simple parent-child relationship. Next, optimization of the parent-child relationship by the microscope system according to the fourth embodiment will be described. In this embodiment, a plurality of images are captured, and when there are a plurality of image files that can be parent images of a certain image, a natural parent-child relationship suitable for observation is optimized. The relationship between the plurality of parent image candidates is divided into two types, that is, when the magnification is different as shown in FIG. 9 and when the magnification is the same as shown in FIG. 10 but the coordinates are different.

Then, as shown in the tree diagram of FIG. 11, when the magnification is different, that is, the magnification of the three images a, b, c is a.
In the case of <b <c, the direct parent image of the high-magnification image c is b, and the image a is the parent image of b.

Further, as shown in FIG. 12, when the magnification is the same but the coordinates are different, there are two images a and b having the same magnification, and there is an image c which can be either of the child images. At the time, the position of the image c on the parent image is set to be more centered. Further, when the image c is near the center of a, but exists near the end on the image b, the parent image of the image c becomes a.

As described above, in the microscope system according to the fourth embodiment, there are a plurality of child images designated from the parent image, and the magnifications of these child images are different. It is possible to create a simple parent-child relationship. Next, optimization of the parent-child relationship by the microscope system according to the fifth embodiment will be described.

In this embodiment, the relationship between the images is the same as in the case of FIG. 9 described in the fourth embodiment. However, the difference is that the user has specified the parent-child relationship between the images a and b and the parent-child relationship between the images a and c. Therefore, unlike the fourth embodiment, the system cannot arbitrarily generate the parent-child relationship between the images a, b, and c and delete the parent-child relationship between the images a and c. In this case, the parent images of c are both a and b as shown in the tree diagrams of FIGS. 13 and 14. By doing so, it is possible to perform the observation in accordance with the stage of the magnification change from the image b to the image c, and it is possible to perform the observation as specified by the user.

As described above, in the microscope system according to the fifth embodiment, when the child image designated from the parent image can be the child image of another image, the optimum parent-child relationship can be generated. it can. Next, the distinction of the parent-child relationship display of the microscope system according to the sixth embodiment will be described.

In this embodiment, it is necessary to distinguish between capturing a child image from a parent image designated by the user or capturing a parent image from a child image and parent-child relationship automatically generated by the system. As shown in FIG. 15, the rectangle indicating the existence of the child image on the TV monitor 5 is displayed in different colors such as red for the user-specified one and blue for the automatically-generated one.

Further, as shown in FIG. 16, the tree diagram on the personal computer monitor 1 displays lines connecting images by changing colors as on the TV monitor 5, or a line thickness shape, etc. To be displayed. Specifically, the parent-child relationship designated by the user refers to designation of the capturing position of the child image on the parent image, that is, designation of the parent image from the child image as in the second and third embodiments. .

On the other hand, what is automatically generated by the system is
The thing optimized by 1st and 4th Example, and the parent-child relationship which the system in 4th and 5th Example newly derived is shown. By switching the display in this way, the progress of the observation is made clearer.

As described above, in the microscope system according to the sixth embodiment, in the first to fifth embodiments,
User-specified parent-child relationships and system-generated parent-child relationships can be displayed separately.

As described above in detail, in the microscope system of the present invention, a parent-child relationship is generated after capturing between a low-magnification image and a high-magnification image that are acquired without having a parent-child relationship. You can

[0049]

As described above, according to the present invention, it is possible to provide a microscope system that flexibly generates a parent-child relationship of a microscope image and efficiently performs observation and reproduction after the observation.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a microscope system according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a partial configuration of the first embodiment in more detail.

3A and 3B are diagrams showing display of a typical menu screen of the microscope system of the first embodiment.

FIG. 4 is a tree diagram showing a parent-child relationship of images displayed on a personal computer screen.

5A and 5B are diagrams showing how a rectangle is displayed at a position of a child image when the parent image is displayed again on the screen.

FIG. 6 is a diagram showing a configuration of a microscope system according to a second embodiment of the present invention.

FIG. 7 is a diagram for explaining a method of moving a stage by automatic scanning.

8 is a diagram showing a rectangle of a child image area shown in the center of the TV monitor 5. FIG.

FIG. 9 is a relationship between a plurality of parent image candidates (when magnifications are different)
FIG.

FIG. 10 is a diagram showing a relationship between a plurality of parent image candidates (when the magnification is the same but the coordinates are different).

FIG. 11 is a tree diagram showing a parent-child relationship between images at different magnifications.

FIG. 12 is a tree diagram showing a parent-child relationship between images when the magnification is the same and the coordinates are different.

FIG. 13 is a tree diagram showing a parent-child relationship between images.

FIG. 14 is a tree diagram showing a parent-child relationship between images.

FIG. 15 is a diagram showing a rectangle indicating the presence of a child image on the TV monitor 5.

FIG. 16 is a tree diagram showing a parent-child relationship between images.

17 (a) to (c) are diagrams showing an observation procedure of the first example.

18 (a) to 18 (c) are diagrams showing an observation procedure of a request-side terminal in the second embodiment.

19 (a) and 19 (b) are diagrams showing an observation procedure of an observation side terminal in the second embodiment.

20 (a) and 20 (b) are diagrams showing the observation procedure of the request-side terminal and the observation-side terminal in the third embodiment, respectively.

FIG. 21 is a diagram illustrating the definition of the parent-child relationship of images.

[Explanation of symbols]

1 ... Monitor for personal computer, 2 ... Personal computer, 3 ... 4, Mouse, 5 ... TV monitor, 6
... Camera control unit, 7 ... TV camera, 8 ...
Microscope, 9 ... Frame memory, 10 ... Compression / expansion board.

─────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-64-20449 (JP, A) JP-A-5-111029 (JP, A) JP-A-6-51209 (JP, A) (58) Field (Int.Cl. ⁷ , DB name) G02B 21/00

Claims (6)

(57) [Claims] 1. A microscope objective lens for a desired specimen .
That an observation image and an image capture means for capturing a still image, the first still image of the objective lens is captured by the image capture means in the case of low magnification, a portion of said pair
By changing the object lens, the magnification is increased and it is taken in.
The second still image and a part of the second still image
Capture by increasing the magnification by switching the objective lens
About the 3rd still image that was created,
Image, the first when the image contained in itself is a child image
Or a parent-child relationship information generation unit that generates parent-child relationship information between the third still images after each image is captured, and a storage unit that stores the parent-child relationship information generated by the parent-child relationship information generation unit. A microscope system characterized by being provided. 2. The parent-child relationship information generating means automatically generates the parent-child relationship information by comparing the position relationship between the position of the parent image and the child image. The microscope system described. 3. The microscope system according to claim 1, wherein the parent-child relationship information generating means allows a user to input the parent-child relationship information. 4. The parent-child relationship between captured still images is displayed in a tree structure diagram based on the parent-child relationship information stored in the storage means. Microscope system. 5. The microscope system according to claim 1, wherein a position and an area of the parent image are displayed on the parent image based on the parent-child relationship information stored in the storage means. . 6. A microscope objective lens for a desired specimen
That an observation image and an image capture means for capturing a still image, the first still image of the objective lens is captured by the image capture means in the case of low magnification, a portion of said pair
Regarding the second still image captured by increasing the magnification by switching the object lens to a high magnification, when the first still image is the parent image and the second still image is the child image,
A parent-child relationship information generating unit that generates parent-child relationship information between these still images after each image is captured; and a storage unit that stores the parent-child relationship information generated by the parent-child relationship information generating unit. Microscope system characterized by.