JP2005099832A - Microscope observation system and microscopic image display method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To observe an image wherein a focal position of a desired target in a microscopic image is changed, on a remote site. <P>SOLUTION: A personal computer 1 fetches a plurality of images having the same coordinates and the same magnification power as a point that is designated by a mouse 3 on a macro image fetched by a TV camera 9 provided in a microscope 8 and displayed on a TV monitor, in the direction of an optical axis of a stage, and controls respective parts to handle, reproduce and display these images as a set of images wherein a focal point for one target is deviated. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a system mainly used for medical / biological microscope observation, and more particularly, to a microscope remote observation system that performs high-efficiency remote pathological diagnosis such as observation in the pathological field, telepathology, and teleconsultation.

  Conventionally, for example, in the medical field, pathological diagnosis using a microscope has been an essential diagnostic item. A system has also been proposed in which a microscope image is transmitted from a remote hospital to a university or the like, and a pathologist at the university makes a diagnosis based on the microscope image. Such systems are roughly classified into two types: systems that handle real-time microscope moving images and systems that handle still images.

  For example, a system that handles still images includes a system that captures and stores microscope still images by parent-child relationships. This system designates an area to be observed with a high-power lens on an image captured with a low-magnification objective lens, and enlarges and captures the image in that area. The low-magnification image is called a parent image, and the high-magnification image is called a child image. In this way, this system can keep the observation process structurally and the amount of data is small, so it can be realized even with a general communication line.

  However, in the above-described system for handling a real-time microscope moving image, since a moving image has a larger amount of information than a still image, a communication line for transmitting the image becomes expensive. That is, at present, it is not practical because there is no means other than using satellites and optical fibers. Further, it is not practical to search for and reproduce all diagnostic records in the course of observation for reasons such as communication lines, disk capacity, and processing capability. There is also a problem that the focus cannot be adjusted after the image is once captured. In particular, in cell diagnosis, it is often the case that a diagnosis is made by looking through a microscope while shifting the focus. However, it is also a problem of conventional systems that observation cannot be performed by such a stand-alone microscope method.

  On the other hand, in the above-described system for handling still images, there is a problem that the focus of an image that has been once captured and transferred when transferring a still image cannot be readjusted later. In particular, in cytodiagnosis, the specimen itself has a certain thickness, and therefore, the focal position is not necessarily one. In such a case, there is a problem that it cannot be used for observation while shifting the focus of a certain object as in normal microscope observation.

  The present invention has been made in view of the above problems, and its object is to enable observation of an image in which the focal position of a desired target in a microscope image is changed in a remote place, and further, operability and data capture. -To provide a microscope remote observation system that improves compression efficiency.

  In order to achieve the above object, in the microscope remote observation system of the present invention, a microscope terminal that captures and transmits a macro image as a whole image of a specimen and a microscope image, and an image transmitted from the microscope terminal are received and displayed. In a microscope remote observation system that has an observation terminal and a communication line that connects the microscope terminal and the observation terminal, and manages the microscope image by a parent-child relationship starting with the macro image, specify on the macro image captured by the microscope terminal And a control means for controlling to capture and display a plurality of images having the same coordinates and the same magnification as the selected location in the optical axis direction of the stage, and handling and reproducing these images as a collection of images shifted in focus with respect to one target. It is characterized by that.

  That is, in the microscope remote observation system of the present invention, the control means captures a plurality of images having the same coordinates and the same magnification as those specified on the macro image captured by the microscope terminal in the optical axis direction of the stage, and these images are captured. It is controlled so that it is handled and reproduced and displayed as a collection of images with the focus deviated for one object.

  According to the present invention, a microscope remote observation system is provided that enables observation of an image in which the focal position of a desired target in a microscope image is changed in a remote place, and further improves operability and data acquisition / compression efficiency. can do.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing the configuration of the microscope remote observation system according to the first embodiment of the present invention.

  As shown in FIG. 1 (a), the microscope remote observation system according to the present embodiment actually operates the microscope to acquire an image and requests the terminal 100, and from the request terminal 100 via the communication line 11. An observation-side terminal 101 that observes a desired microscope image based on the transmitted image data. The requesting terminal 100 includes a personal computer unit 12, a microscope unit 13, and a conference telephone 10, and the observation terminal 101 includes a personal computer unit 12 'and a conference telephone 10'. Further, the requesting terminal 100 is configured in detail as shown in FIG. That is, the personal computer unit 12 includes a personal computer 1 that controls the whole, a CRT display 2, a mouse 3 that is a pointing device, a keyboard 4 that is an input device, a frame memory 5, and a TV monitor 6.

  The microscope unit 13 includes a camera control unit (CCU) 7, a microscope 8, and a TV camera 9. Further, the conference telephone 10 is capable of talking without having a receiver, and is connected to the conference telephone 10 ′ in the observation side terminal 101 via the communication line 11. Note that the personal computer unit 12 ′ in the observation side terminal 101 has the same configuration as the personal computer unit 12 in the request side terminal 100, and a description thereof will be omitted here.

  In such a configuration, the microscope 8 in the requesting terminal 100 is controlled by the personal computer 1 and the movement of the stage 15 in the x, y, and z directions is instructed. The image data is captured by the camera 9 attached to the microscope 8 and is compressed when recording as a file. Further, in remote observation of the microscope image, the image data captured by the camera 9 is transferred to the observation side terminal 101 via the communication line 11.

  The microscope remote observation system according to the first embodiment then displays a plurality of magnified microscope images with different focal points for one target determined by the x and y coordinates, the magnification of the objective lens 14, and the parent image. It has the feature that it can be taken in.

  Next, the parent-child relationship between microscope images will be described with reference to FIG.

  As shown in FIG. 2A, in the present embodiment, the images are all managed by the parent-child relationship that is the relationship of enlargement / reduction by switching the objective lens 14 on the x, y plane, and the same. The image of the same coordinates captured by the objective lens 14 is operated in the z direction, which is the optical axis direction of the stage 15, and is taken in as a plurality of still images and managed together with the parent-child relationship. For example, images 1 to 5 illustrated in FIG. 2A are images in which the same target is captured by shifting the focal positions 1 to 5 illustrated in FIG. In this way, by comparing the microscope images with different focal positions, it is possible to observe a microscope image equivalent to when operating and observing the microscope even in a remote place.

  Hereinafter, the image capturing operation according to the first embodiment will be described in detail with reference to the flowchart of FIG.

  First, in step S <b> 101, image attributes are set using the mouse 3, keyboard 4, and the like and input to the personal computer 1. The attribute values include the x and y coordinates of the stage 15, the magnification of the objective lens 14, and the parent image. In this embodiment, images having the same data are collectively handled as images for the same object.

  Subsequently, in step S102, focusing is performed by moving the stage 15 of the microscope 8 in the z direction. This operation can be performed fully automatically with an autofocus microscope. In step S103, the TV camera 9 captures a microscope image.

  In step S104, the z coordinate of the stage 15 after movement is acquired. A plurality of images of the same object are distinguished by the difference in the z coordinate. This data is recorded in the personal computer 1 together with the other attribute values input in step S101.

  In step S105, an image is captured and compressed. Here, the image data recorded on the frame memory 5 is compressed by an image compression / decompression apparatus (not shown), and the compressed data is recorded as a file.

  Subsequently, in step S106, it is confirmed whether or not the image capturing is to be ended. If the image capturing is to be terminated, the loop is exited. This adjustment may be manual or automatic.

  In step S108, the captured images are sorted in the height order by the z-coordinate regardless of the order of capturing. At this time, the one where the stage 15 is far from the installation base, that is, the one close to the objective lens 14 is above. By this sort, when the images are viewed together, it looks as if the stage 15 has been moved from top to bottom. Of course, the up and down directions may be opposite directions. In this way, all capturing is completed in step S109. Then, after completion of capturing, if communication observation is in progress, the image is transferred to the observation side terminal.

  Images captured as described above can be displayed and observed using a parent-child relationship. That is, as shown in FIG. 4, when there are a plurality of images related to the same object, a scroll bar 19 is displayed beside the image, and the icon 21 is selected by clicking the icons 20 and 22 of the scroll bar 19 with the mouse 3. Move up and down to change the displayed image one frame at a time to a higher or lower z coordinate value.

  This icon 21 indicates the position of the image being displayed among the images sorted by the value in the z direction, and the displayed image can be changed by moving the icon 21. Then, by operating these icons 21, images are displayed in the actual height order. That is, if the icon 21 is moved upward, the upper image can be seen, and if the icon 21 is moved downward, the lower image can be seen.

  The conceptual diagram of the relationship between the images is as shown in FIG. 2 and the tree structure diagram is as shown in FIG. 5. All images belong to the parent-child relationship of the sample unit.

  Since images for the same object have the same attributes other than the z coordinate value, they are handled together. Further, since they are sorted by the z coordinate value, they are arranged in the order of height. These images can be called sequentially or randomly. And the image with respect to the same object is all handled the same from a viewpoint of parent-child relationship. That is, the same parent image or child image can be called up and displayed from any of the plurality of images.

  Further, the general parent-child relationship is the relationship between the image size, that is, the magnification of the objective lens 14 and the image based on the coordinate position (x, y) on the stage 15, and this parent-child relationship is related to the z-coordinate. Is the relationship of multiple images for the same object.

  As described above, in the microscope image observation system according to the first embodiment, it is possible to capture and compare a plurality of images shifted in focus with respect to the same object, and further combine them into one. Management makes remote observation of microscopic images more efficient.

  Next, a second embodiment of the present invention will be described.

  In the first embodiment described above, the z-coordinate of the stage 15 when capturing an image is uniquely set for each image. On the other hand, in the second embodiment, the number of captured images and the interval between the images are set first, and the stage 15 of the microscope 8 is controlled so that images of several frames to several tens of frames are continuously obtained. Enable to capture. In addition, if the reproduction is executed at a high speed, it is possible to reproduce the moving image. The configuration of the second embodiment is substantially the same as the configuration of the first embodiment shown in FIG.

  Hereinafter, the image capturing operation according to the second embodiment will be described with reference to the flowchart of FIG.

  In step S <b> 201, image attributes are set by the mouse 3 and the keyboard 4 and input to the personal computer 1. The attribute values include the x and y coordinates, the magnification of the objective lens 14 and the parent image as in the first embodiment. In this embodiment, in addition to these, the inter-frame pitch p and the number n of captured images are set. To do. The p and n are parameters for continuous image capture.

  Subsequently, in step S202, the stage 15 is moved in the z direction by the autofocus function of the microscope 8 to focus. In step S203, the value at this time is obtained. This focal position Z0 is set as the center coordinate for continuous capture. In step S204, the z coordinate is moved to the maximum value of the capture range. The maximum value and the minimum value at this time are expressed by the following equations.

Z = Z0 + 1/2 * (n-1) pZ = Z0-1 / 2 * (n-1) p
Then, n images are continuously captured in the loop of step S205, and in step S206, the z coordinate of the stage 15 after movement is acquired. A plurality of images of the same object are distinguished by the difference in the z coordinate. This data is recorded together with the other attribute values input in step S211. In step S207, the image is captured and compressed. Here, the image data recorded on the frame memory 5 is compressed by an image compression / decompression apparatus (not shown), and the compressed data is recorded as a file. In step S208, the stage 15 is moved to the next image capture position. Since the distance between the images is set with the pitch p, the coordinates after the movement are expressed by the following equation.

Z = Zp
When the capturing of n images is completed in this way, the loop is exited, and the capturing ends in step S209. Unlike the first embodiment, since the images are taken in the order of height, there is no need to sort again.

  In this way, the captured image is displayed and observed by selecting the image with the scroll bar as in the first embodiment. Here, a significant difference from the first embodiment is that if the image expansion / reproduction speed is high (approximately 1/10 second or less per image), the continuous reproduction of the captured image looks like a moving image. . That is, since the pitch between the images is constant depending on p, it is the same as when the stage 15 is moved at a constant speed in the z direction. The parent-child relationship between images is the same as that in the first embodiment.

  As described above, according to the second embodiment, operability is improved by automating the capture of a plurality of images without worrying about the work by focusing manually on each image by manual selection. can do. Furthermore, by enabling reproduction as a moving image, it can be closer to observation by operating the microscope at hand.

  Next, a third embodiment of the present invention will be described.

  In the first and second embodiments described above, since a plurality of pieces of image data with different focal points are provided, it is necessary to mark a desired specific image among the plurality of images. Therefore, the third embodiment has a function of marking a plurality of images for the same object. The configuration of the third embodiment is almost the same as the configuration of the first embodiment shown in FIG.

  This marking function makes it very easy for a user to mark a specific image. That is, as shown in FIG. 7, if the mark button 23 displayed on the display 2 is clicked with the mouse, the displayed image is marked. At this time, the marking flag in the image attribute data input at the time of capturing the image being displayed is set to “ON” in the system.

  On the other hand, when displaying / reproducing an image, it is necessary to distinguish between the marked one and the other. Therefore, as shown in FIG. 8, the color of the icon 26 on the scroll bar 19 is changed depending on the displayed image in order to distinguish. That is, the icon 26 is not particularly colored in an unmarked image. If the displayed image is changed by operating the scroll bar 19, the icon 26 changes its color if it is a marked image. When the button 25 on which two triangles shown in FIG. 8 are drawn is clicked, the image is not displayed one by one in the order of height (in this case, the button 24 is clicked), but only the marked one is displayed. Will be skipped.

  Furthermore, when a plurality of images are continuously captured, the transfer time naturally becomes longer than the transfer of only one still image. In order to shorten this time, the image compression ratio must be increased, so that the image quality naturally decreases. Therefore, in the present embodiment, the marked image can be recaptured with low compression and high image quality.

  The recapture operation according to the third embodiment will be described below with reference to the flowchart of FIG.

  First, in step S301, the attribute data of the image being displayed is input to the personal computer 1 again with the mouse 3 and the keyboard 4. This data was set at the time of the first capture. Subsequently, in step S302, the stage 15 of the microscope 8 is moved, and in step S303, the objective lens 14 is switched. In step S304, the compression rate is lowered and the image quality is improved.

  In step S305, an image is captured. It should be noted that since a high quality image is obtained by this capture, a low quality image is not necessary. Therefore, in step S306, the original image is deleted. In this way, all the operations are finished in step S307.

  The captured image is transferred from the requesting terminal 100 to the observing terminal 101 and can be viewed simultaneously on both terminals. And this image is shown in the place where the re-captured image was located as shown in FIG.

  As described above, according to the third embodiment, the requesting terminal 100 can grasp where the observer of the observing terminal 101 is paying attention by marking on the observing side. Further, when capturing a child image, if there is a marked image among a plurality of image groups serving as a parent image, the requesting terminal 100 can grasp the position to be focused.

  When the reproduction is performed again at the end of the observation / diagnosis, it is possible to grasp the position where the observer of the observation-side terminal 101 is paying attention. In addition, by combining continuous capture with low compression and recapture of the marked one, it is possible to achieve both a reduction in transfer time and a high image quality where necessary.

  Next, a fourth embodiment of the present invention will be described.

  If the entire image is continuously captured for all objects as in the first to third embodiments described above, the amount of data increases, and even if a specific image is marked as in the third embodiment. It is not recorded in which part of the entire image that you are interested in.

  On the other hand, the fourth embodiment is a system in which only a part of an image is specified and only that part is continuously captured. Here, an image in which only a part is captured is referred to as a partial image, and an image in which the entire target is captured is referred to as a whole image.

  Furthermore, in the fourth embodiment, the whole image can be additionally captured as an image with the focal position changed by using the z-coordinate value of what is noticed in the partial image. The configuration of the fourth embodiment is substantially the same as the configuration of the first embodiment shown in FIG.

  Hereinafter, the partial image capturing operation according to the fourth embodiment will be described with reference to the flowchart of FIG.

  First, in step S401, a partial image capture range is designated. This can be specified on the entire image by moving the mouse pointing 27 to a desired position and clicking the mouse 3, as shown in FIG. In step S401, the designated capture range is also input.

  In step S402, image attributes are input. The difference from the first and second embodiments described above in this step is that the entire image is input as an attribute value instead of the parent image. The setting for continuous capture is determined by the inter-frame pitch p and the captured number n, as in the second and third embodiments described above.

  In step S403, continuous capture is performed. This procedure is the same as in the second embodiment. However, the image data that is compressed and left in the file is only the data related to the image in the capture designation range 28 shown in FIG. When the image data is captured in this way, all capture is finished in step S404.

  The relationship between the images thus captured is as shown in FIG. 13, and FIG. 14 is a tree structure diagram thereof. As shown in the figure, all the images belong to a parent-child relationship in units of samples, and the relationship of a plurality of images with respect to the same object is the same as in the first to third embodiments.

  Furthermore, in the present embodiment, in addition to these relationships, a relationship between the whole image and the partial image is added. The whole image is a full-size image that has been captured in the past, and the partial image is an image in which only a partial region of the same target is captured. A group of partial images obtained by continuously capturing a certain area is referred to as a partial image group. Note that there may be a partial image group in which a plurality of locations are specified on one whole image.

  The relationship of a plurality of images with respect to the same object is established in both the entire image and the partial image. The relationship between the whole image and the partial image is similar to the relationship between the parent image and the child image, but the display method is completely different.

  As shown in FIG. 15, this partial image is displayed by a rectangle 30 indicating its range on the entire image. In this case, it is necessary to distinguish the rectangle 29 indicating the child image and the rectangle 30 of the partial image by changing the display method. As a method for representing this difference, as shown in FIG. 15, a method of changing the color of the rectangle, a method of changing the thickness of the rectangle, a method of rounding the corner of one rectangle, and the like can be considered.

  Here, a plurality of partial image groups exist on one object, one whole image group. As these display / reproduction methods, there are a method using one scroll bar as in the first to third embodiments and a method of providing a scroll bar for each partial image group. First, a display method using one scroll bar will be described with reference to FIG.

  As shown in FIG. 16A, there are two partial image groups 31 and 32 on the screen, and the z coordinate of each image is as shown in FIG. At this time, two partial images having the same z coordinate value are displayed. However, since the maximum value and the minimum value of the z coordinate of each partial image group are not necessarily equal, an image having a different z coordinate value is displayed at that time. For example, when a in FIG. 16B is displayed, a 'is displayed at the same time, and when a plurality of whole images are captured, the z coordinate value closest to the z coordinate of the partial image is displayed. The However, when there are a plurality of partial image groups, since the target to be captured is different, the in-focus position should be different. In that case, it is better that a scroll bar is displayed for each partial image and an image to be displayed can be selected. A display method for displaying a scroll bar for each partial image will be described with reference to FIG.

  As shown in FIG. 17, when the mouse 3 is clicked on a rectangle representing a partial image, a scroll bar 34 is displayed on the side. The reason why it is not displayed from the beginning is that it is an obstacle to the observation of the entire image. The method for changing the partial image to be displayed is the same as the conventional image display method using the scroll bar 35.

  Then, after the display adjustment of one partial image group is finished, it is only necessary to click in the rectangle of another partial image. Then, a scroll bar 34 is displayed next to the partial image, and the display image can be changed. When there are a plurality of whole images, the scroll bar 35 is displayed next to the whole image as in the previous embodiments. If this is moved, the display of the whole image can be changed.

  As described above, according to the fourth embodiment, since the area for continuous capture is small, the capture / compression speed is increased and the amount of data is reduced. Furthermore, since only the target object is continuously captured, the observation process can be left more clearly. In particular, it is possible to record data relating to an important part of an image captured at the maximum magnification of the microscope. In addition, since the entire image can be recaptured only at an appropriate focal position, the amount of data can be reduced. Then, by displaying each partial image, a plurality of attention portions in one target can be simultaneously observed with an image having an appropriate focus.

  Next, a fifth embodiment of the present invention will be described.

  In the above-described fourth embodiment, continuous capture of partial images is specified on the entire captured image. On the other hand, in the fifth embodiment, a partial image range is specified when capturing a child image, and the entire child image is not captured continuously, but only one image is captured. It has the feature to do. The configuration of the fifth embodiment is substantially the same as that of the first embodiment shown in FIG.

  Hereinafter, an image capturing method according to the fifth embodiment will be described with reference to the flowchart of FIG.

  First, in step S501, child image capturing is designated. In step S502, a partial capture range is designated. That is, as shown in FIG. 19, a small rectangle 37 is designated by moving the mouse pointing 36 and clicking the mass 3 in the rectangle 38 of the child image to be captured drawn on the parent image.

  In step S503, the actual partial capture range on the child image is obtained from the designation data in steps S5011 and S502. This is as shown in FIG. That is, in FIG. 20, the numbers of pixels of the parent image and the child image are X and Y, and the relative coordinates of the partial image area 39 when the child image is designated with respect to the child image designation areas x and y are (x1, y1), (x2, If y2), the coordinates (X1, Y1) and (X2, Y2) of the range 40 of the actual partial image on the child image are expressed by the following equations.

X1 = X / x * x1, Y1 = Y / y * y1X2 = X / x * x2, Y2 = Y / y * y2
In step S504, a child image is captured. At this time, only one image of the entire child image is captured. In step S505, partial images are continuously captured. The method at this time is the same as the partial image capturing of the fourth embodiment shown in FIGS. 16 (a) and 16 (b). The relationship between images and display / reproduction are the same as those in the fourth embodiment.

  As described above, according to the fifth embodiment, in addition to the functions of the fourth embodiment, it is possible to specify a portion of particular interest when capturing a child image.

  As described above in detail, according to the microscope remote observation system of the present invention, it is possible to capture and compare a plurality of images shifted in focus with respect to the same object, and to perform remote observation of a microscope image with higher efficiency. It is said. Furthermore, operability can be improved by automating the capture of a plurality of images. Then, the image capture and compression speed can be improved and the data amount itself can be reduced.

  Of course, the present invention is not limited to the above-described embodiment, and various improvements and modifications can be made.

(A) And (b) is a block diagram which shows the structure of the microscope remote observation system which concerns on the 1st Embodiment of this invention. (A) And (b) is a figure for demonstrating the parent-child relationship between images. 6 is a flowchart for explaining an image capturing operation according to the first embodiment; It is a figure which shows the scroll bar displayed when there exist two or more images of the same object. It is a tree structure figure which shows the relationship between images. 10 is a flowchart for explaining an image capturing operation according to the second embodiment; It is a figure which shows the mark used when a user marks a specific image with the marking function of 2nd Embodiment. It is a figure for demonstrating the method of distinguishing the image marked and the other image at the time of the display / reproduction | regeneration of an image. It is a flowchart for demonstrating the operation | movement of the recapture by 3rd Embodiment. It is a tree structure figure which shows the relationship between images. It is a flowchart for demonstrating the operation | movement of the partial image capture by 4th Embodiment. It is a figure for demonstrating designation | designated of the capture range of a partial image. It is a conceptual diagram which shows the relationship between images. It is a tree structure figure which shows the relationship between images. It is a figure for demonstrating the method of distinguishing the rectangle which shows a child image, and the rectangle of a partial image. (A) And (b) is a figure for demonstrating the display method by one scroll bar of the some partial image group which exists on one object and one whole image group. It is a figure for demonstrating the display method which shows a scroll bar for every partial image of the some partial image group which exists on one object and one whole image group. It is a flowchart for demonstrating the image capture method by 5th Embodiment. It is a figure which shows a mode that the small rectangle was designated in the rectangle of the child image drawn on the parent image to be taken in. It is a figure for demonstrating the method of calculating | requiring the actual partial capture range on a child image from designation | designated data.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Personal computer, 2 ... CRT display, 3 ... Mouse, 4 ... Keyboard, 5 ... Frame memory, 6 ... TV monitor, 7 ... Camera control unit, 8 ... Microscope, 9 ... TV camera, 10, 10 '... For meetings Telephone, 11 ... communication line, 12, 12 '... PC section, 13 ... microscope section, 14 ... objective lens, 15 ... stage.

Claims (1)

A microscope terminal that captures and transmits a macro image and a microscope image that are the entire image of the specimen, an observation terminal that receives and displays an image transmitted from the microscope terminal, and a communication line that connects the microscope terminal and the observation terminal. In the microscope remote observation system that manages the microscope image by the parent-child relationship starting with the macro image, an image having the same coordinates and the same magnification as the designated location on the macro image captured by the microscope terminal is arranged in the optical axis direction of the stage. A microscope remote observation system comprising: control means for capturing a plurality of images, and controlling such images to be handled and reproduced and displayed as a collection of images shifted in focus with respect to one object.

Images