JP4600395B2 - Microscope system and image processing method - Google Patents

Description

  The present invention relates to a microscope system and an image processing method used for specimen observation.

2. Description of the Related Art Conventionally, a microscope system that can electrically control movement of a stage on which a sample is placed and switching of magnification of an objective lens is known. In addition, by using such a microscope system, the entire area of the specimen is scanned in detail while capturing the enlarged images of the specimen in sequence, and the obtained large number of enlarged images are connected by software processing to increase the height of the entire area of the specimen. A technique (specimen scanning system) for generating a fine image is also known (see, for example, Patent Document 1). By imaging the entire region of the specimen, virtual observation is possible even when a valuable specimen is not at hand.
USP 6101265 (Japanese translation of PCT publication No. 2002-514319)

  However, in the above-described specimen scanning system, since the entire region of the specimen is scanned finely to generate a high-definition image, an enormous amount of time is required to generate the image. Also, the data amount of the generated image becomes enormous. Therefore, the sample scanning system described above has not been practical. When the operation of the specimen scanning system is started, the microscope system is occupied for a long time until the operation is completed.

  An object of the present invention is to provide a practical microscope system and an image processing method that can efficiently image a specimen in a short time and can reliably reduce the amount of data of a generated image.

  The microscope system of the present invention is a microscope system including a microscope and an imaging device that images a specimen on the stage of the microscope, an acquisition unit that acquires identification information of the specimen on the stage, and the acquisition unit The acquired identification information of the sample is stored in association with the macro image and the micro image of the sample imaged by the imaging device, and stored in the storage unit based on the identification information of the sample. It is determined whether or not the sample is a registered sample, and in the case of determination that the sample is a registered sample, a macro image of the registered sample is newly captured by the imaging device. And additional recording means for storing the micro image in association with the storage means.

  In the above microscope system, the first display means for displaying the macro image of the specimen, the second display means for displaying the micro image of the specimen, the display control of the first display means and the second display means. Display control means for performing, based on identification information of the specimen acquired by the acquisition means, the specimen is a new specimen, or has already been associated with the macro image If it is determined whether the micro image is a registered sample stored in the storage means, and it is determined that the sample is a new sample, the macro image of the sample imaged by the imaging device is A micro image displayed on the first display means and associated with the macro image of the sample displayed on the first display means is displayed on the second display means, and the table The control means reads the macro image of the sample stored in the storage means and displays the macro image on the first display means when determining that it is the registered specimen, and causes the second display means to display it. The micro image to be displayed is switched between a micro image associated with the macro image of the specimen stored in the storage means or a micro image of the specimen captured by the imaging device. Is preferred.

  In the above microscope system, the first display means for displaying the macro image of the specimen, the second display means for displaying the micro image of the specimen, the display control of the first display means and the second display means. Display control means for performing, based on identification information of the specimen acquired by the acquisition means, the specimen is a new specimen, or has already been associated with the macro image For determining whether the micro image is a registered sample stored in the storage means and determining that the sample is a new sample, the macro image of the sample imaged by the imaging device is used. A micro image displayed on the first display means and associated with a macro image of the specimen displayed on the first display means is displayed on the second display means, and the table The control means reads out the macro image of the specimen stored in the storage means and displays it on the first display means when determining that it is the registered specimen, and further displays the second display means. An off-line mode in which the micro image to be displayed on the micro image is associated with the macro image of the specimen stored in the storage means, and the micro image to be displayed on the second display means is captured by the imaging device. It is preferable to switch and display the online mode as a micro image of the specimen.

Further, in the above microscope system, the storage means stores, in addition to the identification information, the macro image, and the micro image, information related to observation history with respect to the sample, and the additional recording means includes the second recording means. When the micro image displayed on the display means is stored, it is preferable to update the observation history information.
In the above microscope system, it is preferable that the observation history information includes date and time information and observer name information related to the storage operation of the micro image displayed on the second display unit.

In the microscope system, the observation history information preferably includes identification information of the microscope system.
Further, in the above microscope system, the storage unit stores information of observation history for the specimen in association with the identification information, the macro image, and the micro image, and stores the information according to an instruction from the outside. Third display means for reading out and displaying a macro image and a micro image associated with the identification information from the storage means, and the predetermined identification information according to an instruction from the outside during the display by the third display means It is preferable to further comprise update means for updating information of the observation history associated with.

  In the microscope system, the storage unit associates the macro image and the micro image with each other including information on a shooting position of each micro image and shooting conditions, and associates the observation target with the stage. When the specimen is placed, the photographing position and photographing condition of the micro image associated with the identification information obtained by the obtaining means are read from the storage means and reproduced, and then the photographing position and photographing condition are determined. Fine-adjusting means for fine-adjusting, and photographing the specimen to be observed in a state after fine-adjusting by the fine-adjusting means to generate a micro-image of the specimen, and the micro-image is displayed on the first display means It is preferable that the image processing apparatus further includes a control unit that stores the macro image in association with the displayed macro image and the identification information acquired by the acquisition unit in addition to the storage unit.

  Further, in the above microscope system, a position recognition marker storage unit that detects and stores a position recognition marker of the sample from an image of a preceding sample imaged by the imaging device, and a sample of the sample acquired by the acquisition unit Based on the identification information, it is determined whether the sample is a new sample or a registered sample in which the macro image and the micro image are already stored in the storage unit, and the sample is a registered sample. A position correction unit that corrects the loading position of the specimen loaded on the stage on the stage to a normal position based on the position recognition marker stored in the position recognition marker storage unit; It is preferable to provide.

In the microscope system described above, the additional recording unit is adjacent to each other among past micro images associated with the macro image of the registered specimen and / or micro images newly captured by the imaging device. It is preferable to combine a plurality of micro images to be stored in the storage means as one micro image.
The image processing method of the present invention is an image processing method of a microscope system including a microscope and an imaging device that images a specimen on the stage of the microscope, and an acquisition step of acquiring identification information of the specimen on the stage. A storing step of associating and storing a macro image and a micro image of the sample imaged by the imaging device with the acquired identification information of the sample, and storing in the storing step based on the identification information of the sample It is determined whether or not the registered sample is a registered sample, and in the combination determined to be the registered sample, a macro image of the registered sample is newly captured by the imaging device. And an additional recording step of adding and storing a micro image of the sample in association with each other.

  The image processing method may further include first display means for displaying a macro image of the specimen and second display means for displaying a micro image of the specimen, the first display means and the second display means. In the image processing method of the microscope system for performing display control of the above, based on the acquired identification information of the specimen, the specimen is a new specimen, or corresponds to the macro image and the micro image that have already been stored If it is determined whether it is a registered sample, and if it is determined that the sample is a new sample, the macro image of the sample imaged by the imaging device is displayed on the first display means, and When the micro image of the specimen imaged by the imaging device is displayed on the second display means and when it is determined that the specimen is the registered specimen, The micro image of the specimen read out and displayed on the first display means, and the micro image displayed on the second display means is the micro image of the specimen stored in the storage step, or It is preferable to switch between the micro image of the specimen imaged by the imaging device.

  According to the present invention, it is possible to provide a practical microscope system and an image processing method that can efficiently image a specimen in a short time and can reliably reduce the data amount of the generated image.

It is a whole block diagram of the microscope system of this embodiment. 1 is a schematic view showing a state in which a specimen 10A is inserted into a microscope system 10. FIG. It is a figure explaining the screens 50 and 60 of the computer 24. FIG. It is a figure explaining the correlation with the log | history database of the computer 24, the macro image preserve | saved at the external storage device 27, the state file, and the micro image. It is a figure explaining an example of a status file. It is a flowchart which shows the procedure of a button input detection process. It is a flowchart which shows the procedure of a load process. It is a flowchart which shows the procedure of the reading process of a micro image. It is a flowchart which shows the procedure of a registration process. It is a flowchart which shows the procedure of the reading process of a macro image. It is a flowchart which shows the procedure of a reproduction process. It is a flowchart which shows the procedure of an automatic log | history registration process. It is a figure explaining the vicinity position of each of the X-axis, the Y-axis, and the Z-axis in the automatic history registration process. It is a flowchart which performs position recognition. It is a figure explaining a position recognition marker. It is a figure explaining another example of a position recognition marker. It is a figure explaining another example of a position recognition marker. It is a figure explaining the example which couple | bonds and memorize | stores a micro image by tiling processing. It is a figure explaining the example which couple | bonds and memorize | stores a micro image by tiling processing.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
As shown in FIG. 1, the microscope system 10 according to this embodiment includes a stage unit 11 on which a specimen 10A to be observed is placed, an illumination unit (12 to 16) that illuminates the specimen 10A, and an enlarged image of the specimen 10A. The imaging unit (17 to 21) to be formed, a CCD camera 22 that captures an enlarged image of the specimen 10A, a control unit 23, a computer 24, and an input device 25 are included.

  Among these, the stage unit 11, the illumination units (12 to 16), the imaging unit (17 to 21), the CCD camera 22, and the control unit 23 are accommodated in the housing 26 of the microscope system 10 (FIG. 2A). )), The computer 24 and the input device 25 are arranged outside the housing 26. The microscope system 10 is a box-shaped microscope system with a built-in CCD camera. An external storage device 27 is attached to the microscope system 10. The external storage device 27 is, for example, a hard disk.

In the interior of the housing 26 of the microscope system 10, the illumination units (12 to 16) are disposed below the stage unit 11, and the imaging units (17 to 21) and the CCD camera 22 are disposed above the stage unit 11. Is done. The microscope system 10 is an apparatus that observes the specimen 10A with transmitted illumination.
Furthermore, the specimen 10A is a preparation specimen. An identification barcode (not shown) is attached as a seal to the preparation portion of the specimen 10A.

Next, each component (12-25) of the microscope system 10 is demonstrated separately.
The stage unit 11 includes an electric stage that can move in the x and y directions, and a counter that detects the x and y positions of the electric stage (both not shown). The stage unit 11 can also be moved from the opening 26A of the housing 26 to the outside (see FIG. 2A). A path when the stage unit 11 moves between the inside and outside of the casing 26 crosses the two observation optical paths 10B and 10C of the microscope system 10.

In this embodiment, in order to simplify the description, it is assumed that the position (including rotation) of the specimen 10A on the stage unit 11 is always the same (even when re-mounted). That is, it is assumed that the specimen 10A can be positioned on the stage unit 11 with good reproducibility. In this case, it can be considered that the coordinate system of the stage unit 11 and the coordinate system of the specimen 10A always coincide.
In order to detect whether or not the preparation (specimen 10A) is placed on the stage unit 11, the stage unit 11 is provided inside the housing 26 as shown in FIGS. 1, 2B, and 2C. The preparation presser 11A is provided at the bottom, and the sensor 39A is provided below the opening 26A. The preparation presser 11A is rotatable about the shaft 11B. When the preparation is not placed on the stage unit 11 (the state shown in FIG. 2B), one end is positioned in the sensor 39A. Then, when the preparation is placed on the stage 11 from the outside of the housing 26 (the state shown in FIG. 2C), the other end is pushed by the preparation, and one end comes out of the sensor 39A. . For this reason, the sensor 39A detects the presence or absence of the preparation (specimen 10A) on the stage unit 11 according to the presence or absence of one end of the preparation presser 11A.

  In addition, when the specimen 10A is placed on the stage unit 11, a sensor 39B is provided inside the housing 26 (for example, above the opening 26A) in order to read a barcode (not shown) of the specimen 10A. A detection signal from the sensor 39B is output to the controller 40 of the control unit 23. The controller 40 acquires the identification information of the specimen 10A based on the output from the sensor 39B.

The identification information may not be detected by the sensor 39B, and may be externally input by the user. This identification information is information for identifying whether the preparation (specimen 10A) placed on the stage is a preparation already registered in the storage device 27. The sensor 39B and the controller 40 generally correspond to “acquiring means” in the claims.
The identification information of the specimen 10A may be ID information unique to the preparation stored in an IC tag fixed to the preparation, for example. Moreover, you may use the image data showing the outline of the sample (cell piece) on a preparation as another identification information. In this case, the outline of the sample on the preparation is read, and it is determined whether or not it is a registered preparation by an image matching method.

  The stage unit 11 is connected to the controller 40 via the stage control circuit 43 of the control unit 23. The stage control circuit 43 adjusts the observation position of the specimen 10 </ b> A by the stage unit 11 by moving the electric stage of the stage unit 11 in the x and y directions based on the control signal from the controller 40. In addition, the value of the counter of the stage unit 11 is read, and signals related to the x position and y position of the electric stage are output to the controller 40.

The illumination units (12 to 16) include an illumination light source 12, a diffuser plate 13, a condenser lens 14, an aperture stop 15 disposed on one observation light path 10B, and an illumination light source 16 disposed on the other observation light path 10C. It consists of
The light emitted from one illumination light source 12 is made uniform by the diffuser plate 13, condensed by the condenser lens 14, and passes through the aperture stop 15 to a partial region (observation point) of the specimen 10 </ b> A on the observation optical path 10 </ b> B. Incident. And the light which permeate | transmitted this partial area | region (observation point) is guide | induced to the image formation part (17-21).

The light emitted from the other illumination light source 16 enters the specimen 10A on the observation optical path 10C. At this time, half of the sample 10A is illuminated in a planar shape. And the light which permeate | transmitted this half area | region is also guide | induced to an image formation part (17-21). The illumination light source 16 is used at the same magnification observation of a wide range (the entire preparation) including the specimen 10A.
The illumination light sources 12 and 16 are connected to the controller 40 via the illumination control circuit 41 of the control unit 23 and controlled.

The diaphragm 15 is connected to and controlled by the controller 40 via the diaphragm control circuit 42 of the control unit 23.
The imaging unit (17 to 21) includes an objective lens unit 17, mirrors 18 and 19, a reduction lens unit 20, and a mirror 21. Here, the mirror 18 is disposed on the observation optical path 10B, and the mirror 19 is disposed on the observation optical path 10C. The observation optical path 10D after reflection by the mirrors 18 and 19 is common. The mirror 21 is an optical element that is arranged on the observation optical path 10D and returns the image turned upside down by the mirrors 18 and 19 to a front image.

  The objective lens unit 17 includes a 40 × objective lens 31, a 10 × objective lens 32, a passage hole 33, and a sensor 34. Any one of the objective lenses 31 and 32 can be inserted into the observation optical path 10B. The type (31 or 32) of the objective lens inserted in the observation optical path 10B is detected by the sensor 34. The passage hole 33 is disposed on the observation optical path 10C.

The objective lens unit 17 is connected to and controlled by the controller 40 via the focus control circuit 44 and the objective lens drive circuit 45 of the control unit 23.
The reduction lens unit 20 includes a 1/2 reduction lens 35, a 1x reduction lens 36, a passage hole 37, and a sensor 38. Any one of the reduction lenses 35 and 36 and the passage hole 37 can be inserted into the observation optical path 10D. The sensor 38 is for detecting the type (35 or 36) of the reduction lens inserted in the observation optical path 10D.

The reduction lens unit 20 is connected to and controlled by the controller 40 via the reduction lens drive circuit 46 of the control unit 23.
The CCD camera 22 is a two-dimensional imaging device using a CCD (charge coupled device), and has a plurality of light receiving units arranged two-dimensionally in the xy direction. The CCD camera 22 captures an image of the specimen 10A formed on the predetermined surface 2A and outputs an image signal. The output destination is the CCD control circuit 47 of the control unit 23.

The CCD control circuit 47 outputs and controls a timing signal to the CCD camera 22 based on a control signal from the controller 40.
The controller 40 is connected not only to each circuit (41 to 47) constituting the control unit 23, the sensor 34 of the objective lens unit 17, and the sensor 38 of the reduction lens unit 20, but also to the computer 24, the input device 25, An external storage device 27 is also connected.

  The computer 24 displays observation images and operation menus on the screens 50 and 60 (FIG. 3), and also serves as a history database server for observation images and the like. In the history database server, in order to prevent the history data from being tampered with, measures for preventing tampering such as digital watermarking are taken. Image data itself and acquired information (such as device settings and time), browsing information (such as time and location on the image), written comments, and processing (such as filtering) are registered as a set in the history database server. The history database server is preferably constructed and connected to a database server on a separate PC, but may be built in the apparatus.

  Even when the image data registered in the database is processed once, the original image data is always kept so that the original image data can be referred to at any time. In addition, the database has a write-once configuration that records all actions related to data correction such as browsing, entry comments, and processing, and is configured so that the history of browsing and correction actions cannot be erased or altered. It is desirable to do. For example, in order to identify a person who accesses the database, a password method is used, or image data is subjected to tampering prevention measures such as a digital watermark.

  The external storage device 27 stores an observation image file and the like. The history database of the computer 24 and the external storage device 27 generally correspond to “storage means” in the claims. The input device 25 is a GUI (Graphical User Interface) for operation displayed on the computer 24, and includes a keyboard 24A (or a keyboard created by software on the computer 24) connected to the computer 24. .

  When the controller 40 takes in the digital signal from the CCD control circuit 47 of the control unit 23, it temporarily stores it in the frame memory, and then outputs it to the screens 50 and 60 (FIG. 3) of the computer 24. As a result, the current still image of the specimen 10A is displayed on the screens 50 and 60 of the computer 24. The screens 50 and 60 also display past sample images (history images) read out from the external storage device 27 by the controller 40 via the history database of the computer 24.

  Here, the current still image of the specimen 10A will be described. An image (hereinafter referred to as a “micro image”) resulting from light reaching the CCD camera 22 through the objective lens (31 or 32) on the observation optical path 10B and the reduction lens (35 or 36) on the observation optical path 10D is a specimen 10A. Is an enlarged image of a partial region (observation point) of the computer 24 and is displayed in the divided region 51 of the screen 50 of the computer 24. The divided area 51 is a micro image display area.

  The components related to the display of the micro image are the optical elements (12 to 15, 31, 32, 18) on the observation optical path 10B, the optical elements (35, 36, 21) on the observation optical path 10D, the CCD camera 22, and the CCD control. This is a divided area 51 of the screen 47 of the circuit 47, the controller 40, and the computer 24, and generally corresponds to the “second display means” in the claims. These components are means for displaying a micro image by photographing the specimen 10A placed on the stage unit 11.

  Shooting conditions (magnification of the objective lens unit, magnification of the reduction lens unit, aperture amount, focal position, illumination brightness) and shooting position (xy position of the stage unit 11) when the micro image is displayed constitute the control unit 23. Each circuit (41 to 47) can be adjusted. The display magnification of the micro image is the magnification (5 ×, 10 ×, 20 ×, 40 ×) of the magnifying optical system inserted in the observation optical path 10B, 10D when the CCD camera 22 captures the magnified image of the specimen 10A. ) And an electronic zoom magnification (for example, 1 to 2 times) set when the CCD camera 22 outputs an image signal.

  In addition, an image (hereinafter referred to as “macro image”) caused by light reaching the CCD camera 22 through the passage hole 33 on the observation optical path 10C and the passage hole 37 on the observation optical path 10D is a wide range (preparation) including the specimen 10A. The whole image) and is displayed in the divided area 61 of the screen 60 of the computer 24. The divided area 61 is a macro image display area. In the divided area 61, rectangular frames 66, 67,... Described later are displayed so as to overlap with the macro image.

  The constituent elements related to the display of the macro image are the optical elements (16, 19) on the observation optical path 10C, the optical element (21) on the observation hole 33 and the observation optical path 10D, the passage hole 37, the CCD camera 22, and the CCD control circuit 47. And a divided area 61 of the screen 40 of the controller 40 and the computer 24, and generally corresponds to the “first display means” in the claims. These components are means for displaying a macro image by photographing the specimen 10A placed on the stage unit 11.

  Thus, when the specimen 10A is placed on the stage unit 11, the current micro image is basically displayed in the divided area 51 of the screen 50 of the computer 24 and the current micro image is displayed in the divided area 61 of the screen 60. A macro image is displayed. However, instead of these current micro images and macro images, the past micro images and macro images (history images) read out from the external storage device 27 by the controller 40 via the history database of the computer 24 are divided regions 51 and 61. Can also be displayed. The screen 50 and the screen 60 may be divided into one display screen and displayed at the same time, or may be selectively displayed alternately. Or you may display on two display apparatuses, respectively.

  As shown in FIG. 4, past micro images and macro images (history images) are associated with each other via a status file (for example, name: ABC.txt) in the history database of the computer 24 and the external storage device 27. The macro image (for example, name: ABC.bmp) is also associated with a registration number (for example, 1) assigned according to the identification information of the specimen 10A. Incidentally, in this embodiment, one macro image (for example, name: ABC.bmp) and a state file (for example, name: ABC.txt) are associated with one registration number (for example, 1), and a large number of micro images (for example, name: ABC.txt). For example, name: DEF001.bmp) is associated.

  As described above, in the history database of the computer 24 and the external storage device 27, registration numbers are assigned so as not to overlap each other according to the identification number of the specimen 10A, and the registration is performed for each registration number (that is, for each specimen). The number, the macro image, and the micro image are associated with each other. The association between the registration number, the macro image, and the micro image is equivalent to the association between the identification information of the specimen 10A, the macro image, and the micro image. The controller 40 automatically assigns the registration number.

  Here, the file names of the macro image and the micro image (for example, ABC.bmp, DEF001.bmp, etc.) are names arbitrarily determined by the controller 40 using alphanumeric characters so as not to overlap each other. The date and time of registration may be used as the file name. In addition, the file name is not limited to the example in which the file name is automatically assigned in this way, but the observer may give the file name. The extension of the file name is not limited to “bmp”, and other formats may be used. Further, the controller 40 automatically associates the state file name (for example, ABC.txt) with the macro image file name (for example, ABC.bmp) (the same name with the extension “txt”). decide.

  Further, as shown in FIG. 5, the status file (for example, name: ABC.txt) includes a micro image file name (for example, name: DEF001.bmp), a shooting position (xy position of the stage unit 11), and shooting conditions. This file associates (magnification, aperture amount, focal position, illumination brightness) and information on observation history (operation type, comment, date, time, observer name) for the specimen 10A. Of the items in the state file, the controller 40 automatically writes the items other than the comments included in the observation history information. The comment is information input by the observer using the keyboard 24A or the like.

  Of the items in the state file, the item essential for associating the micro image with the macro image is information on the photographing position (xy position of the stage unit 11). If the coordinate system of the stage unit 11 and the coordinate system of the sample 10A always coincide with each other in the imaging position (xy position of the stage unit 11) as in the present embodiment, it is not necessary to perform any coordinate conversion processing on the macro image. This is considered to represent the position of the micro image. Therefore, by associating the macro image and the micro image together with the photographing position (xy position of the stage unit 11), it is possible to always grasp a partial region of the macro image corresponding to the micro image.

4 and 5, when the association information is already stored in the history database of the computer 24 and the external storage device 27, the past macro image or micro image (history image) read out by the controller 40 therefrom. ) Are displayed in the divided areas 51, 61, 62, 64 of the screens 50, 60 (FIG. 3) of the computer 24, respectively.
Specifically, a list of macro images is displayed in the divided area 62, one macro image selected from the list is displayed in the divided area 61, and a list of micro images associated with the macro image is divided. It is displayed in area 64. Further, a partial area of the macro image corresponding to each micro image in the list of the divided areas 64 (that is, the shooting position of each micro image) is displayed in the divided area 61 as rectangular frames 66, 67,. Further, one micro image selected from the list of the divided areas 64 is enlarged and displayed in the divided area 51 of the screen 50.

  Further, the sample information (registration number, etc.) of the macro image selected from the list of the divided areas 62 and displayed in the divided area 61 is displayed in the divided area 63. Patient information and comments included in the sample information are information input by the observer using the keyboard 24A or the like. Further, information on the micro image selected from the list of the divided areas 64 and displayed in the divided area 51 (such as a comment of the state file shown in FIG. 5) is displayed in the divided area 65.

  The micro images displayed in the list in the divided area 64 are all the micro images (all the micro images associated with the macro image in the divided area 61) stored in the history database of the computer 24 and the external storage device 27. However, if classification is performed using items included in the state file of FIG. 5 (titles 1, 2,... Of the divided area 69), only a part of the micro images can be displayed in the divided area 64.

  By the way, the divided area 52 of the screen 50 is a display area such as an operation menu, and indicates whether the specimen 10A is placed on the stage unit 11 (online) or not (offline). The divided area 52 is provided with a load button 53, a shooting button 54, a macro reading button 55, a reproduction button 56, a proximity shooting button 57, and an input box 58.

  When the observer clicks the buttons (53 to 57) with a mouse or the like, various instructions are input from the input device 25 to the controller 40. Then, the controller 40 outputs a control signal to each circuit (41 to 47) of the control unit 23 according to an instruction from the outside, and electrically controls each unit of the microscope system 10. In addition, files and data are exchanged with the history database of the computer 24 and the external storage device 27. Further, when the observer inputs a comment (such as a character or a symbol) in the input box 58, the controller 40 stores the stored content (such as the status file in FIG. 5) in the external storage device 27 via the history database of the computer 24. Update.

Next, the operation of the microscope system 10 of this embodiment will be described using the flowcharts of FIGS. When the processing of the flowchart of FIG. 12 is described, reference is made to FIG.
When power is turned on to the microscope system 10 and the observer logs in to the microscope system 10, the controller 40 initializes each part of the microscope system 10 and starts control according to the flowcharts of FIGS. At this time, in the divided area 62 of the screen 60 of the computer 24, a history database of the computer 24 and a list of macro images already stored in the external storage device 27 are displayed.
(Button input detection processing)
First, the button input detection process (S1 to S10) shown in FIG. 6 will be described.

In each of steps S1 to S5, the controller 40 monitors whether or not the load button 53, the shooting button 54, the macro reading button 55, the reproduction button 56, and the proximity shooting button 57 in FIG. 3 have been pressed.
Then, when the load button 53 is pressed (S1 is Yes), the load process of step S6 (details are shown in FIG. 7) is executed. When the photographing button 54 is pressed (S2 is Yes), the registration process in step S7 (details are shown in FIG. 9) is executed. When the macro image reading button 55 is pressed (Yes in S3), the macro image reading process (detailed in FIG. 10) in step S8 is executed. When the reproduction button 56 is pressed (S4 is Yes), the reproduction process of step S9 (details are shown in FIG. 11) is executed. When the close-up shooting button 57 is pressed (S5 is Yes), the automatic history registration process (detailed in FIG. 12) in step S10 is executed.

It should be noted that both the registration process when the photographing button 54 is pressed (details are shown in FIG. 9) and the automatic history registration process when the proximity photographing button 57 is pressed (details are shown in FIG. 12), the registration of the micro image of the specimen 10A. Is done. However, in order to clarify the difference, the micro image registered using the registration process (details are shown in FIG. 9) is flagged as “registered by an observer” and the automatic history registration process (details is shown in FIG. 12). The same flag is not set for the micro image registered using.
(Load processing)
Next, the load process (S11 to S25) in FIG. 7 will be described.

When the load button 53 is pressed, the controller 40 discharges the stage unit 11 from the opening 26A of the housing 26 to the outside in step S11 (state of FIG. 2). At this time, the observer can place the specimen 10 </ b> A on the stage unit 11. In addition, when the specimen 10A is already placed, the specimen 10A can be removed from the stage portion 11.
Thereafter, when the load button 53 is pressed again (Yes at Step S12), the controller 40 takes the stage unit 11 into the opening 26A (Step S13). At this time, the specimen 10A is also taken in together with the stage unit 11. Alternatively, only the stage portion 11 that has become empty after the sample 10A is removed is taken into the interior.

Further, when the stage unit 11 is taken into the housing 26, the controller 40 detects whether there is a preparation (specimen 10A) on the stage unit 11 according to the output of the sensor 39A (step S14). If there is no preparation (sample 10A) (No in step S14), the loading process is terminated and the process returns to the button input detection process of FIG.
On the other hand, when there is a preparation (sample 10A) (step S14 is Yes), the controller 40 reads the barcode on the preparation based on the output of the sensor 39B, and acquires the identification information of the sample 10A (step S15). Then, an inquiry is made to the history database (see FIG. 4) of the computer 24 to determine whether there is a registration number corresponding to the identification information of the sample 10A (step S16). When the registration number is not in the history database (S16 is No), the controller 40 proceeds to the process of step S17 in order to newly register the specimen 10A placed on the stage unit 11.

  In step S17, the illumination light source 16 is turned on, the mirror 18 is retracted, and the passage hole 37 of the reduction lens unit 20 is inserted into the observation optical path 10D. When the specimen 10A reaches the observation optical path 10C, the specimen 10A is shot in two shots, and a macro image is captured. The macro image is temporarily stored in the frame memory and displayed in the divided area 61 of the computer 24 (step S18).

  Next, a new registration number is assigned according to the identification information of the specimen 10A acquired in step S15 (step S19), and a file name (for example, ABC.Bmp) is assigned to the macro image displayed in the divided area 61. At step S20, the two are associated and stored in the history database of the computer 24 and the external storage device 27 (FIG. 4). In the next step S21, a state file (for example, name: ABC.Txt) as shown in FIG. 5 is created and associated with the above macro image and saved in the history database of the computer 24 and the external storage device 27 (FIG. 5). 4).

  As a result of the processing of steps S17 to S21, in the history database of the computer 24 and the external storage device 27, a new registration number, a macro image file (for example, name: ABC.Bmp), and a status file (for example, name) : ABC.txt) and is newly registered. At this time, the number of registered micro images is still 0, and there is no writing in each item of the status file. As a result of such new registration, a new macro image is added to the list of divided areas 62, and sample information (such as a registration number) is displayed in the divided area 63.

  On the other hand, when it is determined in step S16 that the registration number corresponding to the identification information of the specimen 10A is in the history database (S16 is Yes), the controller 40 executes steps S22 to S24. In the processing of steps S22 to S24, when the specimen 10A placed on the stage unit 11 has been registered, past macro images and micro images (history images) are stored from the external storage device 27 via the history database of the computer 24. This corresponds to the process of reading / displaying.

First, in step S22, a macro image associated with a registration number is read. Then, the read macro image is displayed in the divided area 61 of the computer 24 (step S23). At this time, the sample information (registration number, etc.) of the read macro image is displayed in the divided area 63. Thereafter, in step S24, a past micro image reading process associated with the read macro image (for details, S31 to S39 in FIG. 8) is executed. Note that step S25 will be described after the following description of FIG. 8 (that is, detailed description of step S24).
(Micro image reading process)
In step S31 of FIG. 8, a list of micro images associated with the read macro image is read. If the number of items in the list (that is, the number of past micro-image registrations) is 0 (No in step S32), this reading process ends. In this case, nothing is displayed in the divided area 64 of the screen 60 of the computer 24, and the state is the same as after the new registration (S17 to S21) is completed.

  If the number of items in the list (that is, the number of past micro images registered) is 1 or more (Yes in step S32), the file name related to the first micro image in the list is acquired, and the micro image is acquired from the external storage device 27. Read (step S33). In the next step S34, referring to the state file (FIG. 5), shooting information (shooting position, shooting conditions, and observation history) associated with the read micro image is read.

  In the next step S35, based on the read photographing information, whether or not a flag “observer has registered” is set in the micro image (that is, the micro image registered by using the registration process of FIG. 9 described later). Whether or not). If the flag is set as a result of the determination (Yes in S35), the micro image read in step S33 is displayed as a thumbnail in the divided area 64 of the screen 60 of the computer 24 (step S36). In addition, a comment associated with the micro image is displayed in the divided area 65. If the flag is not set (S35 is No), the micro image is registered using the automatic history registration process of FIG. 12 described later, and the thumbnail is not displayed.

  Thereafter, based on the information of the photographing position (xy position of the stage unit 11) of the photographing information read in step S34, the position of the micro image read in step S33 is superimposed on the macro image of the divided region 61 to form a rectangular frame ( For example, a rectangular frame 66) is displayed (step S37). At this time, in order to distinguish between a micro image displayed as a thumbnail in the divided area 64 (registered in a registration process in FIG. 9 described later) and a micro image not displayed in a thumbnail (registered in an automatic history registration process in FIG. 12 described later), for example, It is preferable that the rectangular frame has a different color.

  In step S38, it is determined whether or not reading of all items in the list of micro images has been completed. If there is a micro image that has not yet been read (No in S38), the next micro image in the list is determined in step S39. Read from the external storage device 27. And the process of step S34-S38 similar to the above is repeated. If all the micro images in the list have been read (Yes in S38), this reading process is terminated. This completes the process of step S24 of the load process of FIG.

  As a result of performing the processes of steps S22 to S24, a macro image having a registration number corresponding to the identification information of the specimen 10A placed on the stage unit 11 is displayed in the divided region 61 of the screen 60 of the computer 24. If there is a past micro image (registered in the registration process of FIG. 9 described later) associated with the macro image, the thumbnail is displayed in the divided area 64. In the divided area 63, sample information (registration number, etc.) of the macro image of the divided area 61 is displayed.

  As described above, in the load process of FIG. 7, depending on whether or not the registration number corresponding to the identification information of the specimen 10 </ b> A exists in the history database of the computer 24, the process of steps S <b> 17 to S <b> 21 or Then, the process proceeds to step S25. In step S25, when the stage unit 11 further moves and the specimen 10A reaches the observation optical path 10B, the controller 40 turns on the illumination light source 12, inserts the mirror 18 on the observation optical path 10D, and one of the specimens 10A. The observation state of the partial area is assumed. At this time, the controller 40 captures the specimen 10 </ b> A, captures a micro image, and displays it on the divided area 51 of the computer 24.

When the load process of FIG. 7 is completed, the controller 40 returns to the button input detection process of FIG. In this state, the controller 40 electrically controls each part of the microscope system 10 according to an instruction from the outside, and performs imaging of the specimen 10A while changing the imaging conditions and the imaging position of the micro image displayed in the divided area 51. , Update the micro image. Note that the micro image shooting position can be specified by clicking in the divided area 61 (macro image) by clicking on a portion to be enlarged and observed.
(registration process)
Next, the registration process (S41 to S46) of FIG. 9 will be described.

When the photographing button 54 is pressed, the controller 40 determines in step S41 whether or not the number of registrations for the specimen 10A on the stage unit 11 has already reached the maximum (the limit value of the history database of the computer 24). . Only when there is still room for registration (No in S41), the processing from step S42 is performed.
In step S42, a file name (for example, DEF001.bmp) is assigned to the micro image displayed in the divided area 51, and the registration number acquired in step S15 in FIG. 7 is associated with the macro image displayed in the divided area 61. Then, it is stored in the history database of the computer 24 and the external storage device 27 (FIG. 4). At that time, a digital watermark is added to the micro image to prevent falsification.

  Further, in step S43, the shooting position of the micro image (the xy position of the stage portion) and the shooting conditions (the magnification of the objective lens portion, the magnification of the reduction lens portion, the aperture amount, the focal position, the illumination brightness) are set for each circuit ( 41 to 47) and sensors 34 and 38. Then, the micro image file name, shooting position, and shooting conditions are written in the state file (for example, name: ABC.txt) associated with the registration number acquired in step S15 of FIG.

  In step S44, the information of the observation history (operation type, comment, date, time, observer name) for the specimen 10A is updated and displayed in the divided area 65. When updating the observation history information, a person who can write with a password or the like is specified to prevent tampering. Of the observation history information, the type of operation (for example, registration of a micro image), date, time, and observer name are information that the controller 40 automatically writes. The date and time are, for example, the date and time when the shooting button 54 is pressed. An observer name is a login name to the microscope system 10, for example. On the other hand, the comments included in the observation history information are information such as characters and symbols input by the observer to the input box 58 of the computer 24 using the keyboard 24A or the like.

  Next, the controller 40 displays a thumbnail of the micro image stored in step S42 in the divided area 64 of the screen 60 of the computer 24 (step S45). Further, based on the information on the photographing position (xy position of the stage unit 11) acquired in step S43, the position of the micro image displayed in the divided area 51 is superimposed on the macro image of the divided area 61 (for example, a rectangular frame). A frame 66) is displayed (step S46).

  When the registration process of FIG. 9 is completed in this way, the controller 40 returns to the button input detection process of FIG. Each time the photographing button 54 is pressed, the registration process (S41 to S46) in FIG. 9 is repeated to add a list of micro images. That is, the micro image (enlarged image) is not sequentially captured in the entire region of the sample 10A as in the conventional sample scanning system, but the micro image (observation) displayed in the divided region 51 when the photographing button 54 is pressed. Only the micro-image of the part that the person watched).

  In the microscope system 10 of the present embodiment, the observer decides the gaze position of the specimen 10A by himself / herself, captures a micro image of the position in the history database of the computer 24 and the external storage device 27, and is not particularly important. Since the micro image is not stored, the sample 10A can be efficiently imaged in a short time compared to the conventional sample scanning system, and the data amount of the generated image can be reliably reduced.

  Such an effect is the same not only when a new specimen 10A is placed on the stage unit 11, but also when a specimen 10A on which one or more micro images have already been registered is placed again on the stage part 11. is there. In this case, based on the registration number corresponding to the identification information of the specimen 10A acquired in step S15 of FIG. 7 described above, an external storage device is obtained via the history database of the computer 24 by the processing of subsequent steps S22 to S24. From 27, past macro images and micro images (history images) are read out.

  Then, when the shooting button 54 is pressed (in accordance with an instruction from the outside), the micro image displayed in the divided area 51 (the micro of the portion watched by the observer) is displayed by the registration processing (S41 to S46) of FIG. Image) is added to the history database of the computer 24 and the external storage device 27 in association with the registration number corresponding to the identification information of the specimen 10A acquired in step S15 of FIG. 7 and the macro image displayed in the divided area 61. To remember (additional). Therefore, even in the case of remounting, the specimen 10A can be efficiently imaged in a short time, and the data amount of the generated image can be reliably reduced.

As described above, in the microscope system 10 of the present embodiment, when the specimen 10A is imaged, only necessary portions can be stored as a high-definition micro image when necessary. It is possible to return to the specimen 10A, and it is possible to additionally store only a newly required part as a high-definition micro image. The stored data is a high-definition micro image of the portion watched by the observer and a macro image (same size image) of the specimen 10A, and the data amount does not become enormous as in the conventional specimen scanning system.
(Macro image reading process)
Further, by imaging an important part of the specimen 10A (the part watched by the observer) in association with the whole image of the specimen 10A, virtual observation is possible even when the valuable specimen 10A is not at hand. Become. Next, virtual observation in a state where the specimen 10A is not placed on the stage unit 11 (offline) will be described. This observation also applies to a case where a specimen different from the specimen 10A placed on the stage unit 11 is virtually observed.

When the observer selects one macro image from the list displayed in the divided area 62 of the screen 60 of the computer 24 and presses the macro read button 55 in the divided area 52, the controller 40 displays the macro image shown in FIG. Read processing (S51 to S57) is executed.
In step S51, in order to recognize the macro image selected in the list of the divided areas 62 when the macro reading button 55 is pressed, a registration number associated with the macro image is acquired from the input device 25. Next, in step S52, it is detected whether or not there is a preparation (sample 10A) on the stage unit 11. If there is no preparation (sample 10A) (No in step S52), the process proceeds to step S54 without performing the process in next step S53.

  If there is a preparation (sample 10A) (Yes in S52), the process proceeds to step S53, and the registration number (selection number) acquired in step S51 is the registration number (sample number) corresponding to the identification information of the sample 10A. To determine whether or not. If the two registration numbers match (No in S53), the desired macro image is already displayed in the divided area 61, and a list of micro images associated with the macro image is also displayed in the divided area 64. End the reading process.

  On the other hand, if the two registration numbers are different (S53 is Yes) and there is no preparation (specimen 10A) on the stage unit 11 (S52 is No), the process proceeds to step S54 and the registration number acquired in step S51. Is read from the external storage device 27 via the history database of the computer 24. Then, the read macro image is displayed in the divided area 61 of the computer 24 (step S55). At this time, the sample information (registration number, etc.) of the read macro image is displayed in the divided area 63.

  Thereafter, in step S56, a past micro image reading process (S31 to S39 in FIG. 8) associated with the read macro image is executed. As a result, the past micro image (registered in the registration process of FIG. 9) associated with the macro image is displayed as a thumbnail in the divided area 64. In this state, when the observer selects one micro image from the list of divided areas 64, the micro image can be displayed in the divided area 51.

  Further, when the observer inputs a comment (such as a character or a symbol) into the input box 58 of the computer 24 using the keyboard 24A or the like, the controller 40 monitors the observation history associated with the macro image of the divided area 61 in step S57. Information (see FIG. 5) is updated. Further, when the observer artificially moves up / down, left / right, focus, magnification, etc., the information of the observation history is updated based on the operation. It is preferable to add a flag to such an additional history to distinguish it from the online observation history (S44 in FIG. 9).

  In a pseudo operation (magnification change, etc.), a micro image may be displayed by digital enlargement of the macro image in a region where the micro image has not been acquired, or the micro image may not be displayed due to software restrictions. It may be. Further, it is preferable to prevent modification of a macro image or a micro image (history image) stored in the external storage device 27 during such virtual observation.

  As described above, by executing the macro image reading process (S51 to S57) of FIG. 10, it is possible to virtually observe a high-definition micro image even when the precious specimen 10A is not at hand. . Furthermore, even when observing a certain specimen 10A (during imaging), a history image of another specimen can be read out and displayed (shift from online to offline), so that the microscope system 10 can perform the specimen 10A. This is a practical microscope system 10 without being exclusively used for imaging.

Conversely, the transition from offline to online can be performed smoothly. In this case, after the macro image reading process (S51 to S57) in FIG. 10, the controller 40 prompts the observer to specify the identification information of the necessary specimen 10A and prompt the user to load the necessary specimen 10A. The load process (FIG. 7) may be started. Alternatively, the loading process (FIG. 7) of the specimen 10A that is unilaterally required by the observer may be started. In any case, the transition from offline to online can be performed smoothly, and the practical microscope system 10 is obtained.
(Reproduction process)
Furthermore, in the microscope system 10 of the present embodiment, the shooting position (xy position of the stage unit 11) written in the state file and the shooting conditions (magnification of the objective lens unit, magnification of the reduction lens unit, aperture amount, focal position, (Luminance brightness) can also be reproduced.

When the observer selects one micro image from the list displayed in the divided area 64 of the screen 60 of the computer 24 and presses the reproduction button 56 of the divided area 52, the controller 40 performs the reproduction process (S61 in FIG. 11). To S72).
In step S61, in order to recognize the micro image selected in the list of the divided areas 64 when the reproduction button 56 is pressed, the list number associated with the micro image is acquired from the input device 25. In step S62, a state file (for example, name: ABC.txt) associated with the selected micro image is read from the history database of the computer 24. In step S63, a registration number associated with the read state file is acquired.

  Next, in step S64, it is determined whether or not there is a preparation (sample 10A) on the stage unit 11. When there is a preparation (specimen 10A) (S64 is Yes), the process proceeds to step S65, and the registration number (selection number) acquired in step S63 is the same as the registration number (specimen number) corresponding to the identification information of the specimen 10A. Determine whether or not. If the two registration numbers match (S65 is Yes), the process proceeds to step S66.

  In step S66, with reference to the list number acquired in step S61 in the already read state file, the shooting position (xy position of the stage unit 11) and shooting conditions (magnification of the objective lens unit, reduction lens unit) Read magnification, aperture, focus position, illumination brightness). For example, referring to the list number “2” of the status file (name: ABC.txt) shown in FIG. 5, magnification = 10, aperture amount = 75, x coordinate = 1000, y coordinate = 2000, focus position = 3000, A set of setting states of illumination brightness = 180 is read.

Next (step S67), the controller 40 controls each circuit (41 to 47) based on the photographing position and photographing conditions read from the state file (name: ABC.txt), and controls each part of the microscope system 10. It is actually driven electrically. That is, the setting state of each part of the microscope system 10 is reproduced.
Specifically, the stage control circuit 43 is controlled based on the read “x coordinate, y coordinate” to reproduce the photographing position. Further, the objective lens driving circuit 45, the reduction lens driving circuit 46, and the CCD control circuit 47 are controlled based on the read “magnification”, and the aperture control circuit 42 is controlled based on the “aperture amount”, so that the “focus position”. The focus control circuit 44 is controlled based on the above, and the illumination control circuit 41 is controlled based on “illumination luminance” to reproduce the photographing conditions.

  Thereafter, the controller 40 captures the specimen 10A, captures a micro image, and displays it on the divided area 51 of the computer 24 (step S68). Therefore, the observer can observe the image of the same position and the same brightness of the sample 10A again without repeating complicated adjustment work. When the reproduction process of FIG. 11 is completed in this way, the controller 40 returns to the button input detection process of FIG.

On the other hand, when there is no preparation (sample 10A) (S64 is No) and there is a preparation (sample 10A) but the sample number is different from the selection number (S65 is No), the above steps S66 to 68 are performed. Since “reproduction” cannot be performed, the process proceeds to step S69.
In step S69, the macro image associated with the registration number (selection number) acquired in step S63 is read from the history database of the computer 24 and the external storage device 27. Then, the read macro image is displayed in the divided area 61 of the computer 24 (step S70). At this time, the sample information (registration number, etc.) of the read macro image is displayed in the divided area 63.

In step S 71, the selected micro image is read from the history database of the computer 24 and the external storage device 27 and displayed in the divided area 51 of the computer 24. In step S72, other micro image reading processing (S31 to S39 in FIG. 8) related to the micro image is executed. As a result, other micro images are displayed as thumbnails in the divided area 64, and a rectangular frame is also displayed in the divided area 61. For this reason, it is possible to observe the selected micro image together with other micro images.
(Automatic history registration process)
Further, in the microscope system 10 of the present embodiment, the automatic history registration process (see FIG. 12) is performed using the free time that is not used for imaging (FIG. 9) or virtual observation (FIG. 10) of the specimen 10A. Steps S81 to S93) are executed to reinforce the history database of the computer 24. This process may be started by automatically detecting that the observer is not operating, or may be started when the vicinity photographing button 57 in FIG. 3 is pressed by the observer.

In step S81, it is determined whether or not there is a preparation (sample 10A) on the stage unit 11. If there is no preparation (sample 10A) (No in S81), the automatic history registration process is terminated.
On the other hand, when the preparation (specimen 10A) is on the stage unit 11 (Yes in S81), the process proceeds to step S82, and the macro image associated with the registration number corresponding to the identification information of the specimen 10A is stored in the history database of the computer 24. Read from. And it displays on the division area 61 of FIG.

  Next (step S83), a state file (for example, name: ABC.txt) associated with the read macro image is read from the history database of the computer 24, and a list of micro images associated with the read macro image is displayed. Read (step S83). If the number of items in the list (that is, the number of past micro-image registrations) is 0 (No in step S84), the automatic history registration process ends.

  If the number of items in the list (that is, the number of past registered micro images) is 1 or more (Yes in step S84), the top micro image in the list is read as a gaze image (step S85). In the next step S86, referring to the state file (FIG. 5), the shooting position (xy position of the stage unit 11) associated with the gaze image and shooting conditions (magnification of the objective lens unit, magnification of the reduction lens unit, aperture stop) Volume, focus position, illumination brightness). And each circuit (41-47) is controlled based on the imaging | photography position and imaging | photography condition, and each part of the microscope system 10 is actually driven electrically. That is, the setting state of each part of the microscope system 10 is reproduced.

  Next (step S87), the controller 40 uses the field angle Δx in the x direction, the field angle Δy in the y direction, and the depth of focus Δz (see FIG. 13) determined by the optical system of the microscope system 10 to perform the current photographing. A slight shift from the position (xy position of the stage unit 11) and a slight shift from the current focal position (z position of the objective lens unit 17) are performed. That is, the shooting position and the focus position are finely adjusted. Then, an imaging region is set near the periphery of the position of the gaze image reproduced in step S87.

  For example, as shown in FIG. 13, the vicinity of the position 70 of the gaze image includes two positions 71 and 74 shifted from the position 70 in the y direction by the angle of view Δy, and an angle of view Δx from the position 70 in the x direction. And two positions 75 and 76 shifted by the depth of focus Δz in the z direction from the position 70 are considered (total of six positions). As a first stage, a shooting area may be set at each of these six locations (positions 71 to 76), and the processing after step S88 may be performed. Further, the steps may be further advanced to set a shooting region at each of the positions 77 to 80 whose distance from the position 70 of the gaze image is away from the positions 71 to 76 (second stage), or each of the positions 81 to 84. An imaging region may be set in (third step).

  In step S88, it is determined whether or not a micro image of the nearby shooting region set in step S87 is already registered as a history image. If registered (S88 is Yes), shooting at that position is not performed. If it has not been registered (No in S88), the process proceeds to step S89, where a micro image of the neighboring region is captured by photographing, and is added to the external storage device 27 and stored in association with the macro image of the divided region 61 and the registration number. Further, the micro image information is registered in the history database of the computer 24, and the status file (for example, name: ABC.txt) is updated.

Also, the shooting position of the micro image is displayed in a rectangular frame so as to overlap the macro image displayed in the divided area 61 of FIG. However, in this case, a rectangular frame of a color different from that of the micro image registered by normal photographing (registration processing in FIG. 9) is used, and further, the thumbnail image of the divided region 64 is not displayed so that it is automatically displayed. Differentiate the captured micro-images.
For the imaging region in the vicinity of the position 70 of the gaze image (for example, positions 71 to 76 in FIG. 13), the processing of S87 to S91 is repeated, and the processing such as imaging and additional storage in the vicinity of the position 70 of the gaze image ends. Then, the process proceeds to next Step S92.

  In step S92, it is determined whether or not the reading of all items in the list of micro images has been completed. If there is a micro image that has not been read yet (No in S92), the next micro image in the list is determined in step S93. Read as a gaze image. Then, the same processing (S86 to S92) as described above is repeated. When all the micro images in the list have been read (Yes in S92), this automatic history registration process is terminated.

  The automatic history registration processing (S81 to S93) in FIG. 12 is executed, and processing such as micro-image capturing and additional storage is automatically performed around each micro-image position (gaze image position 70 in FIG. 13). Thus, when performing the observation indirectly offline later (FIG. 10), not only the position where the observer gazes (position 70 in FIG. 13) but also the vicinity (for example, positions 71 to 76) are included. In addition, a high-definition micro image can be observed over a wide range. When the position in the focus direction is included as the vicinity, a micro image having a three-dimensional structure can be generated. This is suitable for observation of a three-dimensional specimen, a semiconductor wafer, an IC chip, a MEMS, and the like. Further, the same automatic history registration processing may be performed by finely adjusting not only the shooting position and the focal position but also other shooting conditions such as magnification.

  The microscope system 10 of the present embodiment is suitable for medical education performed between a student and a teacher or between a doctor and a laboratory technician. The student (examiner) saves the diagnosis result (macro image, status file, and micro image) of the specimen 10A in the external storage device 27 and the history database of the computer 24. After completion of the diagnosis, the student (examiner) gives the teacher (doctor) the registration number corresponding to the identification information of the specimen 10A.

A teacher (doctor) connects the microscope system 10 of the present embodiment to the history database of the computer 24, and uses a registration number taught by a student (examiner) to register already registered macro images and micro images (history). By reading out (image), it is possible to know which position of the specimen 10A was gaze-diagnosed even without the specimen 10A.
Further, when the teacher (doctor) receives the specimen 10A from the student (examination technician) and observes it using the microscope system 10, the teacher selects a location (micro image) where the specimen 10A is diagnosed. Thus, the same imaging position and imaging conditions can be reproduced and the specimen 10A can be observed again, and the diagnosis result of the student (examiner) can be easily reproduced. Further, it can be verified later whether or not the diagnosis result of the student (examiner) is appropriate.

Furthermore, at that time, the teacher (doctor) takes a picture of other positions that should have been closely watched, and the micro image is added to the history database of the computer 24 and saved, so that the student (examination engineer) later. However, it is also possible to check the history database using the microscope system 10.
In addition, since the diagnosis location, the diagnosis date and time, and the name of the person diagnosed in each specimen can be recorded as history information, verification is possible when problems such as oversight or misdiagnosis occur.
(Adjustment of the slide placement position)
In the present invention, when a new preparation is inserted and image recognition is performed from identification information (for example, a photographed macro image) of the preparation (specimen 10A) and it is determined that the sample exists in the history data pace, the image is already photographed. It is preferable to correct the deviation and rotation of the position where the preparation is placed so that the position of the prepared micro image can be accurately reproduced.

With reference to FIG. 14 and FIG. 15, the adjustment of the displacement of the placement position of the preparation will be described. The size of the preparation and the size of the cover glass are standardized, and it is assumed that the upper left position recognition marker 101 and the lower right position recognition marker 102 exist at specific positions on the preparation.
First, the stage is moved to the position of the upper left position recognition marker 101 on the preparation. Therefore, an image is taken and it is confirmed that the upper left position recognition marker 101 exists on the image. The confirmation method can be performed by storing the shape image of the upper left position recognition marker 101 on a computer and determining the upper left position recognition marker 101 when the difference from the image is sufficiently small.

If the presence of the upper left position recognition marker 101 cannot be confirmed, the upper left position recognition marker 101 is searched for by capturing an image while moving the stage by half the captured image size left and right and up and down.
Next, the stage is moved by a small amount so that the upper left position recognition marker 101 becomes the center of the image. At this time, the above confirmation method is used to confirm whether or not the center has been moved.

Thereafter, the stage is moved in the X and Y directions by the difference amount between the positions specified by the upper left position recognition marker 101 and the lower right position recognition marker 102. Therefore, an image is taken and it is confirmed that the lower right position recognition marker 102 exists on the image. Also in this case, the above confirmation method is used to confirm whether or not the center has been moved.
Therefore, a deviation value of the slide position and a correction value for rotation are calculated from the difference amount recognized as the position of the lower right position recognition marker 102 and the original default difference amount. For example, when moving in parallel, the upper left position recognition marker 101 that should be at the stage position X = 10, Y = 8 when the center of the image is reached is X = 11, Y = 9 (right It is recognized that it has moved down one by one). From this data, X = −1 and Y = −1 are stored as parallel movement correction values and used when moving the stage.

  Further, in the case of rotational movement, the difference amount between the positions of the upper left position recognition marker 101 and the lower right position recognition marker 102 is clockwise when XO = 80 and YO = 60 in the ideal data. If it is rotated several degrees, X1 = 79 and Y1 = 61. The amount of rotation can be recognized by solving an equation using a rotation matrix from these data. This data is used as the stage rotation correction amount.

  In the above example, the upper left position recognition marker 101 and the lower right position recognition marker 102 are drawn on the slide (FIG. 14). However, as shown in FIG. And the positions of the upper left vertex 105 and the lower right vertex 106 of the preparation as shown in FIG. 17 may be used as markers by using the edge enhancement processing and the lining processing of the image.

Further, when the micro image is read, the difference amount between the history image and the captured image at the moment is calculated, and the parallel movement and the rotation amount of the stage can be renewed to further improve the accuracy.
(Modification)
In the above-described embodiment, the example in which the identification barcode is pasted on the slide portion of the specimen 10A as a seal and the identification information of the specimen 10A is acquired has been described. However, if the specimen 10A can be associated with the macro image. Any method can be used. For example, a method of binarizing a macro image and recording it as unique data (including a method of engraving a pattern in a prepared portion and a method of recognizing an image by pasting a handwritten memo) can be considered. Further, it is conceivable that the specimen 10A can be uniquely identified by attaching a recording medium such as an IC chip or a magnetic material to the preparation portion and providing means for reading out the data.

  Further, in the above-described embodiment, in order to simplify the description, the position of the specimen 10A on the stage unit 11 is always the same (even when re-mounted), but the present invention is not limited to this. . When the position (including rotation) on the stage unit 11 changes every time the sample 10A is placed, the relative position between the sample 10A and the stage unit 11 on the stage unit 11 (that is, the stage unit 11). It is preferable to associate the micro image with the macro image in consideration of the amount of positional deviation between the coordinate system and the coordinate system of the specimen 10A.

  For example, when the specimen 10A is placed on the stage unit 11, the positional deviation amount XYθ of both coordinate systems is detected, and when a micro image is registered, its photographing position (xy position of the stage unit 11). In addition to the above, it is conceivable to write the positional deviation amount XYθ of the coordinate system in the state file. Further, the position of the micro image on the macro image is obtained by correcting the photographing position of the micro image (xy position of the stage unit 11) by the positional deviation amount XYθ of the coordinate system (that is, performing coordinate conversion processing). Could be written to the state file.

  When detecting the positional deviation amount XYθ of the coordinate system, for example, an edge (corner) of a preparation portion, an identification barcode or label, or the like can be used as the origin position of the specimen 10A. For detecting the displacement amount XYθ, for example, a half macro image of the sample 10A is captured, the edge (or barcode) of the preparation portion is detected by image processing, and pattern recognition is performed, whereby the sample 10A is detected. A method of grasping the coordinate system can be considered.

  Furthermore, instead of the macro image, the same shooting position (xy position of the stage unit 11) is always revisited to capture a high-magnification micro image, and the micro image is captured while moving the stage unit 11 around the micro image. The positional deviation amount XYθ may be obtained by performing image matching with the image. Further, the positional deviation amount XYθ may be always obtained with reference to a fixed point of the stage coordinate system, or may be obtained with reference to the previous sample coordinate system.

  Furthermore, in the above-described embodiment, an example in which a macro image is captured only once (only when newly registered) for one specimen 10A has been described, but the present invention is not limited to this. Each time the sample 10A is reloaded (every time), a macro image may be captured and its history may be saved. In this case, a newly captured macro image is displayed instead of the macro image associated with the identification information of the specimen 10A.

In the above-described embodiment, the example in which the micro image is explicitly stored by pressing the photographing button 54 (registration instruction) has been described, but the present invention is not limited to this. You may make it memorize | store a micro image automatically, when operation (instruction from the outside), such as a comment input, is performed in the place which an observer is worried about.
Furthermore, in the above-described embodiment, an example has been described in which an observer inputs characters, symbols, and the like (comments) from the keyboard 24A of the computer 24 as observation history information, but the present invention is not limited to this. In addition to such text data, audio data may be stored as a comment of history information.

  In the above-described embodiment, the example in which the microscope system 10 and the history database of the computer 24 are connected on a one-to-one basis has been described. However, the present invention is not limited to this. For example, a configuration is possible in which a plurality of microscope systems 10 are connected to one computer 24 (history database), and photographing information is registered from each microscope system 10 in a common history database. In that case, it is preferable to add identification information (such as a serial number) of the microscope system 10 to the observation history information. This makes it possible to determine later which microscope system 10 has been used to register a new (or additional) micro image.

Further, in the above-described embodiment, the example in which the two screens 50 and 60 of the computer 24 are used has been described. However, the present invention can be applied even if the number of screens to be used is one.
In the above-described embodiment, the external storage device 27 is used to store the macro image file, the status file, and the micro image file. Instead, the same data as the history database of the computer 24 is used. Also good.

Furthermore, in the above-described embodiment, the file name is assigned on the assumption that the macro image or the micro image is used on a personal computer or the like. However, if the use is not assumed, the file name can be omitted. Then, a macro image or a micro image may be stored at a predetermined address in a memory in the microscope system 10.
In the above embodiment, the box-shaped microscope system 10 with a built-in CCD camera has been described as an example. However, the present invention can also be applied to a general microscope system. As a general microscope, a configuration in which an objective lens can be replaced, a configuration in which a CCD camera can be attached and detached, and the like can be considered.

Further, in the above-described embodiment, the example in which the stage unit 11 is moved with respect to the fixed imaging system (that is, the imaging unit (17 to 21) and the CCD camera 22) has been described. On the other hand, the imaging system (17-22) may be moved, or both the stage unit 11 and the imaging system (17-22) may be relatively moved.
In the above-described embodiment, an example of the microscope system 10 that observes the specimen 10A with transmitted illumination has been described. However, the present invention can also be applied to a microscope system with reflected illumination (epi-illumination).

  Furthermore, in the above-described embodiment, the microscope system 10 without an eyepiece lens has been described as an example, but the present invention is not limited to this. The present invention may be applied to a microscope system capable of observing a specimen using an eyepiece as in a normal microscope. In this case, for example, the mirror 18 may be replaced with a half mirror, the observation optical path 10B may be divided into two optical paths by the half mirror, the CCD camera 22 may be disposed on one optical path, and the eyepiece may be disposed on the other optical path. Visual observation with an eyepiece is highly sensitive and can respond to fast changes (stage movement). Also in this case, the history of the sample image visually observed can be left as a micro image or a comment.

In the above-described embodiment, the presence or absence of the preparation on the stage unit 11 is detected by the preparation holder 11A and the sensor 39A, but the present invention is not limited to this. The macro image acquisition operation may be performed regardless of the presence or absence of the preparation, and the presence or absence of the preparation on the stage unit 11 may be detected depending on whether or not the macro image has actually been acquired.
Furthermore, in the above-described embodiment, the preparation itself is determined, and if it is a preparation that has already acquired a macro image and a micro image, it is already registered in the existing image recording file without generating a new image recording file. The example in which a new micro image is added and stored in association with the macro image has been described. Although micro images may be stored one by one, as shown in FIG. 18, when a plurality of micro images are adjacent to each other, the micro images are combined by tiling processing and stored as one micro image. You may let them. Furthermore, after that, when the same preparation is refilled into the microscope system and a new micro image is acquired (FIG. 19), if it is adjacent to the previously stored micro image, the past micro image and the new The micro images are preferably combined by tiling processing and stored as one micro image.

Claims (12)

In a microscope system including a microscope and an imaging device that images a sample on the stage of the microscope,
Obtaining means for obtaining identification information of the specimen on the stage;
Storage means for storing the macro image and the micro image of the specimen imaged by the imaging device in association with the identification information of the specimen acquired by the acquisition means;
Based on the identification information of the sample, it is determined whether or not it is a registered sample that has already been stored in the storage means. A microscope system comprising: an additional recording unit that associates and stores the micro image of the specimen newly captured by the imaging device with the macro unit and stores the micro image in the storage unit. The microscope system according to claim 1, wherein
First display means for displaying a macro image of the specimen;
Second display means for displaying a micro image of the specimen;
Display control means for performing display control of the first display means and the second display means,
The display control means stores, in the storage means, the micro image that is related to the macro image, whether the specimen is a new specimen based on the specimen identification information obtained by the obtaining means. And if the sample is determined to be a new sample, the first display means displays a macro image of the sample imaged by the imaging device, and , Causing the second display means to display a micro image associated with the macro image of the specimen displayed on the first display means,
In addition, the display control means reads out the macro image of the specimen stored in the storage means and displays the macro image on the first display means when it is determined that the specimen is the registered specimen, and the first display means (2) The micro image to be displayed on the display means is a micro image associated with the macro image of the specimen stored in the storage means, or a micro image of the specimen imaged by the imaging device A microscope system characterized by switching between the two. The microscope system according to claim 1, wherein
First display means for displaying a macro image of the specimen;
Second display means for displaying a micro image of the specimen;
Display control means for performing display control of the first display means and the second display means,
The display control means stores, in the storage means, the micro image that is related to the macro image, whether the specimen is a new specimen based on the specimen identification information obtained by the obtaining means. In order to determine whether the sample is a registered sample and to determine that the sample is a new sample, the macro image of the sample imaged by the imaging device is displayed on the first display means, And causing the second display means to display a micro image associated with the macro image of the specimen displayed on the first display means,
The display control means reads the macro image of the sample stored in the storage means and displays the macro image on the first display means when determining that it is the registered specimen, An offline mode in which the micro image to be displayed on the second display means is a micro image associated with the macro image of the specimen stored in the storage means, and the micro image to be displayed on the second display means is captured. A microscope system characterized by switching and displaying an online mode as a micro image of the specimen imaged by an apparatus. In the microscope system according to any one of claims 1 to 3,
In addition to the identification information, the macro image, and the micro image, the storage means stores information related to observation history for the specimen in association with each other,
The additional recording means updates the observation history information when the micro image displayed on the second display means is stored. The microscope system according to claim 4, wherein
The observation history information includes date information and observer name information related to the storage operation of the micro image displayed on the second display means. The microscope system according to claim 5,
The observation history information includes identification information of the microscope system. The microscope system according to claim 1, wherein
In addition to the identification information, the macro image, and the micro image, the storage means stores information related to observation history for the specimen in association with each other,
Third display means for reading out and displaying a macro image and a micro image associated with predetermined identification information in accordance with an external instruction from the storage means;
A microscope system, further comprising an updating unit that updates information of the observation history associated with the predetermined identification information in accordance with an instruction from the outside during display by the third display unit. . In the microscope system according to any one of claims 1 to 3,
The storage means, when associating the macro image and the micro image, performs association including information on the shooting position and shooting conditions of each micro image,
When the specimen to be observed is placed on the stage, the imaging position and imaging conditions of the micro image associated with the identification information acquired by the acquisition unit are read from the storage unit and reproduced, Fine adjustment means for finely adjusting the shooting position and shooting conditions;
A micro image of the specimen is generated by photographing the specimen to be observed in a state after the fine adjustment means has finely adjusted, and the micro image displayed on the first display means and the acquisition A microscope system, further comprising a control unit that associates with the identification information acquired by the unit and stores the additional information in the storage unit. The microscope system according to claim 1, wherein
Position recognition marker storage means for detecting and storing the position recognition marker of the specimen from the image of the preceding specimen imaged by the imaging device;
Based on the identification information of the specimen acquired by the acquisition means, whether the specimen is a new specimen or whether it is a registered specimen in which the macro image and the micro image are already stored in the storage means If it is determined that the specimen is a registered specimen, the loading position on the stage of the specimen loaded on the stage is determined based on the position recognition marker stored in the position recognition marker storage means. A microscope system comprising a position correction step for correcting the position of the microscope. The microscope system according to claim 1, wherein
The additional recording unit combines a plurality of micro images adjacent to each other among past micro images associated with the macro image of the registered specimen and / or micro images newly captured by the imaging device. A microscope system characterized in that it is stored in the storage means as one micro image. In an image processing method of a microscope system comprising a microscope and an imaging device that images a specimen on the stage of the microscope,
An obtaining step of obtaining identification information of the specimen on the stage;
A storage step of storing the obtained macroscopic image and microscopic image of the specimen in association with the acquired identification information of the specimen;
Based on the identification information of the sample, it is determined whether or not it is a registered sample that has already been stored in the storage step. And an additional recording step of adding and storing the micro image of the specimen newly imaged by the imaging apparatus in association with the macro image of the image processing method of a microscope system. The image processing method for a microscope system according to claim 11,
An image of a microscope system comprising first display means for displaying a macro image of the specimen and second display means for displaying a micro image of the specimen, and performing display control of the first display means and the second display means. In the processing method,
Based on the acquired identification information of the specimen, it is determined whether the specimen is a new specimen or a registered specimen corresponding to the already stored macro image and micro image,
When it is determined that the sample is a new sample, a macro image of the sample imaged by the imaging device is displayed on the first display means, and a microscopic image of the sample imaged by the imaging device is displayed. Displaying an image on the second display means;
If it is determined that the specimen is a registered specimen, the stored macro image of the specimen is read out and displayed on the first display means, and further displayed on the second display means. A method for processing an image of a microscope system, wherein the image is switched between a micro image of the specimen stored in the storage step or a micro image of the specimen imaged by the imaging device.

Images