JP2023013365A - Cooperative control system of two axis moving stage and camera, image storing method of cooperative control system of two axis moving stage and camera, stage moving method of cooperative control system of two axis moving stage and camera, image capture and display method of cooperative control system of two axis moving stage and camera, and position information displaying method of cooperative control system of two axis moving stage and camera - Google Patents

Abstract

To solve the problem that when monitoring a sample mounted on a well plate having a plurality of wells by an optical microscope to record an image of the sample, there is a risk that an error of a sample or the like occurs and time for confirmation is required because an image file name of the observed image to be stored by a camera includes only information on photographing date and time and it is not easy to add information of an observation sample.SOLUTION: By controlling a moving stage for mounting a well plate and a camera for acquiring an image of an optical microscope by a single system, a well number can be put in an image file name to reduce risk that an error of a sample occurs, and to improve work efficiency by cooperative control.SELECTED DRAWING: Figure 29

Description

2. Description of the Related Art In the biomedical field, a well plate having a plurality of locations called wells for arranging samples is sometimes used when performing experiments and observations of a plurality of parameters. There are multiple types of well plates, such as 24-well plates, 96-well plates, and 384-well plates having 24, 96, and 384 wells. It has a length of about 127.6 mm and a short side length of about 85.4 mm.

Since these well plates were developed for the purpose of conducting experiments and observations of multiple parameters, signals are often detected by a device called a microplate reader that automatically detects signals as shown in Patent Document 1. .
This microplate reader is generally a device that automatically detects the presence or absence of fluorescence emission from each well and the fluorescence emission intensity. Therefore, as shown in this Patent Document 1, the microplate reader is not equipped with an optical system that allows an observer to visually observe a sample.
There is also a well plate that has a bottom surface covered with a cover glass as used in Patent Document 2, and that allows observation with an optical microscope.

Researchers who perform cell culture generally cultured cells in a container called a Petri dish or 35 mm dish, in which the culture part was not divided into multiple areas and which was of a size that was easy to handle. However, since these culture vessels have large capacities, they require a large amount of culture solution and reagents, so there is the problem of high experimental costs when performing experiments that provide parameters. In recent years, in research fields such as regenerative medicine and immunotherapy, where it is assumed that cultured cells are introduced into the human body, reagents and culture fluids have become highly accurate, and there are extremely expensive reagents.

Well plates of 24 wells, 96 wells, and 384 wells have a shape whose overall size is almost fixed as described above so that signal detection with a microplate reader is possible. becomes smaller. A smaller well size implies a smaller volume of media and reagents required per well. The reduction in the amount of reagents consumed indicates that a large number of experiments can be performed with a small budget. It is increasingly being used in manual instruments, such as optical microscopes, where an operator observes a sample through eyepieces and objectives.

Therefore, the well plate used in Non-Patent Document 1 can be mounted on the optical microscope, and a two-axis moving stage that can be controlled by a computer, such as MLS203 manufactured by Thorlabs, Inc., is on sale.

Image data is data recorded by an operator while performing observation using a well plate. This image data is generally saved as an image captured by a digital camera attached to an optical microscope.

In recent years, an open source software called Micromanager (Non-Patent Document 2) has emerged in order to facilitate PC control of equipment connected to an optical microscope. Since it is necessary to purchase and build software, it is not popular because software engineers are required to build the system.

Japanese Patent Application Laid-Open No. 2016-212115 Patent No. 6373864

Nathan Blanke, Veronica Go, Douglas L. Rosene, and Irving J. Bigio,"Quantitative birefringence microscopy for imaging the structural integrity of CNS myelin following circumscribed cortical injury in the rhesus monkey",Neurophotonics, Vol.8, number 1, 015010 ( 2021) Arthur D. Edelstein, Mark A. Tsuchida, Nenad Amodaj, Henry Pinkard, Ronald D. Vale, and Nico Stuurman "Advanced methods of microscope control using micro manager software", Journal of Biology Methods, Vol. 1, number 2, e10 ( 2014) L.D. Bergman, B.E. Rogowitz, and L.A. Treinish, "A rule-based tool for assisting colormap selection", Proceedings of the 6th IEEE Visualization Conference (VISUALIZATION '95) pp. 118-125 (1995)

The method of saving the observation image data of the sample mounted on the well plate is generally performed using the software provided by the camera manufacturer, even when the two-axis movement stage and the camera are connected to the same computer. and is rarely coordinated with software that controls the two-axis motion stage.

When the captured image data is saved using the software provided by the camera manufacturer, it is common that the captured time is used as the saved file name. Therefore, based on the time information, the operator associates and manages the number of the observed well and the saved image information.
The confirmation of the well number is performed using the position information of the two-axis moving stage, but the two-axis moving stage on the market does not have the function of displaying the well number of the well plate. Therefore, the operator frequently checks the observed well number. The method of manually confirming the well number by the operator involves confirming the order of the currently observed well from the position of the objective lens.

The method for observing samples is fluorescence observation, and if the luminescence of the sample is weak, it is necessary to turn off the room lighting and observe. Since it is not easy, there is an inconvenience that the lighting in the room must be brightened each time.

Once the operator makes a mistake in connecting the well number observed and the photograph taken, the experimental results will be inconsistent, and the reliability of the data will be lost.

On the other hand, when the method of observing the sample is fluorescence observation and the luminescence of the sample is weak, it is difficult to focus the microscope and find the target sample. A method of increasing the intensity and a method of observing by changing to a high-magnification objective lens with a high NA are used. However, if the sample has low light resistance or the fluorescence fades quickly, there is a problem that the sample may be damaged during the focus adjustment process. In such a situation, it is important to perform the observation work quickly, so it is not desirable to change the observation conditions, such as changing the exposure time or changing the objective lens.

The present invention has been made in consideration of the above points, and solves the problem that confirmation of the numbers of the wells observed by the operator takes time. In addition, by facilitating the correlation information (linking information) between the stored image of the observed image and the sample information such as the observed well number, the problem of mistaking the relationship between the sample and the image is solved.
Furthermore, the problem of the sample being damaged during the focus adjustment process is solved when the method of observing the sample is fluorescence observation and the light emission of the sample is weak, or when the light resistance of the sample is weak.

In order to solve these problems, the present invention provides a coordinated control system for a 2-axis moving stage and a camera, an image storage method for a coordinated control system for a 2-axis moving stage and a camera, a stage moving method for a coordinated control system for a 2-axis moving stage and a camera, In the image capturing and display method of the coordinated control system of the 2-axis moving stage and camera, and the position information display method of the coordinated control system of 2-axis moving stage and camera, the 2-axis moving stage on which the sample such as the well plate is mounted and Coordinated control of the camera by one controller facilitates confirmation of the number of the well being observed by the operator. In order to facilitate the correlation information (linking information) between the saved image and the sample information such as the observed well number, the sample observation position and well number are inserted into the image save file name. To solve the problem of mistaking the relationship between and the image.
Furthermore, by displaying an image that has undergone image enhancement processing, when the method of observing the sample is fluorescence observation and the luminescence of the sample is weak, or when the light resistance of the sample is weak, the focusing process or searching for the object is performed. Reducing the time reduces the risk of sample damage.

2-axis moving stage and camera cooperative control system, image storage method of 2-axis moving stage and camera cooperative control system, stage moving method of 2-axis moving stage and camera cooperative control system, 2-axis moving stage and camera of the present invention In the image capturing and display method of the linked control system and the position information display method of the linked control system of the two-axis moving stage and the camera, the labor of confirming the work of the operator is reduced and the risk of wrong sample number is reduced. Since the image data and the sample can be easily linked, the efficiency of the data sorting work is also improved.
Furthermore, by displaying an image that has undergone image enhancement processing, when the method of observing the sample is fluorescence observation and the luminescence of the sample is weak, or when the light resistance of the sample is weak, the focusing process or searching for the object is performed. By shortening the time in the process, there is an effect that long-term observation becomes possible by reducing the risk of damage to the sample.

It is a figure which shows the component of the cooperation control system of the two-axis moving stage of this invention, and a camera. 1 is a schematic configuration diagram of an optical microscope; FIG. 1 is a schematic configuration diagram of an optical microscope to which a two-axis moving stage and a camera cooperative control system of the present invention are attached; FIG. 1 is a schematic configuration diagram of a two-axis moving stage of the present invention; FIG. 1 is a schematic configuration diagram of a two-axis moving stage of the present invention; FIG. 1 is a configuration diagram of a motor controller box of the present invention; FIG. It is a figure which shows the structure of a well plate, and the imaging|photography range of this invention. It is a figure which shows the reference position of the movement of the biaxial movement stage of this invention. FIG. 4 is a schematic diagram of a screen displayed on a computer monitor when the two-axis moving stage and camera cooperation control system of the present invention is in operation; FIG. 4 is an explanatory diagram of a graphic user interface for controlling the cooperative control system of the two-axis moving stage and camera of the present invention; FIG. 4 is an explanatory diagram of a graphic user interface for controlling the cooperative control system of the two-axis moving stage and camera of the present invention; FIG. 4 is an explanatory diagram of a graphic user interface for controlling the cooperative control system of the two-axis moving stage and camera of the present invention; It is a figure which shows the relationship between the well number of the long side direction of a 96-well plate, and a motor position. It is a figure which shows the relationship between the well number of the short side direction of a 96-well plate, and a motor position. It is a figure which shows the relationship between the well number of the long side direction of a 24-well plate, and a motor position. FIG. 4 is a diagram showing the relationship between well numbers in the short side direction of a 24-well plate and motor positions. FIG. 4 is a diagram showing the relationship between well numbers in the long side direction of a 384-well plate and motor positions. It is a figure which shows the relationship between the well number of the short side direction of a 384-well plate, and a motor position. FIG. 4 is a flow chart showing a control method of the cooperative control system of the two-axis moving stage and camera of the present invention; FIG. 4 is a flow chart showing a control method of the cooperative control system of the two-axis moving stage and camera of the present invention; FIG. 4 is a flow chart showing a control method of the cooperative control system of the two-axis moving stage and camera of the present invention; FIG. 4 is a flow chart showing a control method of the cooperative control system of the two-axis moving stage and camera of the present invention; FIG. 4 is a flow chart showing a control method of the cooperative control system of the two-axis moving stage and camera of the present invention; FIG. 4 is a flow chart showing a control method of the cooperative control system of the two-axis moving stage and camera of the present invention; FIG. 4 is a flow chart showing a control method of the cooperative control system of the two-axis moving stage and camera of the present invention; It is a figure which shows the image enhancement method of the cooperation control system of the two-axis moving stage of this invention, and a camera. 1 is a schematic configuration diagram of an optical microscope to which a two-axis moving stage and a camera cooperative control system of the present invention are attached; FIG. 1 is a schematic configuration diagram of an optical microscope in which a sample stage supports the rigidity of an optical system; FIG. 1 is a schematic diagram of an optical microscope equipped with a two-axis moving stage and camera cooperative control system of the present invention; FIG. 1 is a schematic configuration diagram of a two-axis moving stage of the present invention; FIG. . 1 is a schematic configuration diagram of an optical microscope to which a two-axis moving stage and a camera cooperative control system of the present invention are attached; FIG. 1 is a schematic configuration diagram of an optical microscope to which a two-axis moving stage and a camera cooperative control system of the present invention are attached; FIG.

FIG. 1 shows a configuration diagram of a cooperative control system for a two-axis moving stage and a camera according to the present invention. A two-axis moving stage and camera cooperative control system 11 controls the two-axis moving stage 30, a motor controller box 31 that drives the motor of the two-axis moving stage 30, the camera 12, the two-axis moving stage 30, and the camera 12. It consists of a controller 13 . The two-axis moving stage and camera cooperative control system 11 includes a motor cable 32 that connects the two-axis moving stage 30 and the motor controller box 31, a motor controller cable 56 that connects the motor controller box 31 and the controller 13, the camera 12 and the controller 13. includes a camera cable 15 connecting the Although the configuration diagram shown in FIG. 1 shows an example in which the two-axis moving stage 30 and the motor controller box 31 are separated and connected by a motor cable 32, the two-axis moving stage 30 and the motor controller box 31 are integrated. It is also possible to convert Here, the controller 13 is a PC (personal computer) or the like in which software for controlling the two-axis moving stage 30 and the camera 12 is installed. The camera 12 is a camera device conforming to the UCB3 format, and if the camera 12 is a camera device conforming to the UCB3 format, the camera cable 15 is a USB3 cable.

The biaxial movement stage 30 has a sample mounting portion 29 for mounting an observation sample such as a well plate. The two-axis moving stage 30 also has an X-axis motor 33 and a Y-axis motor 34 that move the mounted sample in directions substantially perpendicular to each other.

FIG. 2 is a diagram showing a schematic configuration diagram of an optical microscope 8 to which the two-axis moving stage and camera cooperation control system of the present invention is connected and used. The optical microscope 8 is an optical microscope for observing the samples placed in the wells 21 of the well plate 20 mounted on the sample table 3, and has two light sources, a fluorescence excitation light source 60 and a transmitted illumination light source 73. shows an example with
Illumination light emitted from a fluorescence excitation light source 60 is collected by a lens 61 and guided to a fluorescence filter cube 62 . The fluorescence filter cube 62 includes an excite filter 70 that selectively transmits a wavelength effective for fluorescence excitation of the sample from the wavelength of the illumination light, and a dichroic mirror that reflects the wavelength used for fluorescence excitation of the sample and transmits the fluorescence wavelength emitted by the sample. 71, an emission filter 72 is arranged to transmit the fluorescence wavelength of the sample and block the wavelength used for fluorescence excitation of the sample.

The light transmitted through the excite filter 70 is applied to the sample in the well 21 through the objective lens 63 , and the fluorescence emitted by the sample passes through the emission filter 72 and enters the imaging lens 64 . The fluorescence observation image of the sample incident on the imaging lens 64 is reflected by the mirrors 65 , 66 and further the switching mirror 67 , and is observed by the operator through the eyepiece 68 .
The optical microscope 8 has a camera port 18 to which a camera is connected, and by changing the position of the switching mirror 67, fluorescence observation of the sample can also be performed by the camera connected to the camera port 18. Configuration.

The optical microscope 8 has a transmission illumination light source 73 and a condensing lens 74 for condensing and irradiating the light emitted from the transmission illumination light source 73 onto the sample, and is configured to allow observation of a transmission observation image of the sample. there is When observing a transmission observation image of a sample, the configuration is such that the fluorescence filter cube 62 is avoided from the optical path. The transmission illumination light source 73, the condenser lens 74, the objective lens 63, the mirrors 65, and the mirrors 66 are aligned with the high-rigidity microscope frame 1. It is configured so that it can be observed by an operator observing through the lens 68 and by a camera connected to the camera port 18 .

The sample stage 3 is attached to the stage attachment portion 19 . Since the frame 1 of the optical microscope 8 has a high rigidity, even if the sample table 3 is removed from the stage mounting portion 19, the transmission observation image of the sample can be observed through the eyepiece 68 by the operator and the camera. It is configured so that it can be observed by a camera connected to the port 18 .

FIG. 3 is a diagram showing a schematic configuration diagram of an optical microscope 10 to which a two-axis moving stage and a camera cooperative control system of the present invention are connected.
The optical microscope 10 has the sample stage 3 shown in FIG. The optical microscope 10 can store an image of the cooperative control system of the two-axis moving stage and camera of the present invention, a stage moving method of the cooperative control system of the two-axis moving stage and camera, and an image of the cooperative control system of the two-axis moving stage and camera. It is an optical microscope to which a photographing and display method and a position information display method of a cooperative control system of a two-axis moving stage and a camera are applied.
A well plate 20 having a plurality of wells 21 is attached to the biaxially moving stage 30, and by moving the biaxially moving stage 30, the sample in the plurality of wells 21 is observed with an optical microscope. It has become. The camera 12 acquires an enlarged observation image by the objective lens 63 and the imaging lens 64 as image information by causing the fluorescence excitation light source 60 or the transmission illumination light source 73 to emit light. A monitor 14 for displaying an enlarged observation image observed by the camera 12 is connected to the controller 13 via a monitor cable 16 . The monitor 14 displays a graphic user interface for controlling the two-axis moving stage 30 and the camera 12, as well as an image captured by the camera 12 and an image obtained by enhancing the image.

FIG. 4 shows a schematic configuration diagram of the two-axis moving stage 30. As shown in FIG. The two-axis moving stage 30 has a stage fixing portion 37 attached to the stage attaching portion 19 , and an X-axis motor 33 is arranged on the stage fixing portion 37 . The X-axis motor 33 has a function to rotate the X-axis shaft 35 , and by rotating the X-axis motor 33 , the X-stage table 38 mounted on the X-axis shaft 35 is parallel to the rotation axis of the X-axis shaft 35 . It is configured to move in any direction. In this example, the X-axis shaft 35 is formed with a screw with a pitch of 2.54 mm. It is configured to move 54 mm.

A Y-axis motor 34 is arranged on the X stage table 38 . The Y-axis motor 34 has a function of rotating the Y-axis shaft 36 , and by rotating the Y-axis motor 34 , the Y-stage table 39 mounted on the Y-axis shaft 36 is parallel to the rotation axis of the Y-axis shaft 36 . It is configured to move in any direction. In this example, the Y-axis shaft 36 is threaded with a pitch of 2.54 mm like the X-axis shaft 35 . It is configured to move 2.54 mm in the direction corresponding to . The X-axis shaft 35 and the Y-axis shaft 36 are arranged orthogonally. In addition, the X-axis shaft 35 and the Y-axis shaft 36 are configured to be perpendicular to the observation optical system including the objective lens 63 and the imaging lens 64, so that the X-axis motor 33 and the Y-axis motor 34 can be rotated. , the observation position observed by the objective lens 63 and the imaging lens 64 can be moved.

As shown in FIG. 4 , the well plate 20 is mounted on the sample mounting portion 29 of the two-axis moving stage 30 . The well plate 20 has two chamfers 22 arranged on one side in the longitudinal direction to define the orientation of the well plate. A well plate retainer 40 that holds down the chamfered portion 22 of the well plate 20 is arranged on the biaxial movement stage 30 . It is configured to be pressed down.
In this example, the well plate 20 is arranged so that the long side direction is parallel to the rotation axis of the X-axis shaft 35, and the short side direction of the well plate 20 is parallel to the rotation axis of the Y-axis shaft 36. are arranged so that

When the sample to be observed using the optical microscope 10 is a well plate, the moving range of the X stage table 38 and the Y stage table 39 of the two-axis moving stage 30 covers all the wells arranged on the well plate 20. It is sufficient to have a range in which 21 observations are made. Therefore, in the two-axis moving stage 30, limit sensors are arranged to prevent the X stage table 38 and the Y stage table 39 from moving to unnecessary positions, thereby limiting the movement range. . FIG. 5 is a schematic diagram of the configuration of the Y-axis including limit sensors 42 and 43 that limit the movement range of the Y-stage table 39. As shown in FIG.

The Y-axis shaft 36 connected to the Y-axis motor 34 is fixed to a shaft holder 41 at the end that is not attached to the Y-axis motor 34 for the purpose of suppressing axial shake due to the rotation of the Y-axis shaft 36 . ing. Limit sensor blades 44 and 45 are arranged on a Y-stage table 39 that moves in a direction parallel to the rotation axis of the Y-axis shaft 36 according to the rotation direction of the Y-axis motor 34 . The limit sensors 42 and 43 are, for example, non-contact type limit sensors that integrate a light-emitting portion and a light-receiving portion and report when the light emitted from the light-emitting portion is blocked by a shield and does not enter the light-receiving portion. When the limit sensor blades 44 and 45 are positioned at a position where the light emitted from the predetermined light emitting portions 42 and 43 is blocked and does not enter the light receiving portion, an alarm is issued. Therefore, the rotation of the Y-axis motor 34 causes the Y-stage table 39 to approach the Y-axis motor 34 side. It is configured to When the Y-stage table 39 moves away from the Y-axis motor 34 due to the rotation of the Y-axis motor 34 and the limit sensor blade 45 reaches the position where the optical system in the limit sensor 43 is shielded, the limit sensor 43 issues an alarm. It is configured. Regarding the X-axis motor 33 as well, limit sensors are arranged in the direction approaching the X-axis motor 33 and the direction away from the X-axis motor 33 in the same manner as the Y-axis motor 34 .

FIG. 6 is a diagram showing the configuration of the motor controller box 31. As shown in FIG. In this example, the X-axis motor 33 and Y-axis motor 34 are two-phase stepping motors. An example is shown in which power is supplied to the motor via cables 32a and 32b. The direction and amount of rotation of the X-axis motor 33 and the Y-axis motor 34 are commanded from the controller 13 to the controller box 31 via the cable 56. Within the controller 13, the digital controller 51 selects the motor to rotate. and the rotation direction of the motor are given, the pulse generator 52 generates a number of pulse signals corresponding to the amount of rotation of the X-axis motor 33 and the Y-axis motor 34, and the X-axis pulse transmission cable 53 and the Y-axis pulse transmission cable 54 to the motor controller 50 . The motor controller 50 sends the signals of the limit sensors approaching the X-axis motor 33 and away from the X-axis motor 33, the limit sensors approaching the Y-axis motor 34 and the directions away from the Y-axis motor 34, to cables 32c and 32d. , 32e and 32f to grasp the state of the limit sensors in each direction of each axis. Also, the state of each limit sensor can be monitored by the controller 13 via the digital controller 51 .

When the limit sensor is monitored by the controller 13 via the digital controller 51, there is a risk of a time lag of several tens of milliseconds. Therefore, in order to prevent the time lag, when transmitting the number of pulse signals corresponding to the amount of rotation of the X-axis motor 33 and the Y-axis motor 34 by the pulse generator 52 to the X-axis motor driver 46 and the Y-axis motor driver 47, By relaying the motor controller 50, when the limit sensor in the direction of movement by the moving motor is notified, the motor controller 50 immediately does not output the pulse signal to the motor driver. In the method of stopping the transmission of the pulse signal to the X-axis motor driver 46 and the Y-axis motor driver 47 in response to the alarm from the limit sensor by the motor controller 50, it is possible to respond within 10 microseconds or less. Therefore, it forms a safety mechanism without time lag.
In the example of the present invention, the X-axis motor 33 and the Y-axis motor 34 are set to make one rotation when 3200 pulses are given. As described above, one rotation of the X-axis motor 33 and the Y-axis motor 34 causes the well plate 20 to move 2.54 mm in the long side direction and the short side direction. Therefore, applying one pulse of the signal causes the well plate 20 to move about 0.79 microns in the long and short directions.

7 and 8, using a 96-well plate in which 96 wells 21 are formed in the well rate 20 as an example, observation in the well plate 20 by a two-axis movement stage and a camera cooperative control system Explain the shooting range. As shown in FIG. 7, the well plate is numbered from 1 to 12 along its longer side and alphabetically from A to H along its shorter side. is to be identified. Since it is desirable that the two-axis moving stage 30 used in the present invention be low in cost, it is configured so as not to have a scale mechanism for monitoring the position of the well plate 20 in both the long side direction and the short side direction. ing. Therefore, the positional information of the well plate 20 is grasped by the controller 13 by accumulating movement command information given to the X-axis motor 33 and the Y-axis motor 34 .
Therefore, when the power is turned on or when the controller 13 starts controlling the two-axis moving stage 30, it is necessary to acquire the origin information in the control system by the process of grasping the position information using the limit sensor described above. Also, as described above, if the limit sensor fires while the X-axis motor 33 and the Y-axis motor 34 are moving, the safety mechanism that stops the motors is activated even if the pulse signal is being transmitted. Stage position information cannot be grasped due to the accumulation of pulse information. Therefore, in the control method of the two-axis moving stage 30 of the present invention, in order to always grasp the positional information of the well plate 20 by the controller 13, a method of moving the well plate 20 within a range in which the limit sensor does not generate an alarm in normal use is adopted. We are hiring.

As described in the description of FIG. 4 above, the well plate 20 is fixed and mounted on the biaxial movement stage 30 while being pressed by the well plate presser 40 . Therefore, FIG. 8 shows an enlarged view of the vicinity of the corner facing the chamfered portion 22 pressed by the well plate retainer 40. The position of the corner 20a opposed to the chamfered portion 22 pressed by the well plate retainer 40 is It is in a fixed state. A dashed line 80 indicates a movable range defined for the limit sensor in FIGS. That is, in the two-axis moving stage 30 of the present invention, the limit sensors are arranged at the positions of the dashed lines 80b, 80c, 80d, and 80e, which are 0.5 mm inside the four sides of the well plate 20. FIG.

When the power is turned on or when the controller 13 starts controlling the two-axis moving stage 30, the origin is set by moving the X-axis motor 33 at such a speed that the motor can be stopped immediately when the signal from the limit sensor is issued. The controller 13 grasps the position of the dashed line 80b. Thereafter, the controller 13 grasps the position of the dashed line 80c by moving the Y-axis motor 34 at such a speed that the motor can be stopped immediately when the signal from the limit sensor is issued. The positional information of the mechanical origin 80a in FIG. 8 is obtained by this two-axis operation.

In order to always grasp the position of the well plate 20 by the controller 13 during control, the normal drivable range is inside the dashed lines 80b, 80c, 80d and 80e with a margin and includes all the wells 21 of the well 20. Must be a range. In the present invention, the movable range of the well plate 20 by the controller 13, that is, the observation and imaging range is set to the range inside the broken lines 81b, 81c, 81d, and 81e in FIG. As a specific example, the positions of the dashed lines 81b, 81c, 81d, and 81e are set 0.5 mm inside the dashed lines 80b, 80c, 80d, and 80e, respectively, so that the range includes all the wells 21. can. Therefore, as the position information management of the two-axis moving stage 30 by the controller 13, after grasping the position information of the mechanical origin 80a in the origin finding process performed when the power is turned on or when the controller 13 starts controlling the two-axis moving stage 30, X A command to move the well plate by 0.5 mm is given to the axis motor 33 and the Y-axis motor 34 to move the well plate to 81a, the accumulated pulse number given to the X-axis motor 33 and the Y-axis motor 34 is reset to zero, and 81a is controlled by the controller 13. This is the origin (software origin) of the movement command of the two-axis movement stage 30 . By recording the direction and pulse amount of the movement command from the controller 13, the movement amount in the X-axis direction (stage movement distance), which is the movement amount in the long side direction of the well 20 indicated by the arrow 82 in FIG. Y-axis direction movement amount (stage movement distance), which is the movement amount of the shown well 20 in the short side direction, can be managed. The well plate 20 has a long side length of about 127.6 mm and a short side length of about 85.4 mm. By setting the distance to 125.6 mm and the distance from the dashed line 81d to the dashed line 81e to 83.4 mm, the stage can be moved without triggering the limit sensor when the controller 13 drives the well plate.

In the setting of the motor and motor controller of the present invention, it is set to move 2.54 mm when 3200 pulses are given, so the distance of 125.6 mm from dashed line 81b to dashed line 81c is 158236 pulses, A distance of 83.4 millimeters from dashed line 81d to dashed line 81e results in 105070 pulses. Therefore, before movement in the longitudinal direction of the well, if the accumulated number of pulses after movement is greater than 0 and less than 158236, it is known before movement that the well is inside the dashed lines 81b and 81c after movement. Therefore, if the accumulated number of pulses after movement is greater than 0 and less than 105070 before movement in the short side direction of the well, it means that it is inside the dashed lines 81d and 81e after movement. You will understand.

FIG. 9 shows an example of a screen displayed on the monitor 14 when the cooperative control system of the two-axis moving stage and camera of the present invention is being controlled. A monitor 90 has a graphic user interface window 90 that allows control of the two-axis translation stage and the camera, a first window 91 that displays images captured by the camera 12, a second window 92, a second 3 window 93 is displayed. In this example, the first window 91 displays the entire image captured by the camera 12, the second window 92 displays an enlarged image of the image captured by the camera 12, and the third window 92 displays an enlarged image of the image captured by the camera 12. 93 displays an enhanced image obtained by converting the image captured by the camera 12 into a color map so that the object can be easily recognized. Although this example shows an example in which there are three windows for displaying images, the number of windows for displaying images need not be three.

FIG. 10 shows an example of a window 90 of the graphical user interface. If the biaxial movement stage 30 is not provided with a scale, the position of the well plate 20 is grasped by the controller 13 when the power is turned on or the software is started up, as shown in the description of FIGS. Therefore, after recognizing the mechanical origin 80a, it is necessary to move to the software origin 81a and reset the number of accumulated pulses given to the X-axis motor 33 and the Y-axis motor 34 to zero. By pressing the button 101 after starting up the software, the user of the two-axis movement stage and camera cooperation control system of the present invention can perform the process of recognizing the mechanical origin 80a, the process of moving to the software origin 81a, the X-axis motor 33 A series of initialization steps including a step of resetting the accumulated pulse number applied to the Y-axis motor 34 to zero must be performed. When this series of initialization steps is completed, a display indicating that the preparation of the cooperative control system for the two-axis moving stage and the camera is completed is presented on the display window 102 .

The message screen 103 is displayed when there is content to be conveyed from the controller side to the operator, such as notifying that the initialization process is necessary when a stage movement command is input even though the initialization process has not been completed. This is the screen where a message is displayed on the

When the sample to be observed is a well plate, the number of wells of the observed well plate is selected in the well plate selection/display area 104 . In this example, 24, 96, and 384 are options for the number of wells, and the number of wells of the well plate mounted on the biaxial movement stage 30 is selected. The number of selected wells is presented in the well plate selection/display area 104 . If the user wants to add information about the sample to the file name when saving the captured image, the user inputs text in the sample information input/display area 105 . FIG. 10 shows an example in which GFP1 is input, but the input text is displayed until it is rewritten. If you want to input the position where you want to observe the observation material by the distance from the software origin, enter the numerical value in the input areas 106 and 107, and press the movement start button 108 to the target position after inputting the numerical value. 2-axis translation stage 30 moves to its target position. If you want to enter the position where you want to observe the observation material by the well number, enter the numerical value and the alphabet in the input areas 109 and 110, and press the movement start button 111 to the target position after inputting the numerical value and the alphabet. As a result, the two-axis moving stage 30 moves to its target position.

When the movement start button 108 or the movement start button 111 to the target position is pressed and the movement of the two-axis movement stage 30 is completed, the numerical values input to the input areas 106 and 107 and the input areas 109 and 110 are erased, and the movement is completed. The distance from the software origin is displayed in position display areas 120 and 121 based on the information on the position obtained, and the well number is displayed in 122 and 123 . The reason why the numerical values input to the input areas 106, 107 and input areas 109, 110 are erased is that the movement of the two-axis moving stage 30 to the target positions input to the input areas 106, 107 and input areas 109, 110 is completed. Later, when moving to another position due to relative movement of the observation position, etc., the numerical values displayed in the position display areas 120, 121 and 122, 123 and the numerical values entered in the input areas 106, 107 and the input areas 109, 110 may be different, and the operator does not interpret the input areas 106, 107 and the input areas 109, 110, which are not the current position, as the current position. Therefore, when the two-axis moving stage 30 is stationary, the numerical values entered in the input areas 106 and 107 and the input areas 109 and 110 are not displayed.
In the cooperative control system of the two-axis moving stage and the camera of the present invention, when the movement of the two-axis moving stage 30 is completed, the camera 12 takes an image, and the monitor 14 displays the taken image. An image is displayed in any one or all of the window 91, the second window 92, and the third window 93. FIG.

As a method of moving the two-axis moving stage 30, it is also possible to perform relative movement from the current position. FIG. 11 shows an enlarged view of the area where the buttons for relatively moving the window 90 of the graphic user interface are arranged. In the present invention, the buttons 130 to 137 and 140 to 147 are arranged. As soon as this button is pressed, the respective predetermined stage movement is performed. Specifically, when the button 130 is pressed, a movement of 10 microns is made in the positive X direction. In this example, the direction indicated by an arrow 82 in FIG. 8 is the positive direction of the X direction. Also, when the button 131 is pressed, a movement of 10 microns is made in the negative X direction. Also, if the button 132 is pressed, a movement of 10 microns is made in the positive Y direction. In this example, the direction indicated by an arrow 83 in FIG. 8 is the positive direction of the Y direction. If button 133 is pressed, a movement of 10 microns is made in the negative Y direction.

If the button 134 is pressed, the movement is 10 microns each in the positive X direction and the positive Y direction. If the button 135 is pressed, a movement of 10 microns in each of the positive X direction and the negative Y direction is made. If the button 136 is pressed, a movement of 10 microns in each of the negative X direction and the positive Y direction is provided. If the button 137 is pressed, a movement of 10 microns in each of the negative X and negative Y directions is provided.

If button 140 is pressed, there is 100 microns of movement in the positive X direction, and if button 141 is pressed, there is 100 microns of movement in the negative X direction. If button 142 is pressed, 100 micrometers of movement is made in the positive Y direction. If button 143 is pressed, a movement of 100 microns is made in the negative Y direction.
If the button 144 is pressed, the movement is 100 microns each in the positive X direction and the positive Y direction. If the button 145 is pressed, a movement of 100 microns in each of the positive X direction and the negative Y direction is made. When button 146 is pressed, movement is 100 microns each in the negative X direction and the positive Y direction. If the button 147 is pressed, a movement of 100 microns in each of the negative X and negative Y directions is provided.

When the buttons 130 to 137 are pressed, an image is captured by the camera 12 when the movement of the two-axis moving stage 30 is completed. The image is displayed in either or all of the second window 92 and the third window 93 .

In the present invention, when the buttons 140 to 147 are pressed, the camera 12 takes an image each time the two-axis moving stage 30 moves 20 microns, and the monitor 14 displays the taken image. An image is displayed in any one or all of the second window 91, the second window 92, and the third window 93. FIG. The reason why an image is taken every 20 microns is to prevent overlooking an object to be observed in the middle of the observation because a movement of 100 microns is larger than the field of view when a high-magnification objective lens is used.

The window 90 of the graphic user interface allows not only the control of the two-axis moving stage 30 but also the control of the camera 12 . With reference to FIG. 12, buttons 150 to 156 for controlling the camera 12 and performing image-related functions such as saving captured images will be described. Camera condition input/display areas 150 and 151 are input/display areas related to the exposure time of the camera 12 and signal amplification after photographing. The input area/display area 150 is for inputting the exposure time of the camera 12 in units of microseconds. If the input value is within the setting range of the camera 12, the numerical value is displayed. take a picture. Also, if the input value is out of the setting range of the camera 12, the closest numerical value is displayed and the exposure time of the displayed numerical value is used for photographing. Similarly, in the input area/display area 151, the amplification value of the image captured by the camera 12 is input as a numerical value. conduct. If the input value is out of the setting range of the camera 12, the closest numerical value is displayed and the photographed image is amplified with the displayed numerical value. An image captured by the camera 12 under the conditions set using the camera condition input/display areas 150 and 151 is displayed on the first window 91 that displays the captured image on the monitor 14 . In this example, if the number of pixels captured by the camera 12 is equal to or greater than the number of pixels displayed on the monitor 14, the image captured by the camera 12 is reduced to the first window 91 so that the entire image can be displayed. display the image.

The input/display area 152 is an image enhancement condition input/display area for displaying the designation of the image enhancement method and its set value when performing image enhancement on the image captured by the camera 12 . The image captured by the camera 12 is displayed in a second window 92 that performs specified image enhancements and displays the captured image on the monitor 14 . Regarding the image display on the second window 92 as well, if the number of pixels captured by the camera 12 is equal to or greater than the number of pixels displayed on the monitor 14, the entire image captured by the camera 12 can be displayed. Display a reduced image.

The input/display area 153 is an area for inputting/displaying an enlargement magnification for displaying an enlarged image captured by the camera 12. For example, when a numerical value of 1 is input, the image captured by the camera 12 is reduced. is displayed in the third window 93 without When the number of pixels captured by the camera 12 is equal to or greater than the number of pixels displayed on the monitor 14 , a part of the image captured by the camera 12 near the center is displayed in the third window 93 .

When the continuous shooting/display button 154 is pressed, shooting by the camera 12 and image display in the first window 91, the second window 92, and the third window 93 are continuously performed for a certain period of time. This is a convenient function for the operator to focus the optical microscope.
In the cooperative control system of the two-axis moving stage and the camera of the present invention, an image is captured by the camera 12 after each stage is moved. image capture can be performed. When an image is captured, the captured image is displayed in a first window 91, a second window 92 and a third window 93. FIG. When an image is captured by the camera 12, the image is recorded in a cache memory in preparation for storing the captured image, and time information (year/month/day/hour/minute/ seconds) is also recorded in the cache memory.

When the image save button 156 is pressed, the image captured immediately before and displayed in the first window 91, the second window 92, and the third window 93 is stored in the controller's hard disk drive or the like. or a specified storage area with a file name.

2-axis moving stage and camera cooperative control system, image storage method of 2-axis moving stage and camera cooperative control system, stage moving method of 2-axis moving stage and camera cooperative control system, 2-axis moving stage and camera of the present invention In the image capturing and display method of the linked control system and the position information display method of the linked control system of the two-axis moving stage and the camera, the observer can easily confirm the well number of the well plate being observed, and , the controller always grasps the positional information of the two-axis moving stage 30 so that the well number is not mistaken in the management of the photographed images, and the stage position display areas 120, 121, 122, which are predetermined positions of the graphic user interface 90, are displayed. , 123, and when an image taken by the camera 12 is saved, the well number is inserted into the file name together with the stage position to prevent the well number from being mistaken.

The well number along with the stage position to be inserted in the file name can be used to estimate the well position from the stage coordinates if the two-axis translation stage 30 has a scale, or if the two-axis translation stage 30 does not have a scale. In the case of stage movement using a stepping motor, position estimation and well position estimation are performed by managing the number of accumulated pulses from the mechanical or software origin.

In the example of the present invention, the two-axis movement stage 30 does not have a scale and moves the stage using a stepping motor. Regarding the method of estimating the position and the well position by managing the number of accumulated pulses from the software origin, Description will be made with reference to FIGS. 13 to 19. FIG. The software origin 81a is positioned 1 mm from the X direction and 1 mm from the Y direction from the corner 20a of the well plate 20 as described above. When the center of the observation area by the objective lens 63 of the optical microscope 10 is positioned at the software origin 81a, the accumulated pulse number given to the X-axis motor 33 and the Y-axis motor 34 is reset to zero. Further, since the well plate 20 is mounted on the biaxial movement stage 30 using the chamfered portion 22 formed on the well plate 20, the well number closest to the software origin 81a is 1A as shown in FIG. .

As described above, the X-axis motor 33 and the Y-axis motor 34 are set to rotate once when 3200 pulses are given, and the X-axis motor 33 and the Y-axis motor 34 connected to the X-axis motor 33 and the Y-axis motor 34 are set to make one rotation. The well plate 20 is set to move 2.54 mm in the long-side direction and the short-side direction by one rotation of the axial shaft and the Y-axis shaft.

FIG. 13 shows the distance in millimeters from the software origin of the center position of each well in the long side direction (X direction) of the well plate in the case of a 96-well plate in which 96 wells 21 are formed, and the cumulative distance from the software origin. Fig. 3 shows a relational table of the number of pulses; As well plate information, information about the center position of each well from the vertex 20a closest to the well number 1a is open to the public. It can be easily formed based on the information that the point is 1 millimeter away from the nearest vertex 20a in the X direction. Furthermore, the cumulative number of pulses for each well can be easily calculated from 2.54 millimeters being 3200 pulses. FIG. 14 shows the distance in millimeters from the software origin of the center position of each well in the short side direction (Y direction) of the well plate in the case of a 96-well plate in which 96 wells 21 are formed, and the accumulation from the software origin. Fig. 3 shows a relational table of the number of pulses;
15 and 16 show the center position of each well in the long side direction (X direction) and short side direction (Y direction) of the well plate when using a 24 well plate, measured in millimeters from the software origin. 17 and 18 show a relational table of the distance and the cumulative number of pulses from the software origin. FIGS. 17 and 18 show A relational table of the distance in millimeters from the software origin of the center position of each well and the cumulative number of pulses from the software origin is shown.

FIG. 19 shows a method of estimating the well numbers, taking the case of using a 96-well plate as an example. In this example, a well number analogy method is shown for the case where the cumulative pulse information in the X direction is 81,300 pulses and the cumulative pulse information in the Y direction is 27,000 pulses.
In the graphic user interface 90, it is checked whether well plate information has been input in the well plate selection/display area 104, and if there is no information, a message indicating that the well plate information has not been input is displayed in the message display area 103. In addition, the information that the X-axis motor 33 and the Y-axis motor 34 make one rotation in 3200 pulses and the X-axis shaft and the Y-axis shaft rotate one rotation of the X-direction and Y-direction positions in the long side direction and the short side direction Positional information in the X and Y directions is calculated based on the information that it will move by 2.54 mm, and the numerical values are displayed in the positional information areas 120 and 121 .

When the well plate information is entered in the well plate selection/display area 104, 81300 pulses are read as the accumulated pulse information in the X direction and compared with the table shown in FIG. 80755, which is the number of pulses at the center position of the 8th well, and 92,094, which is the number of pulses at the center position of the 8th well. The difference is 545, and it is known that the numerical difference from 92094, which is the number of pulses at the center position of the eighth well, is 10794. Furthermore, by comparing the difference between these values, it can be seen that the pulse number of 81300 is closest to the pulse number of 80755, which is the number of pulses at the center position of the 7th well, and that the well number in the X direction is closest to 7.

Next, similar calculations are performed for the Y direction, and it is found that the well number in the Y direction is B. If character strings are to be displayed in the stage position display areas 122 and 123, the character strings "7" and "B" are added to the character string variables STR_wel_X and STR_wel_Y, respectively. can be easily displayed by inputting
Since this is an example using the 96-well plate shown in FIG. 19, the numerical values in the tables shown in FIGS. 13 and 14 were used. I do.

FIG. 20 is a diagram showing a control method when instructions are given to move the stage in the X and Y directions. Specifically, numerical values are entered in the movement target coordinate (X) input area 106 and the movement target coordinate (Y) input area 107 in the graphic user interface 90, and control is performed when the movement start button 108 to the target position is pressed. FIG. 4 is an explanatory diagram of a flow;
First, the cumulative pulse information X(now) in the X direction and the cumulative pulse information Y(now) in the Y direction, which are the current position information, are read. Next, pulse information X(end) and Y(end) of target positions in the X direction and Y direction are calculated from the position information of the movement destination. This calculation process can be calculated from 2.54 millimeters is 3200 pulses. The direction of movement is calculated by comparing the pulse information at the current position and the pulse information at the destination. That is, when X (end) of the destination is larger than X (now), the direction of movement of X is on the positive side, so the direction of movement information DIR(X) is set to 1, and X (end) of the destination is set to 1. is smaller than X(now), the moving direction of X is on the negative side, so the moving direction information DIR(X) is set to 0. The numbers of transmission pulses P(X) and P(Y) are calculated from the difference between these numerical values as the amount of movement. The numbers of transmitted pulses are P(X) and P(Y) positive numbers.

Here, the movement range is limited in the X direction and the Y direction as described above. 105070 pulses is the allowable maximum pulse number Y (limit) for 83.4 mm, which is the distance from the dashed line 81d to the dashed line 81e in the Y direction. Therefore, before moving, the destination pulse numbers X (end) and Y (end) are compared with X (limit) and Y (limit) respectively, and X (end) and Y (end) are respectively X (limit) and Y (end). If it is larger than Y (limit), or if X (end) and Y (end) are negative numbers, the message display area 103 displays that the allowable range is exceeded without moving.

If the destination is within the allowable range, the digital controller 51 and the pulse generator 52 in the motor controller box 31 are sent the movement method DIR(X), DIR(Y) and the number of pulses P(X), P(Y). to move the stage. After the movement of the stage is completed, the well number estimation shown in FIG. 19 is performed, the well number is displayed, and the accumulated pulse information of the current position is updated. Further, an image is captured by the camera 12, and the captured image is displayed in one or all of the first window 91, the second window 92, and the third window 93. FIG. When an image is captured by the camera 12, the image is recorded in a cache memory in preparation for storing the captured image, and time information (year/month/day/hour/minute/ seconds) is also recorded in the cache memory.

FIG. 21 is a diagram showing a control method when an instruction to move the stage in the X and Y directions is given by well number input. Specifically, when numerical values are entered in the movement target well number (X) input area 109 and the movement target well number (Y) input area 110 in the graphic user interface 90 and the movement start button 111 to the target position is pressed. is an explanatory diagram of the control flow of .

First, in the graphic user interface 90, it is checked whether well plate information has been input in the well plate selection/display area 104. If there is no information, a message indicating that the well plate information has not been input is displayed in the message display area 103. and do not move.

If the well plate information is entered in the well plate selection/display area 104, the X direction cumulative pulse information X (now) and the Y direction cumulative pulse information Y (now), which are the current position information, are read. . Next, pulse information X(end) and Y(end) of target positions in the X direction and Y direction are calculated from the position information of the movement destination. This calculation is obtained from the pulse information corresponding to the central position of each well number shown in the table of FIG. 13 or 14 in the case of a 96-well plate. The direction of movement is calculated by comparing the pulse information at the current position and the pulse information at the destination. That is, when X (end) of the destination is larger than X (now), the direction of movement of X is on the positive side, so the direction of movement information DIR(X) is set to 1, and X (end) of the destination is set to 1. is smaller than X(now), the moving direction of X is on the negative side, so the moving direction information DIR(X) is set to 0. The numbers of transmission pulses P(X) and P(Y) are calculated from the difference between these numerical values as the amount of movement. The numbers of transmitted pulses are P(X) and P(Y) positive numbers.

In the case of this well number input, it is expected that all wells are within the movable range. (end) is compared with X (limit) and Y (limit) respectively, and if X (end) and Y (end) are greater than X (limit) and Y (limit) respectively, or X (end), If Y(end) is a negative number, the message display area 103 displays that the movement is not performed and the allowable range is exceeded.
The steps of moving the stage, estimating the well number, rewriting the current position information, capturing an image, displaying the image, etc. are performed in the same manner as in the example shown in FIG.

FIG. 22 is a diagram showing a control method when instructions to move the stage in the X and Y directions are specified as relative positions. Specifically, it is an explanatory diagram of the control flow when buttons 130 to 137 for simultaneously moving 10 microns in the long side direction and the short side direction in the graphic user interface 90 are pressed. In this case, information about the movement pulse and direction of movement can be obtained by pressing the button. Therefore, after obtaining the current pulse position information, the destination pulse position information X(end) and Y(end) can be calculated according to the button type.

Destination pulse numbers X (end) and Y (end) are compared with X (limit) and Y (limit) respectively, and X (end) and Y (end) are obtained from X (limit) and Y (limit) respectively. is larger, or if X (end) and Y (end) are negative numbers, the message display area 103 displays that the allowable range is exceeded without moving.
If the destination is within the allowable range, the digital controller 51 and the pulse generator 52 in the motor controller box 31 are sent the movement method DIR(X), DIR(Y) and the number of pulses P(X), P(Y). to move the stage. After the movement of the stage is completed, the well number estimation shown in FIG. 19 is performed, the well number is displayed, and the accumulated pulse information of the current position is updated. Further, an image is captured by the camera 12, and the captured image is displayed in one or all of the first window 91, the second window 92, and the third window 93. FIG.

FIG. 23 is a second diagram showing a control method when instructions to move the stage in the X and Y directions are designated as relative positions. Specifically, it is an explanatory diagram of the control flow when buttons 140 to 147 for simultaneously moving 100 microns in the long side direction and the short side direction in the graphic user interface 90 are pressed. In this case, since the amount of movement is relatively large, the movement is performed at once. The amount of one movement is 20 microns, which is 20% of the target amount of movement. This is the process of actually capturing and displaying an image. An image is captured and displayed every time the stage is moved by 20 microns, which is a convenient function when the operator is looking for an observation target.
As in FIG. 22, information about the movement pulse and direction of movement can be obtained by pressing the button. Therefore, after obtaining the current pulse position information, the destination pulse position information X(end) and Y(end) can be calculated according to the button type.

Destination pulse numbers X (end) and Y (end) are compared with X (limit) and Y (limit) respectively, and X (end) and Y (end) are obtained from X (limit) and Y (limit) respectively. is larger, or if X (end) and Y (end) are negative numbers, the message display area 103 displays that the allowable range is exceeded without moving.
If the destination is within the allowable range, the digital controller 51 and the pulse generator 52 in the motor controller box 31 are sent the movement method DIR(X), DIR(Y) and the number of pulses P(X), P(Y). to move the stage. Then, when the movement is completed, image capturing and display are performed. Then, the movement process, the image capturing process, and the image display process are repeated five times until the total amount of movement reaches the instructed amount of movement.
After the stage has moved five times, the well number is estimated as shown in FIG. 19, the well number is displayed, and the accumulated pulse information at the current position is updated.

FIG. 24 is a flow chart for generating an image storage file name in the image storage method of the cooperative control system of the two-axis moving stage and camera and the cooperative control system of two-axis moving stage and camera. When an image captured by a conventional camera in which the two-axis moving stage and the camera are not linked is saved using software provided by the camera manufacturer, only the information about the capturing time is given, but the present invention Since the two-axis moving stage and the camera are controlled in cooperation, useful information such as position information can be inserted into the file name. First, when text such as GFP1 is input in the sample information input/display area 105, the text information is acquired and GFP1 is input to the character string variable STR_sample. Next, when the image is captured by the camera 12, the capturing time information stored in the cache is acquired together with the image, and _20210703101223, for example, is input to the character string variable STR_time. Further, current stage position information is calculated from X(now) and Y(now), which are stage position information in the X direction or Y direction. For example, when X (now) is 81,300 pulses as in the example shown in FIG. 9, the distance is 61.5 mm. Since it is 0.4 mm, a numerical value of 21400 is used, and _X64500_Y21400 is input to the position information character string variable STR_posi.

Next, in the graphic user interface 90, it is checked whether well plate information is entered in the well plate selection/display area 104, and if there is no information, a space is entered in STR_well, which is a character string variable of well number information. , if there is information in the well plate selection/display area 104, if well plate information is entered in STR_well, the well number is estimated from that information, and together with the well number information of the well plate, STR_well , such as _wel96_7B. Then, the image file name is completed by merging each of the above character number variables together with the image format type information. In this example, the tiff format is selected as the image format, and the file name is GFP1_20210703101223_X64500_Y21400_wel96_7B. tiff. This file name contains not only the date and time of image capture, but also the location and well number of the image capture.

FIG. 25 shows a method of generating a character string variable indicating a well number into which a file name is inserted. As for the specific steps, the method of generating the character string variable STR_wel_X for the well number in the X direction and the character string variable STR_wel_Y for the well number in the Y direction shown in FIG. 19 is the same. The variable SRT_wel_header is generated from the well plate information entered in the well plate selection/display area 104 in the graphic user interface 90, and by merging these, the character string variable STR_wel shown in FIG. 24 becomes wel96_7B.

FIG. 26 is a diagram showing a method of generating an enhanced image displayed on the second window 92. As shown in FIG. If the image is a black-and-white image in both the low-luminance region and the high-luminance region, it is not easy to visually recognize a small change in luminance. There is an image enhancement method that can sensitively capture luminance changes by imaging. FIG. 26 shows a color mapping method that is an example of assigning red, green, and blue color image data to a monochrome image. can be easily observed by the observer.
For example, a pixel with a brightness of 3 in a black-and-white image has red 0, green 0 . A pixel with a brightness of 4 is converted to blue 143, and a pixel with a brightness of 4 is converted to red 0, green 0, and blue 147, so that the brightness change is 4, which is a fourfold brightness change.

When the object to be observed is a sample with weak fluorescence emission, a method of adjusting the focus by changing to a high-magnification objective lens with a high NA or adjusting the focus by increasing the brightness of the illumination is used. Therefore, in the case of a sample that easily fades, there is a problem that the observable time is shortened. Also, when looking for an object to be observed, a low-magnification lens that can observe a wide area often has a low NA. Nevertheless, a method of using a high-magnification objective lens with a high NA has also been used.
By using this image enhancement method called color mapping, there is an advantage that it is possible to adjust the focus position even in a dark image, and when searching for an object, it is possible to reduce the magnification without exchanging to a high NA high magnification objective lens. There is an advantage that it becomes easier to find an object with a lens of . In addition, since the work of changing the magnification of the objective lens and changing the camera conditions can be reduced, an effect of shortening the work time can be obtained.

FIG. 27 is a diagram showing a schematic configuration diagram of a second optical microscope 210 to which the two-axis moving stage and camera cooperation control system of the present invention is connected.
The optical microscope 10 shown in FIG. 3 to which the two-axis moving stage and the camera cooperation control system of the present invention are connected is an inverted microscope in which the objective lens 63 is arranged upward. The microscope to which the coordinated control system of the camera and the camera is connected is not limited to the inverted microscope. The optical microscope 210 shown in FIG. 27 has a wafer 25 on which a semiconductor is formed as a sample mounted on a two-axis moving stage 30 . Illumination light from a light source device 76 such as a halogen lamp light source and a condenser lens 77 is incident on the objective lens 63 via a half mirror 78, forming a coaxial epi-illumination optical system. . When a defective portion such as a scratch is observed on the wafer, the operator uses the graphic user interface 90 in the cooperative control system of the two-axis moving stage and the camera to capture and save the image. In the image of the wound, position information is recorded together with information on the date and time when the image was taken. Therefore, it is possible to easily leave the coordinates of information such as scratches. In addition, since an image enhancement method such as a color mapping function is effective in finding fine shallow scratches, the use of the image enhancement method makes it easier to find flaws.

FIG. 28 is a diagram showing an optical microscope 9 to which the two-axis moving stage and camera cooperation control system of the present invention is connected. In the optical microscope apparatus 8 shown in FIG. 2, the rigidity of the microscope frame 1 is high, and even if the sample table 3 is removed, the transmission observation image of the sample can be observed through the eyepiece 68 by the operator and the camera. The optical microscope 9 shown in FIG. 28 has a structure in which the sample stage 3 supports the rigidity of the optical system. The rigidity of the body 2 is insufficient, and the observation of the transmission observation image of the sample is reduced in image quality observed by the operator observing through the eyepiece 68 and the camera connected to the camera port 18. It is a microscope. However, since the optical microscope 9 is lightweight, it is often used in cell culture sites.
The image quality is degraded because the position of the transmission illumination light source 73 and the condenser lens 74 becomes unstable due to the removal of the sample stage 3 .

FIG. 29 is a diagram showing a schematic configuration diagram of a third optical microscope 211 to which a two-axis moving stage and a camera cooperation control system of the present invention are connected. This is an example in which a cooperative control system for a two-axis moving stage and a camera composed of a two-axis moving stage 230, a camera 12, a controller 13, and a motor controller box 31 is connected without removing them. Here, the camera 12, the controller 13, and the motor controller box 31 are the same as those in the example shown in FIG. 3, so their description is omitted.

The optical microscope 9 shown in FIG. 28 is configured so that the sample placed in the wells 21 can be observed when the well plate 20 is placed on the sample stage 3. Therefore, the optical microscope 9 shown in FIG. If the two-axis moving stage 30 shown in FIG. It has a structure different from that of the moving stage 30 .

FIG. 30 shows a schematic diagram of the configuration of the two-axis moving stage 230. As shown in FIG. A two-axis moving stage 230 shown in FIG. 30 has a stage fixing portion 37 attached to the sample stage 3 , and a Y-axis motor 34 is arranged on the stage fixing portion 37 . The Y-axis motor 34 has the function of rotating the Y-axis shaft 36 , and by rotating the Y-axis motor 36 , the Y-stage table 39 mounted on the Y-axis shaft 36 is parallel to the rotation axis of the Y-axis shaft 36 . It is configured to move in any direction. In this example, the Y-axis shaft 36 is formed with a screw with a pitch of 2.54 mm. It is configured to move 54 mm.

An X-axis motor 33 is arranged on the Y-stage table 39 . The X-axis motor 33 has a function to rotate the X-axis shaft 35 , and by rotating the X-axis motor 33 , the X-stage table 38 mounted on the X-axis shaft 35 is parallel to the rotation axis of the X-axis shaft 35 . It is configured to move in any direction. In this example, the X-axis shaft 35 is threaded with a pitch of 2.54 mm like the Y-axis shaft 36 . It is configured to move 2.54 mm in the direction corresponding to . The X-axis shaft 35 and the Y-axis shaft 36 are arranged to be orthogonal to each other, and the X-axis shaft 35 and the Y-axis shaft 36 are also configured to be orthogonal to the observation optical system including the objective lens 63 and the imaging lens 64. .

A well plate 20 is mounted on the sample mounting portion 29 of the biaxially movable stage 330, similarly to the biaxially movable stage 30 shown in FIG. The well plate 20 has two chamfers 22 arranged on one side in the longitudinal direction to define the orientation of the well plate. A well plate retainer 40 that holds down the chamfered portion 22 of the well plate 20 is arranged on the biaxial movement stage 230 . is pressed down.

In this example, the well plate 20 is arranged so that the long side direction is parallel to the rotation axis of the X-axis shaft 35, and the short side direction of the well plate 20 is parallel to the rotation axis of the Y-axis shaft 36. are arranged so that Although not shown in FIG. 30 , limit sensors and the like are provided in the same way as the two-axis moving stage 30 , and the control method thereof is the same as that of the two-axis moving stage 30 .

FIG. 31 shows a partial see-through configuration diagram of the third optical microscope 211 to which the cooperative control system of the two-axis moving stage and the camera of the present invention is connected, and FIG. 32 shows the two-axis moving stage of the present invention. 3 shows a perspective configuration diagram of a third optical microscope 211 to which a coordinated control system of 1 and a camera is connected, as viewed from the Y direction.

The stage fixing portion 37 is attached to the sample stage 3 via the attachment plate 17 . The stage fixing portion 37 is located on the sample placement side, which is the upper surface of the sample table 3 , but the Y-axis motor 34 and the Y-axis shaft 36 are located on the objective lens side of the sample table 3 . Also, the Y-axis motor 34 and the Y-axis shaft 36 are arranged outside the sample stage 3 in the X direction. As shown in FIG. 28, the sample table 3 in the optical microscope 9 has a structure in which the sides facing the eyepiece lens and camera port 18 and the strut of the transmitted illumination 73 are fixed, so that the sample can be moved in the X direction. As for the X-axis motor 33 and the X-axis shaft 35 that move the sample in the Y direction, either the Y-axis motor 34 or the Y-axis shaft 36 that moves the sample in the Y direction is arranged outside the sample stage 3 and the objective lens If it is arranged on the side, it is easier to select the Y-axis motor 34 and the Y-axis shaft 36 for moving the sample in the Y direction.

When the X-axis motor 33 and the X-axis shaft 35 for moving the sample in the X direction are arranged on the objective lens side of the sample table 3, the method of assembling after removing the sample table 3, or the method of assembling the two-axis moving stage 230 from both sides of the sample table 3, the Y-axis motor 34 and the Y-axis shaft 36 for moving the sample in the Y direction are placed on the objective lens side of the sample table 3, and In the case of arranging it outside the sample table 3, the assembled two-axis movement stage 230 can be installed. 3 on the objective lens side and on the outside of the sample table 3.

The two-axis moving stage 230, in which the Y-axis motor 34 and the Y-axis shaft 36 shown in FIG. Although it can be attached to an optical microscope 9 in which the rigidity of the microscope body 2 is insufficient, the optical microscope 8 in which the rigidity of the microscope body 1 is sufficient when the sample stage 3 shown in FIG. Since the Y-axis motor 34 and the Y-axis shaft 36 are positioned closer to the objective lens than the sample table 3, the optical microscope to which the two-axis moving stage 230 is attached can be used even if the sample table 3 is removed. It need not be limited to optical microscopes, where the stiffness of the microscope body 2 is insufficient.

2-axis moving stage and camera cooperative control system, image storage method of 2-axis moving stage and camera cooperative control system, stage moving method of 2-axis moving stage and camera cooperative control system, 2-axis moving stage and camera of the present invention The image capture and display method of the coordinated control system and the position information display method of the coordinated control system of the two-axis moving stage and the camera are useful in areas where multiple conditions are examined for cells, such as drug development and treatment development. It can bring about improvement of work efficiency and reduction of misidentification of cell number. In addition, although the apparatus was conventionally applied only to microscopes with high rigidity and effective automatic stages, the present invention can be applied to inexpensive optical microscopes, and does not handle cells such as wafer inspection. Since it can be applied to microscopes used by users, work efficiency can be improved in many aspects.

1, 2... microscope frame, 3... sample stage, 8, 9... microscope system, 10, 210, 211, 310... microscope system connected to XY stage camera integration system, 11... two-axis movement Coordinated control system for stage and camera 12 Camera 13 Controller 14 Monitor 15 Camera cable 16 Monitor cable 17 Mounting plate 18 Camera port 19 Stage mounting portion 20 Well plate 21 Well 22 Chamfered portion 25 Wafer 29 Sample mounting portion 30, 230 XY stage 31 Motor controller box 32 ... motor cable, 33 ... X-axis motor, 34 ... Y-axis motor, 35 ... X-axis shaft, 36 ... Y-axis shaft, 37 ... Stage fixed part, 38 ... X stage table, 39 ... Y Stage table 40 Well plate holder 41 Shaft holder 42, 42 Limit sensor 44, 45 Limit sensor blade 46 X-axis motor driver 47 Y-axis motor driver 48 ...X-axis motor driver cable, 49...Y-axis motor driver cable, 50...Motor controller, 51...Digital controller, 52...Pulse generator, 53...X-axis pulse transmission cable, 54...Y-axis Pulse transmission cable 55 Digital controller cable 56 Motor controller cable 60 Fluorescence excitation light source 61 Lens 62 Fluorescence filter cube 63 Objective lens 64 Imaging lens 65, 66...Mirror 67...Switching mirror 68...Eyepiece lens 70...Excite filter 71...Dichroic mirror 72...Emission filter 73...Transmitted illumination light source 74, 77...Collection Optical lens 76 Illumination light source 78 Half mirror 80 Movable range defined by limit sensor 80a Mechanical origin 81 Software movable range 81a Software origin 82 . . . X-axis direction movement amount (stage movement distance) 83 .. Y-axis direction movement amount (stage movement distance) 90 . ……Origin return button 102 ……Stage control state display area 103 ……Message display Area 104 Well plate selection/display area 105 Sample information input/display area 106 Movement target coordinate (X) input area 107 Movement target coordinate (Y) input area 108, 111 --Movement start button to target position, 109 --- Movement target (X-axis well number) input area, 110 --- Movement target (Y-axis well number) input area, 120 --- Stage position display (X-coordinate), 121 --- Stage position display (Y coordinate) 122 Stage position display (X-direction well number) 123 Stage position display (Y-direction well number) , 140, 141, 142, 143, 144, 145, 146, 147 -- relative movement designation button, 150, 151 -- camera condition input/display area, 152 -- image enhancement condition input/display area, 153 -- enlargement Display condition input/display area 154 Continuous shooting/display button 155 Image shooting button 156 Image saving button

Claims (26)

A two-axis moving stage on which a sample to be observed with an optical microscope is mounted, a camera device for capturing an optical microscope observation image of the sample to be observed with an optical microscope, and a two-axis moving stage and an image capturing device that are controlled and captured by the camera. In the coordinated control system of the 2-axis moving stage and the camera, which consists of a controller with a function to save images,
Coordinated control of the 2-axis moving stage and the camera, wherein the controller generates an image save name using stage position information of the 2-axis moving stage when saving the image captured by the camera device. system In the coordinated control system of the two-axis moving stage and the camera,
The sample observed with the optical microscope is placed in a well of a well plate having a plurality of wells,
The generated image storage name includes well number information estimated by comparing the position information of each well configured according to the number of wells of the well plate and the stage position information of the two-axis moving stage. A coordinated control system for a two-axis moving stage and a camera according to claim 1 In the coordinated control system of the two-axis moving stage and the camera,
The well number information estimated by comparing the position information of each well and the stage position information of the two-axis moving stage includes two pieces of well number information estimated from the position information of each axis of the two-axis moving stage. A coordinated control system for a two-axis moving stage and a camera according to claims 1 and 2, characterized in that In the coordinated control system of the two-axis moving stage and the camera,
The control of the two-axis moving stage and the camera device by the controller is configured to be instructed by a graphic user interface displayed on the monitor screen,
4. A coordinated control system for a two-axis moving stage and a camera according to any one of claims 1 to 3, wherein an instruction to save an image is given via a graphic user interface. In the coordinated control system of the two-axis moving stage and the camera,
The control of the two-axis moving stage and the camera device by the controller is configured to be instructed by a graphic user interface displayed on the monitor screen,
The position information of each well used for estimating the well number information that constitutes the image storage name is
5. A coordinated control system for a two-axis moving stage and a camera according to any one of claims 1 to 4, characterized in that the state selected by the number of wells selection function of the well plate in the graphic user interface is reflected. In the coordinated control system of the two-axis moving stage and the camera,
In the graphic user interface displayed on the monitor screen, there is an area for selecting the well number, and by selecting the well number,
3. The biaxial movement stage moves so that the selected well number is arranged in the observation field of the microscope according to the position information of each well configured according to the number of wells of the well plate. Coordinated control system for two-axis moving stage and camera according to any one of 1 to 5 In the coordinated control system of the two-axis moving stage and the camera,
In the graphic user interface displayed on the monitor screen, there is an area for selecting movement of either or both of the two-axis moving stages along with their moving directions. 7. A coordinated control system for a two-axis moving stage and a camera according to any one of claims 1 to 6, wherein In the coordinated control system of the two-axis moving stage and the camera,
8. A coordinated control system for a two-axis moving stage and a camera according to any one of claims 1 to 7, wherein after the two-axis moving stage moves, a photograph is taken and displayed on a monitor screen. In the coordinated control system of the two-axis moving stage and the camera,
9. The coordinated control system of the two-axis moving stage and the camera according to any one of claims 1 to 8, wherein the image displayed on the monitor screen is a color map image. In the coordinated control system of the two-axis moving stage and the camera,
After the 2-axis translation stage moves,
10. A cooperative control system for a two-axis moving stage and a camera according to any one of claims 1 to 9, wherein stage position information of the two-axis moving stage is displayed in a graphic user interface. In the coordinated control system of the two-axis moving stage and the camera,
After the 2-axis translation stage moves,
The well number information estimated by comparing the position information of each well configured according to the number of wells of the well plate and the stage position information of the two-axis moving stage is displayed in the graphic user interface. 11. Cooperative control system for two-axis moving stage and camera according to any one of items 1 to 10 In the coordinated control system of the two-axis moving stage and the camera,
12. A coordinated control system for a two-axis moving stage and a camera according to any one of claims 1 to 11, wherein the optical microscope is an optical microscope equipped with an eyepiece. In the coordinated control system of the two-axis moving stage and the camera,
When attaching the two-axis movement stage to the optical microscope, the two-axis movement stage is attached without removing the sample stage in the optical microscope before attaching the two-axis movement stage,
13. The method according to any one of claims 1 to 12, wherein the motor drive shafts of the two axes with short movement distances constituting the biaxial movement stage are positioned closer to the objective lens than the sample stage in the optical microscope. Coordinated control system of 2-axis moving stage and camera A two-axis moving stage on which a sample to be observed with an optical microscope is mounted, a camera device for capturing an optical microscope observation image of the sample to be observed with an optical microscope, and a two-axis moving stage and an image capturing device that are controlled and captured by the camera. In the coordinated control system of the 2-axis moving stage and the camera, which consists of a controller with a function to save images,
Coordinated control of the two-axis moving stage and the camera, wherein the controller uses the stage position information of the two-axis moving stage as a part of the name of the image to be saved when the image captured by the camera device is saved. How the system saves images A two-axis moving stage on which a sample to be observed with an optical microscope is mounted, a camera device for capturing an optical microscope observation image of the sample to be observed with an optical microscope, and a two-axis moving stage and an image capturing device that are controlled and captured by the camera. In the coordinated control system of the 2-axis moving stage and the camera, which consists of a controller with a function to save images,
The sample observed with the optical microscope is placed in a well of a well plate having a plurality of wells,
The image storage name generated when the image captured by the camera device is stored by the controller includes position information of each well configured according to the number of wells of the well plate and stage position information of the biaxial movement stage. An image storage method for a coordinated control system of a two-axis moving stage and a camera, characterized by including well number information estimated by comparing In the image storage method of the cooperative control system of the two-axis moving stage and the camera,
The well number information estimated by comparing the position information of each well and the stage position information of the two-axis moving stage includes two pieces of well number information estimated from the position information of each axis of the two-axis moving stage. 16. The image storage method of the cooperative control system of the two-axis moving stage and the camera according to claim 15, characterized in that In the image storage method of the cooperative control system of the two-axis moving stage and the camera,
The control of the two-axis moving stage and the camera device by the controller is configured to be instructed by a graphic user interface on the monitor screen,
17. An image storage method for a cooperative control system of a two-axis moving stage and a camera according to any one of claims 14 to 16, wherein an instruction to store the image is given via a graphic user interface. In the image storage method of the cooperative control system of the two-axis moving stage and the camera,
The control of the two-axis moving stage and the camera device by the controller is configured to be instructed by a graphic user interface on the monitor screen,
The position information of each well used for estimating the well number information that constitutes the image storage name is
18. The image storage method of the two-axis movement stage and camera cooperative control system according to any one of claims 15 to 17, wherein the state selected by the well number selection function of the well plate in the graphic user interface is reflected. In the image storage method of the cooperative control system of the two-axis moving stage and the camera,
19. The image storage method of the two-axis moving stage and camera cooperation control system according to any one of claims 14 to 18, wherein the optical microscope is an optical microscope equipped with an eyepiece. A two-axis moving stage on which a sample to be observed with an optical microscope is mounted, a camera device for capturing an optical microscope observation image of the sample to be observed with an optical microscope, and a two-axis moving stage and an image capturing device that are controlled and captured by the camera. In the coordinated control system of the 2-axis moving stage and the camera, which consists of a controller with a function to save images,
In the graphic user interface, there is an area for selecting the well number, and by selecting the well number,
A 2-axis moving stage is moved so that a selected well number is arranged in a field of view of a microscope according to position information of each well configured according to the number of wells of a well plate. Stage movement method of coordinated control system of movement stage and camera In the stage moving method of the cooperative control system of the two-axis moving stage and the camera,
21. The stage moving method of the two-axis moving stage and camera cooperative control system according to claim 20, wherein the optical microscope is an optical microscope equipped with an eyepiece. A two-axis moving stage on which a sample to be observed with an optical microscope is mounted, a camera device for capturing an optical microscope observation image of the sample to be observed with an optical microscope, and a two-axis moving stage and an image capturing device that are controlled and captured by the camera. In the coordinated control system of the 2-axis moving stage and the camera, which consists of a controller with a function to save images,
A method of photographing and displaying an image of a coordinated control system of a two-axis moving stage and a camera, characterized in that after the two-axis moving stage moves, the image is captured by the camera device and displayed on a monitor screen. In the image capturing and display method of the coordinated control system of the two-axis moving stage and the camera,
23. The image capturing and displaying method of the two-axis moving stage and camera cooperative control system according to claim 22, wherein the image displayed on the monitor screen is an image including a color map image. In the image capturing and display method of the coordinated control system of the two-axis moving stage and the camera,
24. The image capturing and display method of the two-axis moving stage and camera cooperative control system according to claims 22 and 23, wherein the optical microscope is an optical microscope equipped with an eyepiece. A two-axis moving stage on which a sample to be observed with an optical microscope is mounted, a camera device for capturing an optical microscope observation image of the sample to be observed with an optical microscope, and a two-axis moving stage and an image capturing device that are controlled and captured by the camera. In the coordinated control system of the 2-axis moving stage and the camera, which consists of a controller with a function to save images,
After the 2-axis translation stage moves,
2. Well number information estimated by comparing the position information of each well configured according to the number of wells of the well plate and the stage position information of the two-axis moving stage is displayed in the graphic user interface. Positional Information Display Method of Coordinated Control System for Axis Movement Stage and Camera In the position information display method of the coordinated control system of the two-axis moving stage and the camera,
26. The method of displaying position information of a cooperative control system of a two-axis moving stage and a camera according to claim 25, wherein the optical microscope is an optical microscope equipped with an eyepiece.

Images