JP2023032674A - Microscope auxiliary device, control method of the same and program - Google Patents

Abstract

To shorten the time required for the work of arranging operation means in the vicinity of a work surface without requiring particular means for observing the operation means.SOLUTION: After making a main operation part 10M and a subordinate operation part 10S focused on a focus region of a microscope 100 in an observation range by using a result of analyzing an image obtained by imaging the main operation part 10M and the subordinate operation part 10S operating an object 103 under a visual field of the microscope 100 within the observation range in the microscope 100, the main operation part 10M is moved in the optical axis direction of the microscope 100 to bring it into contact with a work surface 102f or to position it in the vicinity of the work surface in such a state that the focus of the microscope 100 is placed on the work surface 102f where there is the object 103 or the vicinity of the work surface, then, the subordinate operation part 10S is moved in the optical axis direction to a prescribed position according to the position of the main operation part 10M or the object 103, the work preparation for the main operation part 10M and the subordinate operation part 10S to the object 103 is performed.SELECTED DRAWING: Figure 1

Description

The present invention relates to a microscope assisting device and its control method and program.

In the field of life science, for example, operations such as sorting cells under a microscope field and injecting sperm into an egg for in vitro fertilization are performed. Further, in the field of microfabrication, operations such as picking up foreign matter mixed in a semiconductor substrate and measuring the potential of a minute portion with a probe under the field of view of a microscope are performed. These tasks are generally performed using a microscope fitted with microscope aids. A microscope auxiliary device includes a movable part that moves an operating means such as a pipette, a probe, or tweezers that mechanically manipulates an object to be manipulated, and is generally called a "micromanipulator" or simply a "manipulator." .

The movable portion of the microscope auxiliary device uses a mechanical mechanism, a mechanism that converts manual operation into mechanical power by fluid pressure, a motor that is driven by electric power, and the like. Further, the Z direction is defined as the optical axis direction of the microscope, the X direction is defined as the horizontal direction when viewed from the user, and the Y direction is defined as the depth direction when viewed from the user. There is a microscope auxiliary device that is configured to be independently movable in the direction and the Z direction.

Since work using the microscope auxiliary device is often performed near the surface of a petri dish or semiconductor substrate (hereinafter referred to as the "work surface"), the tip position of the operating means in the optical axis direction (Z direction) of the microscope must be adjusted. It must be controlled with high precision. However, when the tip of the operating means or the operating means itself needs to be replaced for each task, the position of the tip of the operating means is not constant due to mounting errors or manufacturing errors between individual pieces of the replaced operating means. Therefore, it is not easy to move the operation means to the vicinity of the work surface even with an electric microscope auxiliary device having an encoder.

Therefore, conventionally, when the tip of the operating means or the operating means itself is replaced, the work of adjusting the position of the tip of the operating means to the focus after slightly lowering the focus of the microscope is repeated, and the operating means is placed near the work surface. leading the way. However, such a method takes a long time to complete the work. For example, in the field of handling cells, there is a demand for shortening the time required to move the operating means to the work surface in order to complete the work in a short time.

In response to this problem, Patent Literature 1 discloses a technique of estimating the position of the operating means by moving a part of a holding member that holds the operating means into the field of view of the microscope. In addition, Patent Document 2 discloses a positioning plate having a mark having a reference point and a reference line for positioning the tip of the operating means within the field of view of the microscope, a set point provided at a predetermined position with respect to the reference point, and the like. ing.

JP 2007-140557 A JP-A-1-219535

For example, in the case of working on a semiconductor substrate or working while observing an object at high magnification, such as cell manipulation, the depth of field is shallow and it is necessary to focus on the semiconductor substrate or Petri dish, which is the work surface. In such a case, with the technique disclosed in Patent Document 1, it is not easy to move even the holding member of the operation means into the field of view of the microscope, and therefore the position of the operation means can be estimated. It takes time to Further, with the technique disclosed in Patent Document 2, it is possible to improve the work in the X and Y directions, but it is not easy to improve the work in the Z direction.

As another method for solving the above problem, a method of measuring the position of the operation means in the Z direction by adding a camera for observing the operation means from the X direction and the Y direction other than the Z direction is conceivable. However, for example, when trying to observe an operating means with a width of several micrometers at high magnification, the depth of field becomes shallow and it is not easy to focus. In addition, the camera requires a high-magnification lens capable of imaging semiconductor substrates and scales for manipulating cells, and a long-focal-length lens that enables observation from a position that avoids the object and its peripheral equipment. Therefore, an increase in the size of the camera is inevitable. Therefore, it may be difficult to newly install a camera for observing the operating means from a direction other than the Z direction in a state where the microscope auxiliary device is installed, and the installation of the camera also poses a problem of cost increase.

SUMMARY OF THE INVENTION The present invention has been made in view of such circumstances, and is intended to shorten the time required for the work of placing an operating means near a work surface without requiring a separate means for observing the operating means. It is an object of the present invention to provide a microscope assisting device that makes it possible.

A microscope assisting device according to the present invention comprises at least one main operating section and at least one secondary operating section for manipulating an object under the field of view of a microscope, driving means for driving the main operating section and the secondary operating section, Analysis means for analyzing an image obtained by capturing an observation range of the microscope; and a control means for performing the step and the third step in this order, wherein the first step includes a state in which the main operation section and the secondary operation section are focused on the focus area of the microscope in the observation range. and the second step is such that, with the microscope focused on or near a work surface on which the object is located, the main operation unit is in contact with or is positioned near the work surface. and a step of moving the main operation section in the direction of the optical axis of the microscope. It is characterized by being a step of moving in the axial direction.

According to the present invention, there is provided a microscope assisting device that does not require a separate means for observing the operating means, and can shorten the time required for the work of placing the operating means near the work surface. be able to.

1 is a diagram showing a schematic configuration of a microscope system according to an embodiment; FIG. 1A and 1B are a diagram showing a schematic configuration of a microscope system and a block diagram of a PC; FIG. FIG. 3 is a diagram showing the structure of a main operation section; FIG. 4 is a diagram showing the structure of a slave operation section; FIG. 4 is a diagram showing an example of arrangement of a main operation section and a sub-operation section before starting work on an object; FIG. 4 is a schematic diagram illustrating a first step of a preparatory step in the first embodiment for arranging the main operation section and the secondary operation section; It is a schematic diagram explaining the 2nd process in a preparation process. FIG. 11 is a front view showing a state in which the main operation unit is moved to an arbitrary position after completion of the second step; FIG. 11 is a plan view showing an example of the positional relationship on the XY plane between the distal end portions of the main operating section and the secondary operating section after the completion of the second step; It is a front view explaining the 3rd process in a preparation process. FIG. 11 is a diagram for explaining a second driving mode of the placement function in the third step; FIG. 11 is a diagram for explaining a third driving mode of the placement function in the third step; FIG. 10 is a diagram showing a reference example of a driving mode of the slave operating section in the third step; 4 is a flow chart for explaining a preparation process in the first embodiment; FIG. 11 is a front view showing a state in the middle of a second step in a preparation step in the second embodiment for arranging the main operation section and the secondary operation section; FIG. 10 is a front view showing a state in which the tip of the main operation portion is in contact with the work surface in the second step; It is a front view which shows the state at the time of implementation of a 3rd process. FIG. 11 is a diagram showing an example of arrangement of a main operating section and a sub-operating section in the third embodiment;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. INDUSTRIAL APPLICABILITY A microscope assisting device according to the present invention is attached to a microscope and used to mechanically manipulate an object to be manipulated such as, for example, minute cells or foreign matter adhering to a semiconductor element. Here, the present invention will be described primarily with reference to microscope systems used for cell manipulation in the life science field. However, the work target of the microscope system is not limited to cells as described in the background art.

<First embodiment>
FIG. 1 is a diagram showing a schematic configuration of a microscope system according to an embodiment of the invention. The microscope system has a microscope auxiliary device (hereinafter referred to as "auxiliary device"), a microscope 100, a personal computer 40 (hereinafter referred to as "PC 40"), and a monitor 200. FIG. 2(a) is a side view showing a schematic configuration of the microscope system. FIG. 2(b) is a block diagram of the PC 40. As shown in FIG. For convenience of explanation, as shown in FIGS. 1 and 2, a three-dimensional orthogonal coordinate system is set for the microscope system. The direction parallel to the optical axis direction of the microscope 100 is defined as the Z direction, and the upward direction as viewed from the user is defined as the positive direction (+Z direction). The left-right direction when the user (observer, user) views the microscope 100 perpendicular to the optical axis direction (Z direction) is defined as the X direction, and the right direction as viewed from the user is the positive direction (+X direction). and Furthermore, the direction perpendicular to the X direction and the Z direction is defined as the Y direction. The Y direction is the front-rear direction when the microscope 100 is viewed by the user, and the depth direction is defined as the positive direction (+Y direction).

The auxiliary device has a main operation movable section 1M, a secondary operation movable section 1S, and a controller 30 (drive circuit) connected to the PC 40. FIG. In the first embodiment, the auxiliary device includes one main operation movable section 1M and one sub-operation movable section 1S, but may have a plurality of them. The controller 30 includes a control section 2 that controls driving of the main operation movable section 1M and the secondary operation movable section 1S. Further, the auxiliary device includes a main input operation unit 20M for inputting the driving direction and driving amount of the main operation movable unit 1M to the controller 30, and a driving direction and driving amount for the slave operation movable unit 1S to the controller 30. sub-input operation unit 20S.

A microscope 100 illuminates a work object 103 (hereinafter referred to as “object 103”) placed on a transparent Petri dish 102 from an illumination optical system 101 arranged at the top, and observes it with an observation optical system 104 at the bottom. . The observation optical system 104 guides an optical image transmitted through the objective lens 104a to an eyepiece lens 104b via other lenses and a refractive optical system (not shown), and allows observation with the naked eye of a user looking into the eyepiece lens 104b. enable. The focal position with respect to the object 103 can be adjusted by moving the objective lens 104a in the Z direction. The objective lens 104a is moved by operating (here, rotating) a knob 105 provided on the microscope 100. FIG. A rotation operation of the knob 105 is converted into a minute movement amount of the objective lens 104a via a deceleration transmission mechanism (not shown).

Although not shown, the microscope 100 includes a plurality of objective lenses 104a ranging from low magnification to high magnification, and is configured so that the desired magnification can be set as appropriate. Further, the microscope 100 is equipped with a camera 106 capable of imaging the field of view (observation range) of the microscope 100 . The observation optical system 104 is provided with spectroscopy means (not shown), and the range observed by the microscope 100 is imaged by the camera 106 by the spectroscopy means, and can be saved as an image or video. .

One of the main operation movable section 1M and the sub-operation movable section 1S constituting the auxiliary device is generally arranged on the left side (-X side) of the microscope 100 and the other on the right side (+X side) of the microscope 100. In order to operate the object 103 under the field of view of the microscope 100, the master operation movable part 1M and the slave operation movable part 1S respectively move the master operation part 10M and the slave operation part 10S in the X direction, the Y direction and the Z direction. . Details of the main operation section 10M and the sub-operation section 10S will be described later.

The main operation section 10M may be configured to be rotatable about a predetermined axis by the main operation movable section 1M, and the same applies to the secondary operation movable section 1S. In addition, the main operation movable section 1M may separately include a high-speed drive mechanism for coarse movement and a high-resolution drive mechanism for fine movement, and the same applies to the slave operation movable section 1S.

The controller 30 has a control section 2, an image processing circuit 2d, a memory 2f, and an input reading circuit 2e, and controls driving of the main operation movable section 1M and the slave operation movable section 1S. The control unit 2 has a control amount calculation unit 2a, an M drive circuit 2b and an S drive circuit 2c.

The image processing circuit 2d obtains an image signal captured by the camera 106, performs image processing, and displays the object 103 on the monitor 200 as a video (image). Further, the image processing circuit 2d has a function of determining whether or not the tip portions of the main operation section 10M and the secondary operation section 10S and the object 103 are in focus. The memory 2 f holds various data required for the operation of the controller 30 and temporarily holds various data generated within the controller 30 .

The input reading circuit 2e reads the operation amount (movement amount) of each of the main input operation section 20M and the secondary input operation section 20S, and transmits them to the control amount calculation section 2a. Based on the input from the input reading circuit 2e, the control amount calculation section 2a calculates the control amount of each of the main operation movable section 1M and the secondary operation movable section 1S. Then, the control amount calculation section 2a sends the control amount of the main operation movable section 1M, which is the calculation result, to the M drive circuit 2b, and sends the control amount of the secondary operation movable section 1S to the S drive circuit 2c. The M drive circuit 2b and the S drive circuit 2c respectively control the driving of the main operation movable section 1M and the sub-operation movable section 1S with the control amount obtained from the control amount calculation section 2a. The main operation movable section 1M and the M driving circuit 2b constitute main driving means, and the sub-operation movable section 1S and the S driving circuit 2c constitute subordinate driving means.

In the first embodiment, the main operation movable section 1M and the slave operation movable section 1S are each provided with a drive circuit. It may be one circuit as long as it can be driven.

The main input operation section 20M and the secondary input operation section 20S are members for inputting the movement direction and the amount of movement for the user to move the main operation section 10M and the secondary operation section 10S, respectively. It takes the form of For example, by displacing the upper end of the main input operation section 20M in the front, rear, left, and right directions, the amount of movement of the main operation section 10M in the XY direction can be input, and by rotating the upper end, the amount of movement in the Z direction of the main operation section 10M can be input. can be input, and the same applies to the secondary input operation section 20S.

This operation example assumes a structure whose lower end is supported and whose upper end is operated. It may be a structure suspended from Further, the main input operation section 20M and the secondary input operation section 20S may be composed of buttons, dials, touch panels, and the like. Further, the PC 40 or the PC operation unit 403 may be used as a substitute for all or part of the functions of the controller 30, the main input operation unit 20M and the secondary input operation unit 20S. For example, the image processing performed by the image processing circuit 2d may be performed by an application of the PC 40 and displayed on the monitor 200, and the PC 40 may determine whether the object 103 is in focus or out of focus. Alternatively, the processing executed by the control amount calculation unit 2a may be executed by an application of the PC 40, and the execution result may be sent to the M drive circuit 2b and the S drive circuit 2c.

The PC 40 has a CPU 400, a primary storage device 401 and a secondary storage device 402, as shown in FIG. 2(b). The PC 40 is communicably connected to the controller 30 , the monitor 200 and the PC operation section 403 . The PC 40 is an arithmetic device that operates in cooperation with an auxiliary device. Primary storage 401 includes RAM. The CPU 400 expands the program read from the secondary storage device 402 into the RAM. In addition, the RAM temporarily holds the calculation results and the like by the CPU 400 . The secondary storage device 402 is, for example, a hard disk drive, and stores programs executable by the CPU 400 and the like.

The monitor 200 is a display device for the PC 40, and displays applications executed by the PC 40, characters for interactive operations, and the like. Also, an observation image of the object 103 can be displayed on the monitor 200 . A PC operation unit 403 is a keyboard, a mouse, a touch panel, and the like, and receives user operations on the PC 40 .

The PC 40 receives data from the controller 30 , performs image processing, data analysis, determination processing, etc., and returns the results to the controller 30 . In the first embodiment, the controller 30 receives the image data output from the camera 106 and the operation amounts of the main input operation unit 20M and the secondary input operation unit 20S. . Also, the functions of the PC 40 may be executed by the control unit 2 and the memory 2f, or conversely, the controller 30 may have a CPU (having the functions of the PC 40).

With the above configuration, the user can operate the main input operation section 20M and the secondary input operation section 20S while looking through the eyepiece 104b (or while viewing the display on the monitor 200) to operate the main operation section 10M and the secondary operation section 10S. can be moved to the desired position. By executing a predetermined program to automatically drive the main operation movable section 1M and the sub-operation movable section 1S, the main operation section 10M and the sub-operation section 10S can be three-dimensionally moved to predetermined positions. Automatic operation is also possible.

FIG. 3(a) is a diagram showing the structure of the main operation unit 10M. FIG.3(b) is an enlarged view of the A part (tip part) in Fig.3 (a). The main operation portion 10M has a main operation pipette 11M, an operation pipette holding portion 12 (hereinafter referred to as “holding portion 12”) holding the main operation pipette 11M, and a shaft portion 13 connected to the main operation movable portion 1M. . By moving the shaft portion 13, the main operation movable portion 1M moves the entire main operation portion 10M.

FIG. 4(a) is a diagram showing the structure of the slave operation section 10S. FIG.4(b) is an enlarged view of the B section (tip part) in Fig.4 (a). The slave operation section 10S has a slave operation pipette 11S, a holding section 12 that holds the slave operation pipette 11S, and a shaft section 13 connected to the slave operation movable section 1S. By moving the shaft portion 13, the slave operation movable portion 1S moves the entire slave operation portion 10S.

In the first embodiment, the holding portion 12 and the shaft portion 13 of the main operating portion 10M and the secondary operating portion 10S are the same. and the shaft portion 13 may have a different configuration.

The master pipette 11M and the slave pipette 11S are each made of a glass tube. The main operation pipette 11M has an outer diameter constricted to 'DM' at the tip and is bent at the diameter DM. Similarly, the slave pipette 11S has an outer diameter narrowed to 'DS' at the tip and is bent at the outer diameter DS. In the first embodiment, the diameter DM is assumed to be smaller than the diameter DS (DM<DS).

The tip portions of the master operation pipette 11M and the slave operation pipette 11S are bent when the master operation portion 10M and the slave operation portion 10S are held obliquely with respect to the horizontal plane as shown in FIG. 1(a). Secondly, the bent tip portions 11Mt and 11St are to be substantially parallel to the horizontal plane. When the outer diameters of the tips of the main operation section 10M and the secondary operation section 10S in the direction of the optical axis of the microscope 100 are defined as dM and dS, respectively, when the curved tip sections 11Mt and 11St are substantially parallel to the horizontal plane, dM≈DM, dS≈DS. Depending on the application, the master operation pipette 11M and the slave operation pipette 11S may be used without bent tips.

Since the master pipette 11M and the slave pipette 11S are each manufactured by processing a glass tube as described above, the outer diameters dM and dS are negligible with respect to the depth of field of the microscope 100 for each individual. There are manufacturing errors that cannot be corrected. Here, the depth of field is defined as the distance in the Z direction determined by the image processing circuit 2d of the controller 30 to be in focus. The depth of field has widths in the positive direction and the negative direction around the focal position of the objective lens 104a, and varies depending on the type of the objective lens 104a.

In addition to the manufacturing error between individual pipettes with outer diameters dM and dS, when the main operating pipette 11M and the slave operating pipette 11S are attached to the respective holders 12, an attachment error to the holders 12 occurs. Therefore, the tip positions of the master operation pipette 11M and the slave operation pipette 11S with respect to the shaft portion 13 often differ each time the master operation pipette 11M and the slave operation pipette 11S are replaced. In the first embodiment, as will be described later, it is possible to reduce the effects of such manufacturing errors and inter-individual errors when the main operation section 10M and the secondary operation section 10S are moved.

FIG. 5 is a diagram showing an arrangement example of the main operation section 10M (main operation pipette 11M) and the secondary operation section 10S (secondary operation pipette 11S) before starting work on the object 103. FIG. FIG. 5(a) is a diagram for explaining an imaging region C captured by the camera 106. FIG. As described above, in the microscope system, the observation optical system 104 splits the light so that the camera 106 can take an image. Therefore, the imaging area C can be the same as the range (field of view of the microscope 100) in which the user can observe the object 103 and the like through the eyepiece 104b, depending on the sensor of the camera 106 and the magnification. FIG. 5(b) is a view (front view) of the object 103 viewed from the user.

An object 103 is placed on the work surface 102f. In the first embodiment, the object 103 is assumed to be a cell, and the work surface 102f is assumed to be the surface of the petri dish 102. FIG. When the object 103 is a cell, the surroundings of the object 103 are usually filled with a medium. At this time, by using a medium having a density lower than that of the object 103, the object 103 can be submerged in the medium and brought into contact with the working surface 102f. In FIG. 5, illustration of the medium is omitted.

In FIG. 5B, the focus area F is simply and schematically shown by a chain double-dashed line. The Z-direction width of the focus area F schematically represents the depth of field. Note that the depth of field of the high-magnification objective lens 104a may be on the order of several microns to submicrons, which is much smaller than that of the object 103, the master operation section 10M, and the secondary operation section 10S. In FIG. 5B, the focus area F is drawn with a certain width in the Z direction for convenience of explanation, and does not reflect the actual size.

In addition, in FIG. 5B, for the sake of convenience, the bent tip portions 11Mt and 11St are drawn parallel to the working surface 102f, but in reality the bent tip portions 11Mt and 11St are inclined with respect to the working surface 102f. state may occur. In that case, the master operation section 10M and the secondary operation section 10S are arranged so that the tips of the bent tip portions 11Mt and 11St are closer to the working surface 102f than the roots of the bent tip portions 11Mt and 11St. It should be noted that, as with FIG. 5(b), the drawings referred to hereinafter, which are front views showing the positional relationship between the main operation section 10M and the sub-operation section 10S, are similar to FIG. 5(b). The portions 11Mt and 11St are drawn parallel to the work surface 102f.

'hM' is the distance from the work surface 102f at the center of the tip of the master operation part 10M, 'hS' is the distance from the work surface 102f at the center of the tip of the secondary operation part 10S, and the distance in the Z direction from the work surface 102f of the object 103 is Let 'hC' be the maximum distance (height in the Z direction). The "center of the tip of the master operation part 10M" is the center of the tip of the curved tip part 11Mt of the master operation pipette 11M, and the "center of the tip of the secondary operation part 10S" is the center of the bent tip part 11St of the slave operation pipette 11S. It is the center of the tip, and the same expression is used in the following description.

When the object 103 is a cell, before starting work on the object 103, as shown in FIG. Generally, an operation unit 10M and a secondary operation unit 10S are arranged. When the object 103 is a cell, the Z-direction height hC is about several μm to 100 μm. Each must be controlled on the order of 10 μm or less.

Next, with reference to FIGS. 6 to 14, the main operation section 10M and the secondary operation section 10S (the bent tip portions 11Mt and 11St) are arranged at predetermined positions with respect to the object 103 before starting work on the object 103. A process for performing the preparation (hereinafter referred to as a "preparation process") will be described.

6A and 6B are schematic diagrams for explaining the first step of the preparatory steps in the first embodiment, FIG. 2 is a front view of the object 103 viewed from the user; FIG.

In the first step, the main operation unit 10M and the secondary operation unit 10S are focused within the imaging region C (within the field of view of the microscope 100). Let 'hM1' be the distance hM of the main operation portion 10M (the bent tip portion 11Mt), and let 'hS1' be the distance hS of the sub-operation portion 10S (the bent tip portion 11St). In the first step, the main operation section 10M and the secondary operation section 10S are arranged at approximately the same distance (hM1≈hS1) from the working surface 102f in the Z direction. Since the objective lens 104a of the microscope 100 generally has a shallow depth of field, the difference between the distances hM1 and hS1 can be reduced to several micrometers or less by using an objective lens 104a with a magnification of several tens of times.

In the Y direction, the main operation section 10M and the secondary operation section 10S are separated from the center of the imaging area C by distances yM1 and yS1, respectively. However, in the first embodiment, yM1=yS1=0 in the first step, and this state is shown in FIG. 6(a). In the X direction, the main operation section 10M and the secondary operation section 10S are separated from the center of the imaging region C by distances xS1 and xM1, respectively.

The input reading circuit 2e acquires X position information QM, Y position information RM and Z position information PM of the main operation movable section 1M from an encoder (not shown) provided on the main operation movable section 1M. Further, the input reading circuit 2e acquires X position information QS, Y position information RS and Z position information PS of the slave operation movable part 1S from an encoder (not shown) provided in the slave operation movable part 1S. The position information of the main operation movable portion 1M in the first step is assumed to be X position information QM1, Y position information RM1 and Z position information PM1. Further, the position information of the slave operation movable part 1S in the first step is assumed to be X position information QS1, Y position information RS1 and Z position information PS1.

It should be noted that when the main operation movable portion 1M and the sub-operation movable portion 1S are driven by stepping motors or the like by open-loop control, it is not necessary to obtain the position information by the encoder. However, in the first embodiment, as described above, the positional information of the main operation movable section 1M and the sub-operation movable section 1S in the X, Y and Z directions is obtained from the encoders.

In the second step that follows the first step, the main operation portion 10M (tip bending portion 11Mt) is brought into contact with or in the vicinity of the work surface 102f while the focus is on the work surface 102f or its vicinity. move to FIG. 7 is a schematic diagram for explaining the second step of the preparatory step. FIG. 7(a) is a front view showing a state in the middle of the second step, and FIG. 7(b) is a front view showing a state after the second step is completed.

As described above, the focal position with respect to the object 103 can be adjusted by the user operating the knob 105 provided on the microscope 100 to move the objective lens 104a in the Z direction. When the user moves the objective lens 104a to move the focus area F toward the work surface 102f, the controller 30 causes the main operation unit 10M to move following the focus area F. It controls the main operation movable part 1M. The operation of causing the main operation unit 10M to follow the focus area F is performed by calculating the contrast value of the main operation unit 10M in the imaging area C and controlling the position of the main operation unit 10M so that the contrast value becomes high. can be done. Since such a technique is publicly known, detailed description is omitted.

Instead of using the contrast value, for example, the main operation unit 10M may be moved based on the results of distance measurement by phase difference distance measurement means provided in the imaging sensor of the camera 106. FIG. Also, the secondary operation section 10S can be moved to follow the main operation section 10M, and a method for doing so will be described later in a second embodiment.

When the second step shown in FIG. 7B is completed, the tip of the main operation portion 10M is in contact with the working surface 102f. For example, while the main operation unit 10M is being moved toward the work surface 102f, the main operation unit 10M is vibrated in the Y direction at a low frequency, and the movement of the main operation unit 10M is observed by the camera 106. When the tip of the main operation unit 10M contacts the work surface 102f, the vibration (motion) of the main operation unit 10M changes. It can be determined that contact has occurred. When the tip of the main operation portion 10M is in contact with the work surface 102f and the distance between the center of the tip of the main operation portion 10M and the work surface 102f is 'hM2', hM2=dM/2.

In the first embodiment, the tip of the main operation portion 10M is brought into contact with the work surface 102f. not. In this case, the movement of the main operation unit 10M is controlled so that the image of the main operation unit 10M has a predetermined contrast value at a position in the +Z direction from the focus area F while the work surface 102f is in focus, A method or the like of estimating the distance hM2 from a predetermined contrast value is used. At that time, the means for estimating the distance hM2 is not limited. Let 'PM2' be the Z position information of the main operation unit 10M indicated by the encoder of the main operation movable unit 1M when the second step is completed.

FIG. 8 is a front view showing a state in which the main operation section 10M is moved to an arbitrary Z-direction position after the second step is completed. The distance between the parts in the three-dimensional space can be calculated using the position information of each encoder of the main operation movable section 1M and the sub-operation movable section 1S.

As shown in FIG. 8, the height hM of the center of the tip of the main operation portion 10M from the working surface 102f is expressed by the following equation 1. Here, as described above, since the following formula 2 is established from FIG. 7B, the following formula 1 can be rewritten as the following formula 3. Also, the distance hS in the Z direction from the work surface 102f at the center of the distal end of the secondary operation section 10S is expressed by Equation 4 below. Equation 5 below is obtained from Equations 4 and 2 below, and the distance hM−hS in the Z direction of the main operation portion 10M to the slave operation portion 10S is expressed by Equations 3 and 5 below to Equation 6 below.

Thus, in the second step, the Z-direction distance hS from the work surface 102f at the center of the distal end of the secondary operation section 10S and the Z-direction distance hM−hS between the main operation section 10M and the Z-direction information PM1, It can be represented by PM2, PM, PS1, PS and the outer diameter dM, which is a constant. As a result, by executing the first step and the second step without bringing the slave operation portion 10S into contact with the work surface 102f, the Z-direction distance of the slave operation portion 10S from the work surface 102f and the distance of the slave operation portion 10S from the work surface 102f to the main operation unit 10M in the Z direction.

In the first embodiment, the auxiliary device has one main operation section 10M and one secondary operation section 10S, but may have a plurality of main operation sections 10M and secondary operation sections 10S. In that case, the first step and the second step can be performed as described above by acquiring position information from the encoder of each operation movable portion for each operation portion.

FIG. 9 is a plan view showing an example of the positional relationship on the XY plane between the distal end portion of the main operating portion 10M and the distal end portion of the secondary operating portion 10S after the completion of the second step. As described with reference to FIG. 7, in the first embodiment, the secondary operation part 10S is not moved in the second step. position.

The distance xM in the X direction from the image center O of the tip center of the main operation section 10M is expressed by the following formula 7 using the X position information QM obtained by the encoder provided on the main operation movable section 1M. Similarly, the distance yM in the Y direction from the center of the tip of the main operation section 10M to the center of the image is expressed by the following equation 8 using the Y position information RM obtained by the encoder provided on the main operation movable section 1M. In the first embodiment, the distance yM1=0 in the following equation 8, but the values of the distances xM1 and yM1 are determined using a known technique (detailed description is omitted) to determine the sensor size of the camera 106. and the magnification of the objective lens 104a.

Further, the distance xS in the X direction from the center of the tip of the slave operation section 10S to the center of the image is expressed by the following equation 9 using the X position information QS obtained by the encoder provided in the slave operation movable section 1S. Similarly, the distance yS in the Y direction from the center of the tip of the slave operation section 10S to the center of the image is expressed by the following formula 10 using the Y position information RS obtained by the encoder provided in the slave operation movable section 1S. . In the first embodiment, the distance yS1=0 in Equation 10 below. The magnification of the objective lens 104a can be easily calculated.

In this way, X position information QM1, QS1 and Y position information RM1, RS1 in the first step and X position information QM, QS and Y position in the second step of the master operation unit 10M and the slave operation unit 10S Acquire information RM and RS. As a result, the distances xM and xS in the X direction and the distances yM and yS in the Y direction from the image center O of the centers of the tips of the main operating section 10M and the secondary operating section 10S can be obtained.

FIG. 10 is a front view for explaining the third step of the preparatory steps. In the third step, at least the Z-direction position of the tip center of the secondary operation portion 10S is moved to the Z-direction position of the tip center of the main operation portion 10M or a position in the vicinity thereof. In the state shown in FIG. 10, the object 103 is in contact with the work surface 102f, and the vicinity of the center of the object 103 is in focus. The tip of the main operation section 10M (tip bending section 11Mt) is arranged in the focus area F (at approximately the same height as the center of the object 103). In FIG. 10, the Z-direction position of the tip center of the slave operation portion 10S (tip bending portion 11St) before the third step is shown by the solid line after the first step (the position at the distance hS1 from the work surface 102f). ), but may be placed at another position.

Here, 'hM-hS', which is the amount of movement required to move the Z-direction position of the tip center of the secondary operation section 10S to the same position as the Z-direction position of the tip center of the main operation section 10M, is expressed by Equation 6 above. , it is represented by the following formula 11 by setting PS=PS1.

Therefore, if the controller 30 controls the slave operation movable section 1S to move in the -Z direction by 'PM1-PM', the Z-direction position of the tip center of the slave operation section 10S will be the tip of the main operation section 10M. It can be moved to the same position as the center Z-direction position or to a position in the vicinity thereof.

Conventionally, when the center of the tip of the slave operation unit 10S is moved to the focus area of the microscope 100, it is slightly moved in the Z direction, so it takes a long time to complete the movement. On the other hand, in the first embodiment, by performing the first step, the second step, and the third step in this order as described above, the secondary operation unit 10S is moved from the main operation unit 10M within the focus area F. It is possible to shorten the time for moving to a Z-direction position substantially the same as the center of the tip. Also, in the first embodiment, cameras for capturing images of the main operation section 10M and the secondary operation section 10S from the X direction and the Y direction are not required, and therefore the problem of cost increase does not occur. Furthermore, since the distance hS from the work surface 102f to the center of the tip of the slave operation unit 10S can be obtained, the time required to move only the slave operation unit 10S to an arbitrary position near the work surface 102f can be shortened. becomes possible.

Next, with reference to FIGS. 10 to 14, the manner in which the slave operation section 10S is driven in the third step will be described in detail. In FIGS. 10 to 14, for convenience of explanation, the diameter dS of the secondary operation portion 10S (the tip bent portion 11St) is drawn large.

In the third step, the controller 30 has a function of operating the sub-operation movable section 1S so that the sub-operation section 10S is automatically arranged at a predetermined position according to the positions of the main operation section 10M and/or the target object 103. Define as "placement function". As described with reference to FIG. 10, the first driving mode of the placement function is to place the secondary operation section 10S with respect to the main operation section 10M such that hM≈hS≈hC/2. It is a mode. Next, the second and third driving modes will be described.

11(a) and 11(b) are a plan view and a front view for explaining the second driving mode of the placement function. In the second driving mode of the placement function, when the object 103 is outside the imaging region C, the secondary operation unit 10S is moved to a position where it does not come into contact with the object 103. FIG. Note that the position of the secondary operation portion 10S indicated by the dashed line in FIG. 11B is such that the distance from the work surface 102f to the center of the tip of the secondary operation portion 10S in the Z direction is the distance from the work surface 102f to the center of the tip of the main operation portion 10M. This position is approximately the same as the distance. In FIG. 11B, the solid line indicates the position of the secondary operation unit 10S after execution of the second driving mode of the arrangement function.

As indicated by the dashed line in FIG. 11(a), the object 103 is outside the imaging region C, and the distance from the working surface 102f to the center of the tip of the main operation unit 10M is as shown in FIG. 11(b). It may be smaller than the Z-direction height of the object 103 . In this case, if the center of the tip of the secondary operation section 10S is aligned with the center of the tip of the main operation section 10M in the Z direction, the secondary operation section 10S may come into contact with the object 103. FIG. In order to avoid this problem, when the target object 103 is outside the imaging region C, the Z-direction position of the tip center of the secondary operation unit 10S is adjusted so that the secondary operation unit 10S does not contact the target object 103. It is positioned at a distance H1 from the working surface 102f.

The distance H1 must satisfy the condition 'H1>hC+(dS/2)'. As described with reference to FIG. 5, 'hC' is the Z-direction height of the object 103 from the work surface 102f, and 'dS' is the follower in the optical axis direction (Z-direction) of the microscope 100. It is the outer diameter of the tip of the operation part 10S.

Note that there are an infinite number of positions where the relationship 'H1>hC+(dS/2)' is satisfied. Therefore, using the constant α, the slave operation unit 10S is moved to a position where the relationship 'H1=α+hC+(dS/2)' is satisfied. Here, the constant α may be set to a value close to zero (0) when the slave operation unit 10S is to be brought closer to the target object 103 . Also, in FIG. 11, an example in which the object 103 is outside the imaging region C is taken up, but the second driving mode of the placement function is that the object 103 is inside the imaging region C and is viewed from the Z direction. It can also be applied when the object 103 and the slave operation unit 10S overlap each other.

12(a) and 12(b) are a plan view and a front view for explaining the second driving mode of the placement function. In the second driving mode of the placement function, when the center of the tip of the main operation part 10M is near the work surface 102f, the secondary operation part 10S is moved to a position out of contact with the work surface 102f. FIG. 13 is a reference when the center of the tip of the main operation section 10M is in the vicinity of the working surface 102f and the center of the tip of the secondary operation section 10S is aligned with the center of the tip of the main operation section 10M in the Z direction. It is a front view which shows an example.

As shown in FIG. 13, when the distance hM from the working surface 102f to the center of the tip of the main operation portion 10M is smaller than 1/2 of the outer diameter dS of the tip of the secondary operation portion 10S (hM<dS/2). There is In this case, if the secondary operation section 10S is moved from the solid line position to the dashed line position so that the center of the tip of the secondary operation section 10S is aligned with the center of the tip of the main operation section 10M in the Z direction, the secondary operation The portion 10S comes into contact with the working surface 102f.

Therefore, as shown in FIG. 12B, in the second driving mode of the arrangement function, the distance H2 from the working surface 102f to the center of the tip of the sub-operation unit 10S is greater than dS/2 (H2>dS/ 2) Move the slave operation unit 10S. As a result, contact with the work surface 102f of the sub-operation unit 10S can be avoided.

Note that there are an infinite number of positions where the relationship 'H2>dS/2' is satisfied. Therefore, using the constant β, the slave operation unit 10S is moved to a position where 'H2=β+(dS/2)'. The constant β may be set to a value close to zero (0) when the slave operation unit 10S is desired to be close to the work surface 102f.

Next, the preparation process described above will be described with reference to a flow chart. FIG. 14 is a flow chart explaining the preparation process. Each processing (step) indicated by S number in FIG. This is realized by controlling the driving of the unit 1S. However, the process of S104 is excluded.

In S101, the CPU 400 instructs the controller 30 to move the main operation section 10M and the secondary operation section 10S in the direction of the focus area F. FIG. The controller 30 drives the main operation movable section 1M and the secondary operation movable section 1S in accordance with instructions from the CPU 400, thereby moving the main operation section 10M and the secondary operation section 10S toward the focus area F. FIG. Although the main operation section 10M and the secondary operation section 10S are moved by instructions from the CPU 400 here, the user may operate the main input operation section 20M and the secondary input operation section 20S to move them.

In S102, the CPU 400 determines whether or not both the main operation section 10M and the secondary operation section 10S are in focus in the focus area F by the image processing circuit 2d. The determination in S102 can be made by analyzing the image obtained from the camera 106 in a known direction, and detailed description is omitted. If the CPU 400 determines that it is in focus (Yes in S102), it advances the process to S103, and if it determines that it is out of focus (No in S102), it returns the process to S101. In S103, the CPU 400 transmits positional information in the X, Y, and Z directions of the main operating section 10M and the secondary operating section 10S detected by the encoders provided on the main operating section 1M and the secondary operating section 1S to the controller. 30 and stored in the primary storage device 401 .

In S104, the user focuses on or near the work surface 102f. When the user determines that the desired position has been focused and inputs a signal indicating that the process of S104 has ended from the PC operation unit 403 to the PC 40, the CPU 400 advances the process to S105. The CPU 400 may analyze an image obtained from the camera 106 to determine whether or not the working surface 102f or its vicinity is in focus. S101 to S104 are the processing in the first step described above.

In S105, the CPU 400 instructs the controller 30 to move the main operation unit 10M toward the work surface 102f. By the controller 30 driving the main operation movable portion 1M in accordance with an instruction from the CPU 400, the main operation portion 10M moves toward the work surface 102f. Here, the CPU 400 instructs the controller 30 to move the main operation section 10M, but instead of this, the controller 30 drives the main operation movable section 1M according to the operation input of the main input operation section 20M by the user. Therefore, the process of S105 may be performed.

In S106, the CPU 400 determines whether or not the main operation unit 10M has come into contact with the work surface 102f or moved to the vicinity thereof. If the CPU 400 determines that the movement of the main operation unit 10M has been completed (Yes in S106), the CPU 400 advances the process to S107. is returned to S105. In S107, the CPU 400 acquires, via the controller 30, positional information in the X, Y, and Z directions of the main operating section 10M detected by the encoder provided on the main operating movable section 1M, and stores it in the primary storage device 401. do. At this point, the secondary operation unit 10S has not moved from the position after the processing in S103, so there is no need to acquire the position information of the secondary operation unit 10S. S105 to S107 are the processing in the second step described above.

In S108, the CPU 400 focuses the microscope 100 on the working position. Note that the processing of S108 may be performed by the user. In S109, the CPU 400 determines whether or not the object 103 is present in the imaging region C. The determination in S109 can be made by image analysis of the imaging area C. FIG. When the CPU 400 determines that the object 103 is not present in the imaging region C (No in S109), the process proceeds to S110. In S110, the CPU 400 sends an instruction to the controller 30 to move the slave operation unit 10S to a position where 'hS>hC+(dS/2)'. Note that, as already explained, 'dS' is the outer diameter of the tip of the secondary operation section 10S, and 'hC' is the height of the object 103 in the Z direction. The controller 30 receives instructions from the CPU 400 and operates the slave operation movable section 1S. This process ends when the process of S110 ends.

When the CPU 400 determines in S109 that the object 103 is present in the imaging region C (Yes in S109), the process proceeds to S111. In S111, the CPU 400 determines whether or not the relationship 'hM>dS/2' holds true. As already explained, 'dS' is the outer diameter of the tip of the secondary operation portion 10S, and 'hM' is the distance from the working surface 102f to the center of the tip of the main operation portion 10M. If the CPU 400 determines that the relationship 'hM>dS/2' is established (Yes in S111), the process proceeds to S112.

In S112, the CPU 400 sends to the controller 30 an instruction to move the slave operation unit 10S to a position where 'hS=hM'. As already explained, 'hS' is the distance from the working surface 102f to the center of the tip of the sub-operation section 10S. The controller 30 receives instructions from the CPU 400 and operates the slave operation movable section 1S. This process ends when the process of S112 ends.

When the CPU 400 determines in S111 that the relationship 'hM>dS/2' does not hold (No in S111), the process proceeds to S113. In S113, the CPU 400 sends to the controller 30 an instruction to move the slave operation unit 10S to a position where 'hS>dS/2'. The controller 30 receives instructions from the CPU 400 and operates the slave operation movable section 1S. This process ends when the process of S113 ends. S109 to S113 are the processes in the third step described above.

In the first embodiment, the determination processing is executed by the CPU 400, but the determination processing may be executed by providing a CPU in the controller 30, or by controlling the amount calculation unit 2a or the image processing circuit 2d. It may be executed with a judgment function.

<Second embodiment>
In the second embodiment, another method of moving the main operating section 10M and the secondary operating section 10S in the second step and the third step in the preparatory steps described above will be described.

FIG. 15 is a front view showing a state in the middle of moving the main operation unit 10M toward the work surface 102f in the second step. In the first embodiment, the secondary operation section 10S is not moved in the Z direction after the first step. In contrast, in the second embodiment, the controller 30 controls the main operation section 10M and the slave operation section 10M such that the Z-direction position of the tip center of the secondary operation section 10S follows the Z-direction position of the tip center of the main operation section 10M. Move the operation unit 10S.

FIG. 16(a) is a front view showing a state in which the tip of the main operation portion 10M is in contact with the work surface 102f in the second step (the state in which the second step is completed). FIG. 16(b) is an enlarged view of region D in FIG. 16(a). In FIGS. 16A and 16B, in order to set a follow-up distance L, which will be described later, for the purpose of avoiding contact of the slave operating unit 10S with the target object 103, the target object 103 is indicated by a dashed line directly below the slave operating unit 10S. showing.

Let 'hS2' be the distance hS in the Z direction from the work surface 102f to the center of the tip of the slave operation part 10S, and let 'PS2' be the Z position information PS of the slave operation part 10S detected by the encoder of the slave operation movable part 1S. . The secondary operation section 10S moves while maintaining a follow-up distance L with respect to the main operation section 10M. At that time, it is desirable to avoid contact of the slave operation unit 10S with the target object 103 . Therefore, the follow-up gap δ is included in the follow-up distance L so that the slave operation unit 10S does not contact the target object 103 regardless of the Z-direction position of the main operation unit 10M.

The follow-up distance L is the distance from the Z-direction position of the center of the tip of the main operation section 10M to the lower end of the outer periphery of the secondary operation section 10S. Any value larger than zero (0) can be set for the following gap δ. It is desirable to take into account variations in size.

As shown in FIG. 16(b), the follow-up distance L is expressed by Equation 12 below using hC, dS, and dM. As already described, 'hC' is the maximum distance in the optical axis direction (Z direction) from the working surface 102f of the object 103. FIG. 'dM' is the outer diameter of the tip of the main operating portion 10M, and 'dS' is the outer diameter of the tip of the secondary operating portion 10S. The controller 30 moves the slave operation part 10S while maintaining the follow-up distance L when executing the second step. Accordingly, it is possible to prevent the secondary operation unit 10S from coming into contact with the target object 103 during execution of the second step.

FIG. 17 is a front view showing a state during execution of the third step. When the third step is performed, the center of the tip of the secondary operation section 10S and the center of the tip of the main operation section 10M are separated by the follow-up distance L in the Z direction. Therefore, if the secondary operation section 10S is moved toward the work surface 102f by the follow-up distance L in the Z direction, the position of the tip center of the secondary operation section 10S and the position of the tip center of the main operation section 10M in the Z direction will match. , a main operation section 10M and a sub-operation section 10S can be arranged.

In FIG. 17, when the distance hS from the work surface 102f to the center of the tip of the slave operation portion 10S is smaller than the distance hS1 (see FIG. 6) (hS<S1), the distance to move the slave operation portion 10S is set to It can be shorter than the first embodiment. Therefore, it is possible to shorten the time required for the third step. In order to minimize the time required for the third step while preventing the slave operation unit 10S from contacting the target object 103 in the second step, the following gap δ included in the following distance L should be set to zero (0 ). On the other hand, when 'hS<hS1' in FIG. This is advantageous in that it is possible to prevent the contact of the sub-operation unit 10S.

Furthermore, by using the method of moving the slave operation unit 10S in the second embodiment, it is possible to use the method of moving the slave operation unit 10S minutely while moving the focusing area F minutely to repeat focusing by moving the slave operation unit 10S minutely. Also, it is possible to reduce the time required to place the secondary operation unit 10S at a predetermined position. Note that the microscope system may be configured so that the user can select between the first embodiment and the second embodiment as appropriate. Selection (switching) between the first embodiment and the second embodiment can be performed by operating (inputting) to the PC 40 .

<Third Embodiment>
After performing the first and second steps described above, if there is a step of working only with the operating means, the slave operating means may be out of the field of view. 18A and 18B are schematic diagrams showing a method of driving the slave operation unit 10S according to the third embodiment. 18A is a plan view showing the arrangement of the main operation section 10M and the secondary operation section 10S before the first step, showing a state where the secondary operation section 10S is not within the field of view of the microscope 100. FIG. In this state, the user cannot visually recognize the secondary operation unit 10S through the eyepiece 104b, nor can the user visually recognize the secondary operation unit 10S through the image captured by the camera 106. FIG.

In the state of FIG. 18A, the first step described in the first embodiment can be performed by observing the field of view of the microscope 100 through the eyepiece 104b and by analyzing the photographed image obtained by the camera 106. cannot be executed. Therefore, as shown in FIG. 18B, first, the slave operation section 10S is moved within the XY plane so that at least the tip of the slave operation section 10S is within the imaging area C. Then, as shown in FIG. When at least the tip of the secondary operation section 10S enters the imaging region C, a change appears in the captured image or the field of view. At that time, if the Z-direction positions of the secondary operation section 10S and the main operation section 10M are aligned using the position information of the secondary operation section 10S acquired from the encoder provided in the secondary operation movable section 1S, the secondary operation section 10S It becomes easy to confirm whether or not the tip of has entered the imaging region C. Thus, when the tip of the slave operation section 10S enters the imaging region C, the first step can be performed.

Although the target object 103 is shown in FIG. 18A, it does not matter here whether the user can visually recognize the target object 103 or not. FIG. 18B shows a state in which the slave operation unit 10S is moved to a position that does not overlap the object 103 when viewed in the Z direction.

Although the present invention has been described in detail based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms without departing from the gist of the present invention can be applied to the present invention. included. Furthermore, each embodiment described above merely shows one embodiment of the present invention, and it is also possible to combine each embodiment as appropriate.

The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

1M Main operation movable section 1S Sub-operation movable section 2 Control section 2a Control amount calculation section 2b M drive circuit 2c S drive circuit 2d Image processing circuit 10M Main operation section 10S Sub-operation section 30 Controller 40 PC
100 microscope 106 camera 400 CPU

Claims (10)

driving means for driving at least one main operating section and at least one secondary operating section for manipulating an object under the field of view of the microscope;
The first step, the second step and the third step are performed by the analysis means for analyzing the image obtained by imaging the observation range with the microscope by controlling the driving of the driving means using the analysis result of the image. in that order;
The first step is a step of bringing the main operation section and the secondary operation section into a state in which the focus region of the microscope is focused in the observation range,
In the second step, with the microscope focused on or near a work surface on which the object is located, the main operation unit is positioned in contact with or near the work surface. A step of moving the part in the optical axis direction of the microscope,
The microscope assisting apparatus, wherein the third step is a step of moving the secondary operation section to a predetermined position corresponding to the position of the main operation section or the object in the direction of the optical axis. 2. A microscope auxiliary apparatus according to claim 1, wherein, in said third step, said control means moves said secondary operation section to a position in said optical axis direction of said main operation section or a position in the vicinity thereof. . In the third step, the control means controls the secondary operation so that the position of the center of the tip of the main operation section in the optical axis direction coincides with the position of the center of the tip of the secondary operation section in the optical axis direction. 3. The microscope assisting device according to claim 2, wherein the part is moved. The width of the tip of the secondary operation portion in the optical axis direction is larger than the width of the tip of the main operation portion, and the position of the center of the tip of the secondary operation portion in the optical axis direction is set in the third step. When the tip of the secondary operation part comes into contact with the work surface when the tip of the operation part is aligned with the center of the tip of the operation part, the control means moves the tip of the slave operation part to a position where it does not contact the work surface in the third step. 3. A microscope assisting device according to claim 2, wherein said sub-operating section is moved to a position. In the third step, the control means makes the distance of the secondary operation section from the work surface in the optical axis direction greater than the maximum distance of the object from the work surface in the optical axis direction. 2. A microscope assisting device according to claim 1, wherein said sub-operation unit is moved to a position. When the object is not within the observation range, the control means adjusts the distance of the secondary operation section from the work surface in the optical axis direction to the distance of the object from the work surface in the optical axis direction. 6. A microscope assisting device according to claim 5, wherein said sub-operation unit is moved to a position larger than the maximum distance. When the object and the secondary operation portion are moved in a direction in which the object and the secondary operation portion overlap each other when viewed from the optical axis direction, the control means adjusts the distance of the secondary operation portion from the work surface in the optical axis direction to the object. 6. A microscope assisting apparatus according to claim 5, wherein said sub-operation unit is moved to a position larger than the maximum distance from said work surface in said optical axis direction of said object. In the second step, the control means adjusts the movement of the main operation section so that the center of the tip of the secondary operation section and the center of the tip of the main operation section in the optical axis direction maintain a predetermined follow-up distance. Move the slave operation part,
In the optical axis direction, L is the follow-up distance, hC is the maximum distance of the object from the working surface, dM is the width of the tip of the main operation section, dS is the width of the tip of the secondary operation section, and is less than zero. The follow-up distance is expressed by L=hC+δ+(dS/2)−(dM/2), where δ is an arbitrary large value. microscope auxiliary equipment. A control method for a microscope auxiliary device comprising at least one main operating section and at least one secondary operating section for manipulating an object under the field of view of a microscope, comprising:
a step of analyzing an image obtained by imaging an observation range with the microscope;
a step of driving the main operation unit and the secondary operation unit in order of a first step, a second step, and a third step using the result of analyzing the image;
In the first step, the main operation section and the secondary operation section are brought into focus in the focus area of the microscope in the observation range,
In the second step, with the microscope focused on or near a work surface on which the object is located, the main operation unit is positioned in contact with or near the work surface. moving the unit in the direction of the optical axis of the microscope,
In the third step, the auxiliary operation section is moved in the direction of the optical axis to a predetermined position corresponding to the position of the main operation section or the object. A program that causes a computer to function as the analysis means and the control means of the microscope auxiliary device according to any one of claims 1 to 8.

Images