JP6246551B2 - Controller, microscope system, control method and program - Google Patents

Controller, microscope system, control method and program Download PDF

Info

Publication number
JP6246551B2
JP6246551B2 JP2013221498A JP2013221498A JP6246551B2 JP 6246551 B2 JP6246551 B2 JP 6246551B2 JP 2013221498 A JP2013221498 A JP 2013221498A JP 2013221498 A JP2013221498 A JP 2013221498A JP 6246551 B2 JP6246551 B2 JP 6246551B2
Authority
JP
Japan
Prior art keywords
image
means
sample
microscope
imaging
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2013221498A
Other languages
Japanese (ja)
Other versions
JP2015082097A (en
Inventor
宇範 康
宇範 康
Original Assignee
株式会社キーエンス
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社キーエンス filed Critical 株式会社キーエンス
Priority to JP2013221498A priority Critical patent/JP6246551B2/en
Publication of JP2015082097A publication Critical patent/JP2015082097A/en
Application granted granted Critical
Publication of JP6246551B2 publication Critical patent/JP6246551B2/en
Application status is Active legal-status Critical
Anticipated expiration legal-status Critical

Links

Description

  The present invention relates to time-lapse photography using a microscope.

  It is referred to as time-lapse photography that imaging is performed at a certain imaging interval in order to capture how cells are dividing and growing. At the time of photographing, a microscope system of motorized control and a chamber for controlling the temperature and humidity of cells, the concentration of CO 2 (to maintain a constant pH indicating acidity and alkalinity) and the like may be used. The shooting interval of time-lapse shooting is from several minutes to several hours, and the total shooting period required to shoot a plurality of scheduled shots may extend from several hours to several days.

  During time-lapse photography, a living organism as a sample grows and moves in a container. Therefore, it may be necessary to change the control parameters used for imaging halfway. According to Patent Document 1, it is described that the time-lapse shooting is temporarily interrupted to change the shooting conditions. Specifically, when transitioning to the pause mode, changing the observation position, the illumination light amount, the time-lapse shooting interval, and the number of shots while displaying the image of the sample in real time is described. However, in order to display a real-time image, illumination light must be applied to the sample, and phototoxicity becomes a problem.

  When photographing a cell, it is necessary to irradiate the cell with light, but when performing fluorescence observation in particular, it is necessary to irradiate the cell with strong excitation light. In time-lapse photography, cells are often exposed to light, which may damage the cells. Thus, the adverse effect of light on a specimen is generally called phototoxicity. In the case where ultraviolet light is used, which is said to be toxic to cells, the phototoxicity is further adversely affected and thus requires caution.

  According to Patent Document 2, it is described that a cell is pattern-recognized to obtain a center of gravity of the cell, and tracking an observation visual field (shooting range) according to a shift of the center of gravity. Thus, time-lapse imaging can be performed by tracking cells in a direction (X-axis direction and Y-axis direction) intersecting the optical axis direction (Z-axis direction) of the objective lens.

JP, 2006-220904, A JP, 2013-050666, A

  In the invention of Patent Document 2, since the next observation position can be automatically determined using an image acquired in the past, the problem of phototoxicity can be alleviated. However, since the center of gravity of cells is relied upon for pattern recognition, type-laps imaging may fail because the center of gravity can not be accurately determined. This is because cells are deformed or divided as they grow, so long-term time-lapse photography fails in pattern recognition and can not properly track the position of cells.

  Therefore, an object of the present invention is to increase the success rate of time-lapse imaging while reducing the influence of phototoxicity on a sample.

The present invention is, for example, a control device that controls a microscope used in a time-lapse experiment in which a sample is irradiated with excitation light at regular intervals to photograph the sample ,
A plurality of images obtained by capturing a sample with a microscope and control parameters for capturing each image, the capturing position of the image with respect to the XY stage of the microscope, capturing time, and capturing the image Storage means for correlating and storing control parameters including any of the speculum methods set in the microscope;
Accepting means for accepting designation of a control parameter by an operator for selecting an image during the time-lapse experiment;
Selection means for selecting an image associated with the control parameter received by the reception means from the plurality of images stored in the storage means;
Display means for displaying the image selected by the selection means;
And operating means operable by the operator,
Changing means for changing the imaging position of the sample, which is the control parameter , according to the operation content of the operation means;
Imaging control means for moving the XY stage so that the sample moves to the imaging position after change by the changing means , irradiating the excitation light to the sample, and causing the microscope to execute the next imaging in the time-lapse experiment And.

  According to the present invention, it is possible to increase the success rate of time-lapse imaging while reducing the influence of phototoxicity on a sample.

1 (A) and 1 (B) are perspective views of a microscope. FIG. 2 is a block diagram showing the main parts of the microscope system. FIG. 3 is a view showing an example of the Z stack image. FIG. 4 is a diagram showing an example of an image management method. FIG. 5 is a block diagram showing functions implemented in the control device. FIG. 6 is a diagram showing an example of the operation screen involved in the control parameter change operation. FIG. 7 is a view showing an example of the operation screen involved in the control parameter changing operation. FIG. 8 is a view showing an example of the operation screen involved in the control parameter changing operation. FIG. 9 is a flowchart showing the control parameter changing operation.

  One embodiment of the present invention is shown below. The particular embodiments described below will help to understand various concepts such as the superordinate, intermediate and subordinate concepts of the present invention. Further, the technical scope of the present invention is determined by the scope of the claims, and is not limited by the following individual embodiments.

  FIG. 1A and FIG. 1B are perspective views of a microscope 1 which is the core of the microscope system. As shown in FIG. 1 (A), the microscope 1 uses the housing itself as a dark room, so the user does not have to prepare a dark room. The user (operator) can access the XY stage 6 disposed below the transmission illumination optical system 5 when the upper surface cover 190 is opened. The XY stage 6 is an example of a mounting unit on which a sample is mounted. By closing the top cover 190, a dark room is formed. The user can replace the filter cube (such as a color filter) mounted on the filter turret 14 by opening the front cover 191.

  FIG. 2 is a block diagram showing the main parts of the microscope system 100. As shown in FIG. The microscope 1 is controlled by a controller 2 shown in FIG. The control device 2 is, for example, an information processing device (personal computer: PC) in which a control program is installed. That is, the PC functions as the control device 2 of the microscope 1. Thus, the microscope system 100 includes the microscope 1 and the control device 2.

  The microscope 1 is a microscope capable of acquiring a monochrome image, a color image, and a fluorescence image of the sample 3, but may be a microscope that acquires only one of the images. A container unit 7 for holding the sample 3 is fixed to the XY stage 6. The sample 3 may be called a sample, a sample, a sample or a work. The container unit 7 has a container such as a preparation, a dish or a well and a holder for supporting the container. The illumination light output from the transmission illumination light source 4 is irradiated onto the sample 3 through the transmission illumination optical system 5 including a condenser lens and the like. The transmissive illumination optical system 5 may be provided with a mechanical shutter for light shielding. The illumination light from the transmissive illumination light source 4 is used when acquiring a monochrome image or a color image of the sample 3. When fluorescence observation is performed on the sample 3, excitation light is output from the fluorescence epi-illumination light source 8. The excitation light passes through the fluorescent epi-illumination optics 9 and the excitation filter 10. The fluorescent epi-illumination optical system 9 may be provided with a mechanical shutter for light shielding. The excitation filter 10 is a wavelength selective filter that transmits only the wavelength component to be excitation light among the light output from the fluorescent epi-illumination light source 8. The excitation light is further reflected by the dichroic mirror 11, passes through the objective lens of the objective lens unit 12, and is irradiated onto the sample 3. The dichroic mirror 11 is also a wavelength selective mirror and reflects excitation light, but transmits fluorescence emitted by a fluorescent reagent (also called a fluorescent dye or a fluorescent dye) added to the sample 3. The objective lens unit 12 has an electric revolver rotated by the motor 13 and a plurality of objective lenses mounted on the electric revolver. The filter turret 14 has four apertures, three of which are fitted with different filter cubes with the excitation filter 10, the dichroic mirror 11 and the absorption filter 16, but with one remaining aperture It is not attached. The unmounted aperture of the filter is used in acquiring the bright field image. The filter turret 14 is rotated by the motor 15. The absorption filter 16 is a wavelength selective filter that transmits only the necessary wavelength components of the light from the sample 3. The imaging optical system 17 includes an imaging lens for imaging the sample 3 on the imaging surface of the imaging device 18. The color filter 24 is, for example, a liquid crystal tunable filter, acquires R, G, and B images in order by switching transmission wavelengths, and combines these in the image processing unit 19 to create a color image. The color filter is not limited to the liquid crystal tunable filter described above, and a color filter turret that mechanically switches filters of different transmission wavelengths may be disposed. The image processing unit 19 performs various image processing such as amplifying and A / D converting an image signal output from the photographing device 18 and further performing shading correction. The control unit 20 controls each part of the microscope 1 in accordance with an instruction from the control device 2. For example, the control unit 20 controls the motor group 21 to move the XY stage 6 in the X axis direction or the Y axis direction, or controls the motor 22 to call the objective lens unit 12 sometimes called a Z stage as the Z axis Move in the direction. By moving the objective lens unit 12 in the Z-axis direction, the in-focus position is changed, and auto focusing is performed. The XY stage 6 and the Z stage may have a manual adjustment mechanism. Here, the Z-axis direction is the optical axis direction of the objective lens, and the X-axis direction and the Y-axis direction are directions orthogonal to the Z-axis direction. The communication unit 23 is a unit that receives an instruction from the control device 2 and transmits information and image data from the control unit 20 to the control device 2. The control unit 20 includes, for example, a microprocessor, a CPU, an LSI, an FPGA, an ASIC, and the like. That is, it may be realized by software and its execution means, may be realized only by hardware, or may be realized by a mixture of the former and the latter.

  In the control device 2, the CPU 30 executes a control program stored in the storage device 31 to control the microscope 1 or causes the display unit 34 to display image data received through the communication interface 32. The communication unit 23 and the communication interface 32 may be connected by a general communication protocol such as USB, IEEE 1394, or LAN, or may be connected by a dedicated communication protocol. The storage device 31 includes a memory such as a ROM and a RAM, a hard disk storage device, and the like. The storage device 31 may include a semiconductor storage device such as an SSD, an optical disk, and a magnetic disk. Also, it may be a memory card as used in consumer digital cameras. The operation unit 33 is an example of operation means operable by the operator, and is, for example, an input device such as a keyboard or a pointing device. The display unit 34 provides a UI for setting control parameters for controlling the microscope 1 and a UI for displaying an observation result (still image or moving image).

  The microscope system 100 can perform Z-stack image shooting (Z-stack shooting) and time-lapse shooting. The Z stack is a technology for capturing a plurality of images (Z stack images) by imaging the sample 3 while changing the distance in the optical axis direction (Z axis direction) between the sample 3 and the lens. That is, the Z stack image refers to a plurality of images (layer images / slice images) of the sample 3 acquired at each in-focus position while shifting the in-focus position by moving the Z stage little by little. During observation, the sample 3 such as a microorganism moves not only in the X axis direction and the Y axis direction but also in the Z axis direction. Therefore, by acquiring the plurality of images by shifting the in-focus position little by little in the Z-axis direction, it becomes easy to acquire the image focused on the sample 3. The number of images acquired by one Z stack is referred to as the Z stack number or the Z stack number. That is, the Z stack number Nz indicates the number of images constituting the Z stack image. Time-lapse imaging refers to an imaging method of acquiring one or more images at regular time intervals. Samples 3 such as microorganisms grow and change during observation. Therefore, time-lapse shooting is effective in knowing the process. In the present embodiment, basically, the Z stack and the time-lapse shooting are combined and executed. That is, the microscope 1 executes the Z stack at each imaging timing that arrives at constant time intervals. The number of images acquired in all steps from the start to the end of the time-lapse imaging is referred to as a total imaging number Nt.

[Z stack image]
When acquisition of the Z stack image (that is, execution of the Z stack) is instructed through the operation unit 33, the CPU 30 instructs the control unit 20 of the microscope 1 to execute the Z stack. At this time, the CPU 30 sets control parameters (for example, exposure time, Z stack number Nz, total number of shots Nt, shooting start time, light source type, filter, objective lens magnification, Z) set by the user through the operation unit 33 The movement range W or the like of the stage is also transmitted to the control unit 20. The control unit 20 sets the exposure time and the like in the imaging device 18 in accordance with the control parameters, and controls the motor 13 to move the Z stage (the objective lens unit 12) to the imaging start position. The control unit 20 turns on the light source at the shooting start time to start shooting. The control unit 20 divides the movement range W by the number of photographed Z stacks Nz to obtain the movement pitch P which is one movement distance, and drives the motor 13 to move the Z stage accordingly. That is, the Z stage moves by the movement pitch P each time one image is acquired. The movement and imaging of the Z stage are repeatedly performed until the number of imagings matches the number Nz of the Z stacks. As a result, the specified number of images (Z stack number Nz) are acquired. The control unit 20 creates metadata including control parameters (shooting time, objective lens, Z position of Z stage) when shooting each image, and attaches it to each image. The Z position of the Z stage substantially indicates the distance between the objective lens and the sample 3. Position data indicating the Z position corresponds to distance data indicating the distance. The control unit 20 may transmit the image to the control device 2 through the communication unit 23 each time an image is acquired, or each time acquisition of a plurality of images for the number N of Z stacks is completed, these images may be transmitted. May be sent to the control device 2 collectively. The control unit 20 may assign position data (distance data) indicating the Z position as metadata to each image, but the CPU 30 may create such metadata based on control parameters. Since the control parameters set in the microscope 1 include information on the movement range and movement pitch, the CPU 30 can obtain the Z position of each image by dividing the movement range by the movement pitch. The CPU 30 stores the plurality of layer images received through the communication interface 32 in the storage device 31. As described above, the storage device 31 acquires a plurality of images acquired by photographing the sample while changing the distance between the objective lens and the sample 3 in the microscope 1 and light of the objective lens when each image is acquired. It functions as a storage unit that stores in association with position data indicating a position in the axial direction. The plurality of layer images are Z stack images. The CPU 30 may construct a three-dimensional image from the Z stack image in response to an instruction from the operation unit 33 and display it on the display unit 34.

  FIG. 3 is a view showing an example of the Z stack image. This Z stack image is composed of six layer images. The Z position of the Z stage is written in the metadata of each layer image.

[Image management]
FIG. 4 is a diagram showing an example of an image management method. There are several ways to manage the relationship between each image acquired and the control parameters used when acquiring that image. In the example shown in FIG. 4, the CPU 30 creates a folder in a tree shape for each channel indicating a point (X and Y coordinates), shooting time, and microscopy, and saves an image. As a result, the CPU 30 can select and display an image corresponding to a point, a time, and a microscopic method designated by the user on the display unit. The shooting time may be a shooting start time, or may be a shooting period (a time from the shooting start time to the shooting end time).

  Instead of this, the CPU 30 may create a file name that can distinguish the point, the time, and the microscopy, and assign it to each image. For example, the CPU 30 can generate XY ## _ T %%%% _ CH $$ _ Z &&&. Create a file named bmp. The CPU 30 extracts a point ##, a time %%%%, a channel $$, and a Z coordinate &&& from the control parameter to create a file name and assign it to a file storing an image. By adopting such a naming rule, the CPU 30 can specify the control parameter used to acquire the image from the file name, read out the image corresponding to the control parameter designated by the user from the storage device 31, and display It can be displayed on 34.

  Also, the CPU 30 may create metadata from the control parameters and assign it to the image file. The CPU 30 may create a file list database to manage the relationship between control parameters and images.

  FIG. 5 is a block diagram showing functions implemented in the control device 2 when the CPU 30 executes a control program. The control device 2 is a control device that controls the microscope 1 used for time-lapse experiments. The imaging control unit 401 transmits and controls a control signal to the microscope 1, receives an image signal transmitted from the microscope 1, creates image data, and stores the image data in the image storage unit 407. If image data has already been created by the microscope 1, the CPU 30 creates metadata, adds it to this image data, and stores it in the image storage unit 407. The imaging control unit 401 causes the microscope to perform imaging at regular intervals in order to execute time-lapse imaging. When executing the Z stack, the imaging control unit 401 controls the microscope 1 to change the distance between the objective lens and the sample 3 within a predetermined movement range including the reference position. The sample 3 is photographed. In this case, the image storage unit 407 acquires a plurality of images acquired by photographing the sample 3 while changing the distance between the objective lens and the sample 3 in the microscope 1 and light of the objective lens when acquiring each image. The position data (distance data) indicating the Z position in the axial direction is associated and stored. The data indicating the association between the image and the Z position may be converted into a database and stored in the data storage unit 403. Thus, the distance between the objective lens and the sample 3 can be expressed by the position (Z position) in the Z axis direction of the Z stage. When changing the control parameter, the imaging control unit 401 stops the light emission of the light sources 4 and 8 that output excitation light, or controls a light shielding mechanism such as a mechanical shutter to shield light. According to the present embodiment, the phototoxicity to the sample 3 is reduced or the fading of the sample 3 is alleviated as compared with the prior art that requires the illumination of the illumination light of the sample 3 when changing the parameter. it can.

The image storage unit 407 is a plurality of images obtained by shooting the sample 3 with the microscope 1 and control parameters when shooting each image, and the shooting position of the image with respect to the XY stage 6 of the microscope 1, shooting time And the control parameters including the speculum method set in the microscope 1 at the time of taking an image, in association with each other and stored. The parameter designation unit 405 functions as a reception unit that receives designation of a control parameter to be changed during the time-lapse experiment. The image selection unit 406 selects, from the plurality of images stored in the image storage unit 407, an image associated with a control parameter common to at least a part of the control parameter to be changed accepted by the parameter specification unit 405. Do. The parameter change unit 408 changes the control parameter in accordance with the operation content and the operation amount of the operation unit 33. The imaging control unit 401 controls the microscope according to the changed control parameter to cause the microscope 1 to execute the next imaging in the time-lapse experiment. The preview image creation unit 409
The preview image creation unit 409 creates, as a preview image, an image according to the control parameter changed according to the operation content and the operation amount of the operation unit 33, and displays the image on the display unit 34. The preview image creation unit 409 moves and displays the selected image in the display area of the display unit 34 according to the operation amount of the operation unit 33. For example, when the image is dragged by the operation unit 33 and moved, the image is moved and displayed according to the amount of drag and the direction of the drag. The part of the image located outside the display area is not displayed. This allows the user to visually indicate how the observation position (shooting range) changes. The parameter change unit 408 changes the shooting position according to the amount of operation of the operation unit 33. When the image is dragged by the pointing device of the operation unit 33, the imaging position is changed according to the amount of dragging. For example, if the position of the sample 3 is shifted from the center of the observation position, the image of the sample 3 is moved to the center of the display area to position the image of the sample 3 at the center of the image. ) Is changed.

  The preview image creation unit 409 may create the preview image by changing the luminance value of the selected image according to the exposure time input by the operation unit 33. In a typical microscope, the sample 3 is photographed in real time to display an image of the sample 3 by applying the changed exposure time in practice. Therefore, the sample 3 must be illuminated. On the other hand, in the present invention, by creating and displaying the preview image, the user can visually grasp what kind of image is acquired by the change of the exposure time. In addition, since irradiation of illumination light is unnecessary, phototoxicity can be reduced.

  The preview image creation unit 409 may create the preview image by changing the luminance value of the selected image according to the setting of the binning of the microscope 1 or the setting of the gain input by the operation unit 33. Binning is a method of combining a plurality of pixels in a pseudo manner into one pixel and increasing the number of light receiving elements constituting one pixel to increase a sensitivity in a pseudo manner. The gain is an amplification factor of the signal output from the light receiving element. In any case, the brightness of the image can be increased. In addition, when the change of the magnification of the objective lens is designated, the preview image generation unit 409 may enlarge the selected image according to the changed magnification to generate a preview image. In any case, the user can visually grasp what kind of image is acquired by changing the parameter. In addition, phototoxicity can be reduced.

  FIG. 6 is a diagram showing an example of the operation screen involved in the control parameter change operation. An operation screen 600 displayed on the display unit 34 is a user interface for changing control parameters (control parameters) of the microscope 1. The display area 601 displays the selected image. Below the display area 601, controls for specifying parameters are arranged. The point designation unit 602 is a control for designating acquired XY coordinates of an image. The time designation unit 603 is a control for designating the time when the image is acquired. The speculum specifying unit 604 is a control for specifying a speculum (channel) used for acquiring an image. The file selection unit 605 is a control for selecting an image file. When one of the points (for example, XY03) is designated in the point designation unit 602, the CPU 30 refers to a folder whose name is the designated point, and lists the names of subfolders existing in the folder in the time designation unit 603. To display. When one of the times (for example, T0036) is designated in the time designation unit 603, the CPU 30 refers to the folder whose name is the designated time, and the name of the subfolder existing in the folder is designated by the speculum identification unit 604. Listed and displayed. When one of the speculum methods (e.g. CH03) is designated in the speculum identification designation unit 604, the CPU 30 refers to the folder having the designated speculum name as the name of the file existing in the folder. The file selection unit 605 enumerates and displays them. In this example, the pointer 606 is manipulated by the user, and Z001. An image file named tif is selected, and the image file is rendered in the display area 601. The control parameter setting unit 607 is a control for specifying the number of shots, the shooting interval, the exposure time, the binning, the gain, the magnification, the filter, and the like. The user can move the pointer 606 through the operation unit 33 or drag and move the image displayed in the display area 601. In the present embodiment, the spot is selected first, and then the time, the microscopic examination and the file name are selected to realize the image selection, but the order of these may be different.

  FIG. 7 is a view showing an example of the operation screen involved in the control parameter changing operation. In FIG. 7, the pointer 606 indicates that the image is dragged in the lower left direction of FIG. The parameter change unit 408 converts the movement amount of the image by the pointer 606 into a change amount of XY coordinates, and calculates XY coordinates of the next shooting.

  FIG. 8 is a view showing an example of the operation screen involved in the control parameter changing operation. In FIG. 8, since the magnification of the objective lens is changed by the pointer 606, a preview image created in a pseudo manner according to the changed magnification of the preview image creation unit is displayed in the display area 601.

  FIG. 9 is a flowchart showing the control parameter changing operation.

  In step S901, the CPU 30 determines whether a change in control parameter is instructed through the operation unit 33 before time-lapse shooting or during time-lapse shooting. If it is instructed to change the control parameter, the process advances to step S902.

  In S902, the CPU 30 causes the display unit 34 to display an operation screen 600.

  In S903, the CPU 30 (parameter designation unit 405) receives designation of control parameters required to select an image. As described above, a point is designated in the point designation unit 602, a time is designated in the time designation unit 603, a speculum is designated in the speculum designation unit 604, and a file name is selected in the file selection unit 605. . In this way, control parameters required to select an image are specified.

  In step S904, the CPU 30 (image selection unit 406) traces the folder tree according to the designated point, time, and microscopy, selects and reads the file of the file name selected by the file selection unit 605, and displays the file. An image is rendered and displayed in an area 601.

  In S 905, the CPU 30 (parameter designation unit 405) receives a control parameter change operation. The change of control parameters is performed through the operation screen 600. If it is a change of a point (X and Y coordinates), it is accepted by dragging the image by the pointer 606. The control parameter setting unit 607 can accept any change operation such as the number of shots, exposure time, binning, and magnification.

  In step S906, the CPU 30 (parameter change unit 408) determines control parameters in accordance with the amount of operation of the operation unit 33. If it is XY coordinates, the parameter changing unit 408 moves the shooting position (XY coordinates) of the selected image in parallel according to the drag amount in the XY direction with respect to the selected image, thereby determining the XY coordinates of the next shooting. The XY coordinates may be realized not only by dragging but also by cursor input with a keyboard, designating the center of the next photographing position by mouse click, or directly inputting numerical values (pixel unit / micron unit). . The Z coordinate may only be changed if the Z stack is selected. In this case, the parameter changing unit 408 sets the Z position when the selected image is acquired as the shooting reference position in the Z direction at the next shooting. For example, when the user selects an image that is most in focus with respect to the sample 3, it will be possible to appropriately determine the imaging reference position in the Z direction of the next imaging. Alternatively, the focusing determination unit 404 may determine the focusing state (a focusing value such as a contrast value) of the captured image. The parameter changing unit 408 determines the Z position of the best in-focus image as the next photographing reference position.

  The parameter change unit 408 may change control parameters related to the brightness of the image. The parameter designation unit 405 reads control parameters such as exposure time, gain, and binning associated with the selected image, and displays the read control parameters on the control parameter setting unit 607 of the operation screen 600. The user changes these parameters through the operation unit 33. The parameter change unit 408 receives the change of the parameter through the operation unit 33, the parameter specification unit 405, and the control parameter setting unit 607. The preview image creation unit 409 creates a preview image by changing the luminance value of the image according to the control parameter related to the brightness changed by the user, and displays the preview image in the display area 601. It is assumed that the exposure time of the acquired image is 100 ms and the post-modification exposure time is 120 ms. In this case, the preview image generation unit 409 changes each pixel value of the selected image to 1.2 times. Also, the preview image creation unit 409 creates a preview image in which the brightness is changed by image processing in the same way for binning and gain.

  Further, the change of the imaging interval after the next time may be accepted through the operation screen 600. For example, the imaging interval may be changed from 15 minutes to 30 minutes by the pull-down menu.

When the CPU 30 (parameter changing unit 408) detects that the enter button 610 shown in FIG. 6 is pressed by the pointer 606, the amount of change at that point is determined, and the control parameter calculation unit Calculated The control parameter at the time of the next photographing is determined as follows, for example, from the control parameter of the selected image and the amount of change thereof.
● (X and Y coordinates at next shooting) = (X and Y coordinates of selected image) + (Parallel movement of image) * (Length per image pixel [um / pixel])
● (Z stack reference position at next shooting) = (Z position of selected image)
● (Camera parameter at next shooting) = (Indicated value on the operation screen 600)
In S 907, the CPU 30 (parameter change unit 408) sets the determined control parameter in the microscope 1 through the imaging control unit 401. In step S908, the CPU 30 (shooting control unit 401) monitors a timer to determine whether the next shooting timing has arrived. When the next imaging timing comes, the process advances to step S909. In step S909, the CPU 30 (the imaging control unit 401) instructs the microscope 1 to execute imaging.

  The CPU 30 may read the control parameter of the selected image from the metadata of the image file, or may read it from the database stored in the data storage unit 403.

  The present invention may be applied to multipoint imaging in which a plurality of images are acquired while changing XY coordinates. The CPU 30 allows the user to select whether to change the control parameters for the remaining other points when changing the control parameter for one of the multiple points for which the multipoint shooting is performed. Good. When it is selected to change the control parameter for another point, the CPU 30 allows the user to select another point for which the control parameter is to be changed. The subsequent processing is as described above. When the change of the control parameter is completed at all points where the control parameter is to be changed, a complete button (not shown) may be pressed. After this, the next imaging timing is waited, and imaging is performed with the control parameter after the change from the next time.

[Z stack]
In this embodiment, in time-lapse shooting with Z-stacking, the in-focus position is determined from the Z-stack image acquired at the past imaging timing, and the in-focus position is referred to as the imaging reference position (hereinafter simply referred to as reference position). Shooting may be performed. The reference position is a position serving as a reference for determining the movement range W of the Z stage in the Z stack. For example, the upper limit position Zhi and the lower limit position Zlo of the movement range W are determined such that the reference position is at the center position in the movement range W. That is, the movement range W is a range from the upper limit position Zhi to the lower limit position Zlo. The movement range W is the movement distance of the Z stage in one Z stack. Assuming that this movement distance is 2L, the Z position that is + L from the reference position is the start position (or end position) of the Z stack, and the Z position that is −L from the reference position is the end position (or start position) of the Z stack It becomes. Of course, this is only an example, and if the movement direction of the sample 3 can be predicted, the movement range of the Z stage may be determined so as to cover the movement direction. For example, if it is known that the sample 3 moves (grows) in the + direction from the reference position, the reference position may be determined as the start position.

  The focus determination unit 404 may determine the image in which the sample 3 is most in focus, or the user may determine the image by visual observation. On the operation screen 600, the user causes the display area 601 to sequentially display a plurality of images constituting the Z stack image. For example, it is assumed that there are five images (Z0001.tif to Z0005.tif) acquired at the (n-1) th photographing timing. The user uses the file selection unit 605 to execute Z0001. tif to Z0005. Images acquired at different Z positions up to tif are sequentially selected. Z0001. tif to Z0005. The tifs are displayed in order, and the user visually determines the in-focus state. Thereby, the user visually selects an image which is most in focus on the sample 3. Here, Z0002. Suppose tif is selected. The CPU 30 executes Z0002. The Z position at the time of shooting is read out from the metadata of tif and set as the reference position Zr. The setting unit 402 determines the upper limit position Zhi and the lower limit position Zlo of the Z stage used at the n-th imaging timing based on the reference position Zr. As described above, the upper limit position Zhi and the lower limit position Zlo may be set around the reference position Zr. In addition, for an asymmetric observation target such as an observation target extending upward from the in-focus position, the distance from the reference position Zr to the upper limit position Zhi and the distance from the reference position Zr to the lower limit position Zlo are set asymmetrically. You may For example, when the user acquires five images on the upper side from the reference position Zr and sets two images on the lower side of the reference position Zr, the setting unit 402 sets 5 pitches to the reference position Zr. The added Z position is determined as the upper limit position Zhi, and the Z position obtained by subtracting 2 pitches from the reference position Zr is determined as the lower limit position Zlo. In this case, imaging may be performed also at the reference position Zr. Thereby, an asymmetric Z stack setting is realized. The setting unit 402 sets the determined movement range W (upper limit position Zhi and lower limit position Zlo) to the microscope 1 through the imaging control unit 401.

  After moving the Z stage to the reference position, the control unit 20 and the CPU 30 execute the focusing operation (auto focus) to search for the in-focus position, and shift the movement range so that the found in-focus position is at the center. May be

As described above, by tracking the in-focus position using the Z stack image acquired in the past or immediately before, it is possible to eliminate the need for a separate focusing operation or to reduce the time required for the focusing operation. As a result, the time for which the sample 3 is irradiated with light is also shortened, so that phototoxicity can be reduced.
By narrowing, the number of images constituting the Z stack image is reduced.

  As described above, in the microscope system according to the present embodiment, a plurality of images obtained by photographing the sample 3 with the microscope 1 are stored in association with control parameters when the respective images are photographed. The control parameters may include the imaging position of the image with respect to the XY stage 6, the imaging time, and the speculum method set in the microscope when imaging the image. The microscope system accepts specification of control parameters for selecting images during time-lapse experiments. For example, at least one of a point, a time, a speculum, etc. is designated by the user. The displayed image is an image that is useful for changing control parameters. The microscope system selects an image associated with the received control parameter from the plurality of images stored in the image storage unit 407. For example, in the example shown in FIG. 6, an image (e.g., Z0001.tif) associated with a point, a time, and a speculum is selected. The display unit 34 displays the selected image. Thus, the user visually confirms the image of the sample 3. Note that this image is an image acquired in the past, not a real-time image acquired in real time. A user who is an operator operates an operation unit 33 such as a pointing device, a keyboard, or a console. The microscope system changes at least a part of control parameters in accordance with the operation amount and the operation content of the operation unit 33. That is, the user operates the operation unit 33 while checking the displayed image, and inputs an instruction to change the control parameter. The microscope system controls the microscope 1 according to the changed control parameters to cause the microscope 1 to perform the next imaging in the time-lapse experiment. As described above, in the present embodiment, since the past image can be displayed and the control parameter can be changed without capturing the image in real time, the influence of the phototoxicity on the sample can be reduced. In addition, since the control parameter can be changed while confirming the sample 3 in the past image, the control parameter can be set more accurately, and the success rate of the time-lapse imaging can be increased.

  The display unit 34 of the microscope system may display an image according to the control parameter changed according to the operation amount of the operation unit 33 or the operation content as a preview image. This allows the user to visually confirm how control parameter changes are reflected. Since the preview image is created based on the past image, it is not necessary to apply illumination light to the sample 3 to acquire a new image. Thus, the effects of phototoxicity can be reduced.

  The display unit 34 of the microscope system may move and display the selected image in the display area 601 according to the operation amount of the operation unit 33. For example, the image is moved within the display area 601 by the drag operation of the pointer 606. The microscope system changes the imaging position among the control parameters in accordance with the operation amount of the operation unit 33. For example, the user can move the image so that the image of the sample 3 is at the center of the observation field (display area 601). Here, since the movement amount of the image in the display area 601 is proportional to the movement amount of the imaging position, conversion is possible by a simple calculation.

  The microscope system may create and display a preview image by changing the luminance value of the selected image according to the exposure time input by the operation unit. Conventionally, when the user changes the exposure time, the influence of the change in the exposure time is confirmed by acquiring and displaying an image by reflecting it on the microscope 1 in real time. On the other hand, in the present invention, since the preview image is generated by image processing, it is not necessary to acquire an image in real time. In other words, the effects of phototoxicity can be reduced. Also, the user can confirm the influence of the change of the exposure time while confirming the preview image.

  The microscope system may create the preview image by changing the luminance value of the selected image according to the setting of the binning of the microscope or the setting of the gain input by the operation means. As described above, with regard to control parameters related to the brightness of the image, the user can confirm the influence of the change by the preview image without acquiring the real-time image, and the influence of phototoxicity can be reduced.

  The microscope system may enlarge the selected image according to the changed magnification to create a preview image when a change in magnification of the objective lens is designated. As for control parameters related to magnification in this manner, the preview image allows the user to confirm the influence of the change without acquiring a real-time image, thereby reducing the influence of phototoxicity. This embodiment is applicable not only to the enlargement of the magnification but also to the reduction.

  Note that the microscope 1 may be a microscope that executes a Z-stack that captures a plurality of images while changing the distance between the objective lens and the sample 3. In this case, the control parameter includes the position in the Z direction that indicates the distance between the objective lens and the sample 3 when each of the plurality of images is captured. Therefore, the microscope system may set the position in the Z direction associated with the selected image as the reference position in the Z direction of the image to be captured next time. As a result, the Z position can also be changed without acquiring a real time image, thereby reducing the influence of phototoxicity. In particular, with regard to the sample 3 grown in the Z-axis direction, by correcting the Z position with the passage of time, it will be possible to obtain an image in focus at the location of interest. That is, the success probability of time-lapse shooting increases.

  The microscope 1 may be a fluorescence microscope which irradiates excitation light to the sample 3 to which the fluorescent reagent is added, and photographs fluorescence emitted from the sample 3. When changing the control parameter, the imaging control unit 401 may stop the light source that outputs the excitation light, or may shield the excitation light with a mechanical shutter or the like. This will certainly reduce phototoxicity.

  In the microscope system of the present embodiment, the microscope 1 and the control device 2 are separated, but may of course be integrated. Further, although the distance between the objective lens and the sample 3 is changed by moving the objective lens unit 12, the stage on which the sample 3 is mounted may be moved in the Z-axis direction. The microscope may be inverted or upright.

Claims (11)

  1. A control device for controlling a microscope used in a time-lapse experiment in which a sample is irradiated with excitation light at regular intervals to photograph the sample ,
    A plurality of images obtained by capturing a sample with a microscope and control parameters for capturing each image, the capturing position of the image with respect to the XY stage of the microscope, capturing time, and capturing the image Storage means for correlating and storing control parameters including any of the speculum methods set in the microscope;
    Accepting means for accepting designation of a control parameter by an operator for selecting an image during the time-lapse experiment;
    Selection means for selecting an image associated with the control parameter received by the reception means from the plurality of images stored in the storage means;
    Display means for displaying the image selected by the selection means;
    And operating means operable by the operator,
    Changing means for changing the imaging position of the sample, which is the control parameter , according to the operation content of the operation means;
    Imaging control means for moving the XY stage so that the sample moves to the imaging position after change by the changing means , irradiating the excitation light to the sample, and causing the microscope to execute the next imaging in the time-lapse experiment And a controller.
  2.   The control device according to claim 1, wherein the display unit displays an image according to a control parameter changed according to an operation amount of the operation unit as a preview image.
  3. The display means moves and displays the image selected by the selection means in the display area of the display means according to the operation amount of the operation means.
    The control device according to claim 2, wherein the change unit changes the imaging position among the control parameters according to the operation amount.
  4.   4. The image forming apparatus according to claim 2, further comprising: a creation unit configured to create the preview image by changing the luminance value of the image selected by the selection unit according to the exposure time input by the operation unit. Control device.
  5.   The image forming apparatus further includes creation means for creating the preview image by changing the luminance value of the image selected by the selection means according to the setting of the binning of the microscope or the setting of the gain input by the operation means. The control device according to claim 2 or 3.
  6.   The image forming apparatus further comprises creation means for creating the preview image by enlarging the image selected by the selection means according to the post-change magnification when change of magnification of the objective lens of the microscope is designated. Item 4. A control device according to item 2 or 3.
  7. The microscope is a microscope that performs Z stack imaging that captures a plurality of images while changing the distance between the objective lens of the microscope and the sample.
    The control parameter includes a position in the Z direction indicating a distance between the objective lens and the sample when each of the plurality of images is captured.
    7. The apparatus according to claim 1, wherein the changing unit sets the position in the Z direction associated with the image selected by the selecting unit as a reference position in the Z direction of an image to be captured next time. The control device according to any one of the preceding claims
  8. The microscope is a fluorescence microscope which irradiates excitation light to the sample to which a fluorescent reagent is added, and photographs fluorescence emitted from the sample.
    The control device according to any one of claims 1 to 7, wherein the imaging control means stops the light source outputting the excitation light when changing the control parameter.
  9. A microscope system comprising a microscope and a controller for controlling the microscope,
    The microscope is
    A placement means on which the sample is placed;
    An objective lens,
    Moving means for moving the objective lens to change the distance between the objective lens and the sample;
    An imaging lens for imaging light from the sample;
    And imaging means for imaging light from the sample.
    The controller is
    A plurality of images obtained by capturing a sample with a microscope and control parameters for capturing each image, the capturing position of the image with respect to the XY stage of the microscope, capturing time, and capturing the image Storage means for correlating and storing control parameters including any of the speculum methods set in the microscope;
    Accepting means for accepting designation of a control parameter by an operator for selecting an image during time-lapse experiments in which excitation light is irradiated to a sample at predetermined intervals and the sample is photographed ;
    Selection means for selecting an image associated with the control parameter received by the reception means from the plurality of images stored in the storage means;
    Display means for displaying the image selected by the selection means;
    And operating means operable by the operator,
    Changing means for changing the imaging position of the sample, which is the control parameter , according to the operation content of the operation means;
    Imaging control means for moving the XY stage so that the sample moves to the imaging position after change by the changing means , irradiating the excitation light to the sample, and causing the microscope to execute the next imaging in the time-lapse experiment And a microscope system characterized by having:
  10. A control device for controlling a microscope used in a time-lapse experiment in which a sample is irradiated with excitation light at regular intervals to photograph the sample ;
    A plurality of images obtained by capturing a sample with a microscope and control parameters for capturing each image, the capturing position of the image with respect to the XY stage of the microscope, capturing time, and capturing the image Storage means for correlating and storing control parameters including any of the speculum methods set in the microscope;
    Accepting means for accepting designation of a control parameter by an operator for selecting an image during the time-lapse experiment;
    Selection means for selecting an image associated with the control parameter received by the reception means from the plurality of images stored in the storage means;
    Display means for displaying the image selected by the selection means;
    And operating means operable by the operator,
    Changing means for changing the imaging position of the sample, which is the control parameter , according to the operation content of the operation means;
    Imaging control means for moving the XY stage so that the sample moves to the imaging position after change by the changing means , irradiating the excitation light to the sample, and causing the microscope to execute the next imaging in the time-lapse experiment A program characterized by acting as
  11. A control method for controlling a microscope used in a time-lapse experiment in which a sample is irradiated with excitation light at regular intervals to photograph the sample ,
    A plurality of images obtained by capturing a sample with a microscope and control parameters for capturing each image, the capturing position of the image with respect to the XY stage of the microscope, capturing time, and capturing the image A storage step of storing in a storage means in association with a control parameter including any of the speculum methods set in the microscope.
    An accepting step of accepting designation of a control parameter by an operator for selecting an image during the time-lapse experiment;
    A selection step of selecting an image associated with the control parameter received in the reception step from the plurality of images stored in the storage means;
    A display step of displaying the image selected in the selection step on display means;
    A changing step of changing a photographing position of the specimen which is the control parameter in response to operation contents of the possible operating means operated by said operator,
    An imaging control step of moving the XY stage so that the sample moves to the imaging position after the change in the changing step , irradiating the excitation light to the sample, and causing the microscope to execute the next imaging in the time-lapse experiment And a control method.
JP2013221498A 2013-10-24 2013-10-24 Controller, microscope system, control method and program Active JP6246551B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2013221498A JP6246551B2 (en) 2013-10-24 2013-10-24 Controller, microscope system, control method and program

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2013221498A JP6246551B2 (en) 2013-10-24 2013-10-24 Controller, microscope system, control method and program

Publications (2)

Publication Number Publication Date
JP2015082097A JP2015082097A (en) 2015-04-27
JP6246551B2 true JP6246551B2 (en) 2017-12-13

Family

ID=53012695

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2013221498A Active JP6246551B2 (en) 2013-10-24 2013-10-24 Controller, microscope system, control method and program

Country Status (1)

Country Link
JP (1) JP6246551B2 (en)

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7372984B2 (en) * 2004-05-05 2008-05-13 California Institute Of Technology Four-dimensional imaging of periodically moving objects via post-acquisition synchronization of nongated slice-sequences
JP4928081B2 (en) * 2005-02-03 2012-05-09 オリンパス株式会社 Microscope imaging apparatus, recording control method, and recording control program
JP5058444B2 (en) * 2005-02-10 2012-10-24 オリンパス株式会社 Micrograph apparatus and micrograph apparatus control method
JP2006259701A (en) * 2005-02-16 2006-09-28 Nikon Corp Intermittent photographing device and microscope system having same
JP4970869B2 (en) * 2005-09-12 2012-07-11 オリンパス株式会社 Observation apparatus and observation method
JP4996319B2 (en) * 2007-04-26 2012-08-08 オリンパス株式会社 Laser microscope and its image display method
JP2009008846A (en) * 2007-06-27 2009-01-15 Nikon Corp Browsing apparatus for multidimensional image
DE102010041794A1 (en) * 2010-09-30 2012-04-05 Carl Zeiss Microlmaging Gmbh Microscope system, microscopy method and computer program product
JP5803442B2 (en) * 2011-08-31 2015-11-04 株式会社ニコン Microscope control apparatus, microscope apparatus, image processing apparatus, and program

Also Published As

Publication number Publication date
JP2015082097A (en) 2015-04-27

Similar Documents

Publication Publication Date Title
EP2282230B1 (en) Fully automatic rapid microscope slide scanner
US10001622B2 (en) Multifunction autofocus system and method for automated microscopy
US20060204072A1 (en) System for creating microscopic digital montage images
US7932504B2 (en) Microscope system and VS image production and program thereof
US20100141752A1 (en) Microscope System, Specimen Observing Method, and Computer Program Product
US7253420B2 (en) Scanning microscope system
JP6186634B2 (en) System equipped with digital microscope and sample inspection method using the same
Edelstein et al. Advanced methods of microscope control using μManager software
JP5643210B2 (en) Whole slide fluorescent scanner
US7978898B2 (en) Image acquiring apparatus, image acquiring method, and image acquiring program
EP2519855B1 (en) Sensor for microscopy
Yazdanfar et al. Simple and robust image-based autofocusing for digital microscopy
JP5875812B2 (en) Microscope system and illumination intensity adjustment method
JP2008139488A (en) Microscope device
US8000560B2 (en) Virtual slide generation device, virtual slide generation method, virtual slide generation program product and virtual slide generation program transmission medium
RU2522123C2 (en) System and method for enhanced predictive autofocusing
US20070058054A1 (en) Observation apparatus
JP2007310231A (en) Image acquiring device, image acquiring method and image acquiring program
JP4878913B2 (en) Microscope system, microscope image synthesis method, and program
US7486886B2 (en) Photo-micrographing device and its control method
JP5577885B2 (en) Microscope and focusing method
EP1762878B1 (en) Observation apparatus and observation method
JP2017194699A5 (en)
JP4936867B2 (en) Enlarged image observation apparatus and enlarged image observation method
CN102854615B (en) Full-automatic scanning system and method for microscopic section

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20160909

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20170714

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20170807

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20171005

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20171027

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20171115

R150 Certificate of patent or registration of utility model

Ref document number: 6246551

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150